WO2016201145A1 - Data-charge phase data compression architecture - Google Patents

Data-charge phase data compression architecture Download PDF

Info

Publication number
WO2016201145A1
WO2016201145A1 PCT/US2016/036767 US2016036767W WO2016201145A1 WO 2016201145 A1 WO2016201145 A1 WO 2016201145A1 US 2016036767 W US2016036767 W US 2016036767W WO 2016201145 A1 WO2016201145 A1 WO 2016201145A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
computing
block
compressed bitstream
image data
Prior art date
Application number
PCT/US2016/036767
Other languages
French (fr)
Inventor
Hye-Yeon Cheong
Masaru Ikeda
Ali Tabatabai
Original Assignee
Sony Corporation
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sony Corporation filed Critical Sony Corporation
Priority to EP16808320.2A priority Critical patent/EP3295670A4/en
Priority to JP2017564411A priority patent/JP6614420B2/en
Priority to CN201680032410.1A priority patent/CN107637079B/en
Publication of WO2016201145A1 publication Critical patent/WO2016201145A1/en

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/124Quantisation
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/103Selection of coding mode or of prediction mode
    • H04N19/11Selection of coding mode or of prediction mode among a plurality of spatial predictive coding modes
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/129Scanning of coding units, e.g. zig-zag scan of transform coefficients or flexible macroblock ordering [FMO]
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/13Adaptive entropy coding, e.g. adaptive variable length coding [AVLC] or context adaptive binary arithmetic coding [CABAC]
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/146Data rate or code amount at the encoder output
    • H04N19/15Data rate or code amount at the encoder output by monitoring actual compressed data size at the memory before deciding storage at the transmission buffer
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/157Assigned coding mode, i.e. the coding mode being predefined or preselected to be further used for selection of another element or parameter
    • H04N19/159Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/172Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a picture, frame or field
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/176Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/46Embedding additional information in the video signal during the compression process
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/59Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/593Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving spatial prediction techniques
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/189Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the adaptation method, adaptation tool or adaptation type used for the adaptive coding
    • H04N19/192Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the adaptation method, adaptation tool or adaptation type used for the adaptive coding the adaptation method, adaptation tool or adaptation type being iterative or recursive

Definitions

  • Image sensors are widely used in devices such as a digital camera to produce digital images.
  • An analog-to-digital converter (ADC) is used in an image sensor to convert analog signals into digital signals. More specifically, an ADC converts an analog pixel signal from a pixel array into digital data. Compressing techniques may be used to compress the digital data representing an image, as well as to preserve the quality of the image.
  • Implementations generally relate to data-charge phase data compression and related to compression of image pixel data that is captured by an area analogue-to-digital (AD) sensor based on a predetermined output area AD coding or capturing order.
  • an apparatus includes one or more processors, and includes logic encoded in one or more tangible media for execution by the one or more processors and when executed operable to perform operations including: computing prediction values for image data, where the image data is data-charge phase data, and where the computing of prediction values is based on inter-block prediction; computing residual data based on the prediction values; quantizing the residual data; entropy encoding the quantized residual data; and refining an inverse quantized residual data based on one or more of the residual data and a number of left-over bit-budget after entropy encoding.
  • computing prediction values for image data where the image data is data-charge phase data, and where the computing of prediction values is based on inter-block prediction
  • computing residual data based on the prediction values
  • quantizing the residual data
  • software is encoded in one or more non-transitory computer-readable media for execution by the one or more processors.
  • the software When executed, the software is operable to perform operations including: computing prediction values for image data, where the image data is data-charge phase data, and where the computing of prediction values is based on inter-block prediction; computing residual data based on the prediction values; quantizing the residual data; entropy encoding the quantized residual data; and refining an inverse quantized residual data based on one or more of the residual data and a number of left-over bit-budget after entropy encoding.
  • a method includes computing prediction values for image data, where the image data is data-charge phase data, and where the computing of prediction values is based on inter-block prediction.
  • the method also includes computing residual data based on the prediction values.
  • the method also includes quantizing the residual data.
  • the method also includes entropy encoding the quantized residual data.
  • the method also includes refining an inverse quantized residual data based on one or more of the residual data and a number of left-over bit-budget after entropy encoding.
  • FIG. 1 illustrates a block diagram of an area analogue-to-digital converter (ADC), which may be used for implementations described herein.
  • ADC area analogue-to-digital converter
  • FIG. 2 illustrates an example graph diagram showing data from a correlated double sampling (CDS) based operation, according to some implementations.
  • CDS correlated double sampling
  • FIG. 3 is a block diagram of an example data compressor, according to some implementations .
  • FIG. 4 illustrates an flow diagram of an example method for compressing image data, according to some implementations.
  • FIG. 5 illustrates an example bitstream structure, according to some embodiments
  • FIG. 6A illustrates a block diagram of example D-phase data, according to some implementations .
  • FIGS. 6B-6E illustrate block diagrams of example sub-sample coding blocks, according to some implementations.
  • FIG. 7A illustrates a block diagram of an example input picture, according to some implementations.
  • FIGS. 7B and 7C illustrate block diagrams of example sub-pictures, according to some implementations.
  • FIG. 8A illustrates a block diagram of an example input picture, according to some implementations.
  • FIGS. 8B and 8C illustrate block diagrams of example sub-pictures, according to some implementations.
  • FIG. 9 illustrates a block diagram of an example input picture, where the coding order is a raster order, according to some implementations.
  • FIG. 10 illustrates a block diagram of an example input picture, where the coding order is a zigzag order, according to some implementations.
  • FIG. 11 illustrates a block diagram of an example input picture, according to some implementations.
  • FIG. 12 illustrates a block diagram of an example computing system, which may be used for implementations described herein.
  • Implementations generally relate to data-charge phase data compression.
  • implementations employ inter-block prediction based techniques for data compression of data-charge phase (D-phase) data such as that of an area analogue-to-digital (AD) sensor.
  • D-phase data-charge phase
  • AD area analogue-to-digital
  • inter-block prediction involves dependency between blocks.
  • Some implementations may also employ intra-block prediction.
  • an apparatus computes prediction values for image data, where the image data is data-charge phase data, and where the computing of prediction values is based on inter-block prediction.
  • the apparatus also computes residual data based on the prediction values.
  • the apparatus also quantizes the residual data.
  • the apparatus also entropy encodes the quantized residual data.
  • the apparatus also refines an inverse quantized residual data based on one or more of the residual data and a number of left-over bit-budget after entropy encoding. As a result, both subjective and objective performance improves compared to intra-prediction based techniques.
  • FIG. 1 illustrates a block diagram of an area analogue-to-digital converter (ADC) 100, which may be used for implementations described herein.
  • Area ADC 100 includes areas or blocks 102 of pixels, which generate pre-charge phase (P-phase) data and data-charge phase (P-phase) data.
  • area ADC 100 performs parallel processing in the blocks 102.
  • blocks 102 are shown as squares. In various implementations, blocks 102 may be any shape such as rectangles, circles, etc.
  • the settling time for area ADC 100 may be significantly reduced, where the settling time may be the time for the sensor to reach a stable output once it is turned on. Such reduced settling times is beneficial for high-speed cameras.
  • FIG. 2 illustrates an example graph diagram 200 showing data from a correlated double sampling (CDS) based operation, according to some implementations.
  • CDS-based operations involve both P-phase data and D-phase data.
  • Area ADC 100 performs CDS as a noise reduction technique, in which CDS data is generated from the difference between the D-phase data and the P-phase data.
  • the CDS data reflects the canceling of noise in an image.
  • area ADC 100 stores P-phase data, because the timing to obtain the P-phase data and the D-phase data is different. For example, area ADC 100 obtains and stores the P-phase data. When area ADC 100 obtains the D-phase data, ADC 100 may then fetch the P-phase data, and perform CDS in order to cancel noise. As described in more detail below, P-phase compression is desirable and is therefore performed.
  • FIG. 3 is a block diagram of an example data compressor 300, according to some implementations.
  • data compressor 300 may be a part of area ADC of FIG. 1.
  • data compressor 300 includes a prediction unit 302, a quantizer 304, an entropy encoder 306, and a refinement unit 308.
  • area ADC 100 may operate in two modes.
  • the first mode and simpler of the two modes, is referred to as pulse-code modulation (PCM).
  • the second mode is referred to as differential pulse-code modulation (DPCM).
  • DPCM differential pulse-code modulation
  • a 1-bit signal (e.g., a 0 or 1 bit) indicates which mode to apply.
  • area ADC 100 sends each bit-plane in an order from the most significant bit (MSB) to least significant bit (LSB), until all of the bit-budget is used.
  • area ADC 100 may perform middle point reconstruction of un-coded bits.
  • PCM mode may be applied to a random access (RA) pixel array, where each block is randomly accessed. In other words, each block may be accessed independently, and independently encoded and/or decoded, where there are no dependencies on surrounding blocks.
  • RA random access
  • a given pixel block is an 8x1 block, or 8x1 pixels.
  • the pixel block may be any size (e.g., 4x4, etc.).
  • the block is 8x1 pixels.
  • Each pixel is 12 bits and is to be compressed to 6 bits, but may be compressed to any other number of bits (e.g., 4 bits, etc.), depending on the particular implementation.
  • area ADC 100 first computes the bit budget. With a 48-bit budget, one bit is used for indicating PCM mode. As such, in this example, there remain 47 bits. Area ADC 100 then removes the header bit. Area ADC 100 then codes each block from MSB to LSB until the bit budget is used up. In some implementations, area ADC 100 then performs middle-point reconstruction in order to minimize errors.
  • FIG. 4 illustrates an flow diagram of an example method for compressing image data, according to some implementations.
  • the method is initiated in block 402, where data compressor 300 computes prediction values for image data.
  • the image data is data-charge phase data.
  • the image data is partitioned into blocks, and data compressor 300 computes the prediction values for each block of the image data, and computes prediction values based on inter-block prediction. In other words, data compressor 300 computes the prediction values based on comparing samples of a first block to samples of a second block.
  • data compressor 300 computes the prediction values based on comparing samples of a first block to samples of a second block.
  • inter-block prediction has an advantage of higher performance and may have a tradeoff of requiring larger random access units.
  • data compressor 300 may computes prediction values based on intra-block prediction.
  • intra-block prediction has an advantage of requiring smaller random access units and may have a tradeoff of lower performance.
  • the computing of prediction values is based on sample data in a same position in a different block.
  • the computing of prediction values may be based on sample data in a same position in a previous block.
  • data compressor 300 computes residual data based on the prediction values.
  • data compressor 300 quantizes the residual data.
  • data compressor 300 quantizes the residual image data based on a quantization value.
  • data compressor 300 determines the quantization parameter (qn) at the mode decision process sent in the bitstream. In some implementations, there are two possible techniques, which are described in more detail below.
  • data compressor 300 selects a predetermined quantization value (qn).
  • the quantization value may be based on a predetermined bit-depth.
  • data compressor 300 may select the quantization value from a set of quantization values ranging from a predetermined number (e.g., 0, 1, 2, etc.) to a predetermined bit-depth. For example, if the
  • the quantization value can be 0 to 11 (12 possible quantization values).
  • Data compressor 300 tests for different quantization values.
  • data compressor 300 selects the quantization value from one or more predetermined quantization values. Such quantization values may change depending on the particular implementation.
  • the mode decision may compare these 2 predetermined quantization values. As such, this technique simplifies the mode decision process. Also, only 1 bit is needed to signal the quantization value in the header.
  • data compressor 300 entropy encodes the quantized residual data.
  • data compressor 300 performs entropy encoding based on Golomb coding (e.g., exponential-Golomb coding, progressive-Golomb coding, etc.). Other entropy coding may be used, depending on the particular implementation.
  • data compressor 300 refines an inverse of the quantized residual data.
  • data compressor 300 refines the inverse quantized residual data based on one or more of the residual data and a number of left-over bit- budget after entropy encoding.
  • FIG. 5 illustrates an example bitstream structure 500, according to some implementations.
  • bitstream structure 500 is used for DPCM mode, where various values described above are sent in the bitstream in bitstream structure 500.
  • a mode field 502 (labeled "Mode") stores a 1-bit signal (e.g., 0 or 1) that indicates whether to use PCM or DPCM.
  • a quantization field 504 (labeled "Qn") stores a quantization value.
  • An entropy encoding field 506 stores entropy encoding values.
  • a residual field 508 (labeled "Residual”) stores residual data.
  • bitstream structure 500 is based on sub-sampled image data. Also, scan order may be variable, and a uniform scan is desirable between all areas.
  • data compressor 300 is described as performing the steps as described in the implementations herein, any suitable component or combination of components of area ADC 100 or any suitable processor or processors associated with area ADC 100, and/or data compressor 300, and/or any suitable system may perform the steps described.
  • FIG. 6A illustrates a block diagram of example D-phase data 600, according to some implementations.
  • D-phase data 600 is D-phase image data from an area ADC such as area ADC 100.
  • D-phase data 600 is basically the same as a raw Bayer image. Unlike P-phase data, D-phase data has a much higher dynamic range and there exists stronger spatial correlation.
  • the primary differences of D-phase data 600 from raw Bayer include the following.
  • Each block does not consist of consecutively neighboring samples of Bayer image. Also, as described in more detail below, each sample of a block belongs to different area. For example, an area the size may be 4x4, and the block size may be 4x2.
  • D-phase data 600 is partitioned into 4x4 areas 602, 604, 606, 608, 610, 612, 614, 616, 618, 620, 622, 624, 626, 628, 630, and 632 (indicated with bold lines).
  • the size of each area is 4x4.
  • Other sizes are possible (e.g., 6x6, 8x8, 10x10, etc.), depending on the particular implementation.
  • Each 4x4 area includes a coding block such as coding block 640 (indicated with dotted lines).
  • the coding blocks are 4x2.
  • Coding blocks of 4x2 are typical for Bayer image coding. Other sizes are possible (e.g., 4x4, 4x3, 4x1, etc.), depending on the particular implementation.
  • Each coding block has a predetermined number of samples. In this example, there are 8 samples. The particular number of samples per coding block will depend on the particular implementation.
  • Each coding block has associated sample positions.
  • coding block 640 has sample positions Al, A2, A3, A4, A5, A6, A7, and A8.
  • data compressor 100 separates D-phase data 600 into different sub-sample coding blocks based on the location of each sample in the original coding blocks.
  • the sub-sample coding blocks are 4x2.
  • Other sizes are possible (e.g., 4x4, 4x3, 4x1, etc.), depending on the particular implementation.
  • FIGS. 6B-6E illustrate block diagrams of example sub-sample coding blocks 650, 660, 670, and 680, according to some implementations.
  • sub-sample coding block 650 includes the samples corresponding to Al, Bl, CI, Dl, El, Fl, Gl, and HI. Because each sample of a given sub-sample block belongs to different areas of input picture 600, there is less correlation. Even though there is less correlation, data compressor 300 may still compress the D-phase data based on inter-block predictions.
  • the computing of prediction values is based on sample data in a same position in a different coding block.
  • the computing of prediction values may be based on sample data in a same position in a previous block.
  • sub-sample coding block 660 includes the samples corresponding to A2, B2, C2, D2, E2, F2, G2, and H2.
  • sub-sample coding block 670 includes the samples corresponding to
  • sub-sample coding block 680 includes the samples corresponding to
  • data compressor 300 encodes all of the pixels in a given sub-sample block independently. For example, data compressor 300 may encode all of the pixels in sub-sample block 650, then sub-sample block 660, then sub-sample block 670, and then sub-sample block 680. Then actual order may vary and will depend on the particular implementation.
  • Area ADC coding block consists of sub-sampled data, spatial correlation is much less within a block compared to conventional coding block with consecutive samples. Thus, this makes the DPCM mode less effective if intra-prediction (samples predicted from samples within the same block) is used.
  • implementations described herein support both PCM mode and DPCM mode. Implementations enable access to neighboring blocks for the prediction for DPCM mode. In some implementations, some of these techniques may require extra memory to store other blocks. Also, a random access unit may become bigger.
  • FIG. 7A illustrates a block diagram of an example input picture 700, according to some implementations.
  • Input picture 700 includes D-phase data.
  • input picture 700 has a picture size of 36x24, an area ADC size of 3x3, and an embedded block coding (EBC) block size of 4x1.
  • EBC embedded block coding
  • there is a single color format (not R, Gr, Gb, B colors to simplify the illustration).
  • FIGS. 7B and 7C illustrate block diagrams of example sub-pictures 710 and 720, according to some implementations.
  • Each sub-picture 710 and 720 has 24 blocks having a block size of 4x1.
  • the samples each of sub-picture 710 and 720 are based on the positions in input picture 700
  • FIG. 8A illustrates a block diagram of an example input picture 800, according to some implementations.
  • input picture 800 is the same as input picture 700 of FIG. 7A.
  • FIGS. 8B and 8C illustrate block diagrams of example sub-pictures 810 and 820, according to some implementations.
  • sub-pictures 810 and 820 are the same as sub-pictures 710 and 720 of FIG. 7B.
  • sub-picture 830 Also shown is a sub-picture 830.
  • the predictor of a current sample is the sample at the same position in previous block or sub-picture.
  • inter-block prediction is used, where the predictor is from a different block.
  • sub-picture 720 is used to predict sub-picture 710.
  • sub-picture 720 is used to predict sub-picture 730.
  • extra memory may be used to store sub-pictures.
  • the coding order may be fixed, block, or frame level adaptive. Various coding orders are possible depending on the particular implementations .
  • FIG. 9 illustrates a block diagram of an example input picture 900 where the coding order is a raster order, according to some implementations.
  • the coding order is a raster order.
  • the random access unit size is smaller (indicated with a bold rectangle).
  • sub-picture 0, sub-picture 3, sub-picture 6 are all intra-prediction coded (yellow samples).
  • FIG. 10 illustrates a block diagram of an example input picture 1000 where the coding order is a zigzag order, according to some implementations.
  • the coding order is a zigzag order.
  • the random access unit size is bigger (indicated with a bold rectangle).
  • only sub- Picture 0 is all intra-prediction coded (yellow samples). Both objective and subjective performance are better than raster order.
  • FIG. 11 illustrates a block diagram of an example sub-picture 1100, according to some implementations.
  • FIG. 12 is a block diagram of an example computing system 1200, which may be used for implementations described herein.
  • computing system 1200 may be used to implement area ADC 100 of FIG. 1 or any other associated devices.
  • computing system 1200 may include a processor 1202, an operating system 1204, a memory 1206, an input/output (I/O) interface 1210, and sensor 1212.
  • a software application 1208 may be stored on memory 1206 or on any other suitable storage location or computer-readable medium.
  • Software application 1208 provides instructions that enable processor 1202 to perform the functions described herein.
  • computing system 1200 may not have all of the components listed and/or may have other components instead of, or in addition to, those listed above.
  • the components of computing system 1200 may be implemented by one or more processors or any combination of hardware devices, as well as any combination of hardware, software, firmware, etc.
  • routines of particular embodiments including C, C++, Java, assembly language, etc.
  • Different programming techniques can be employed such as procedural or object oriented.
  • the routines can execute on a single processing device or multiple processors. Although the steps, operations, or computations may be presented in a specific order, this order may be changed in different particular embodiments. In some particular embodiments, multiple steps shown as sequential in this specification can be performed at the same time.
  • Particular embodiments may be implemented in a non-transitory computer- readable storage medium for use by or in connection with the instruction execution system, apparatus, system, or device.
  • Particular embodiments can be implemented in the form of control logic in software or hardware or a combination of both.
  • the control logic when executed by one or more processors, may be operable to perform that which is described in particular embodiments.
  • Particular embodiments may be implemented by using a programmed general purpose digital computer, by using application specific integrated circuits, programmable logic devices, field programmable gate arrays, optical, chemical, biological, quantum or nanoengineered systems, components and mechanisms may be used.
  • the functions of particular embodiments can be achieved by any means as is known in the art.
  • Distributed, networked systems, components, and/or circuits can be used. Communication, or transfer, of data may be wired, wireless, or by any other means.
  • a "processor” includes any suitable hardware and/or software system, mechanism or component that processes data, signals or other information.
  • a processor can include a system with a general-purpose central processing unit, multiple processing units, dedicated circuitry for achieving functionality, or other systems. Processing need not be limited to a geographic location, or have temporal limitations. For example, a processor can perform its functions in "real time,” “offline,” in a “batch mode,” etc. Portions of processing can be performed at different times and at different locations, by different (or the same) processing systems. Examples of processing systems can include servers, clients, end user devices, routers, switches, networked storage, etc.
  • a computer may be any processor in communication with a memory.
  • the memory may be any suitable processor-readable storage medium, such as random-access memory (RAM), read-only memory (ROM), magnetic or optical disk, or other tangible media suitable for storing instructions for execution by the processor.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
  • Compression Of Band Width Or Redundancy In Fax (AREA)
  • Transforming Light Signals Into Electric Signals (AREA)

Abstract

Implementations generally relate to data-charge phase data compression. In one implementation, a method includes computing prediction values for image data, where the image data is data-charge phase data, and where the computing of prediction values is based on inter-block prediction. The method also includes computing residual data based on the prediction values. The method also includes quantizing the residual data. The method also includes entropy encoding the quantized residual data. The method also includes refining an inverse quantized residual data based on one or more of the residual data and a number of left-over bit-budget after entropy encoding.

Description

PCT APPLICATION
DATA-CHARGE PHASE DATA COMPRESSION
ARCHITECTURE
Cross References to Related Applications
This application claims priority from U.S. Patent Application Serial No. 14/736,631, entitled DATA-CHARGE PHASE DATA COMPRESSION ARCHITECTURE, filed on June 11, 2015, which is hereby incorporated by reference as if set forth in full in this application for all purposes.
Background
[01] Image sensors are widely used in devices such as a digital camera to produce digital images. An analog-to-digital converter (ADC) is used in an image sensor to convert analog signals into digital signals. More specifically, an ADC converts an analog pixel signal from a pixel array into digital data. Compressing techniques may be used to compress the digital data representing an image, as well as to preserve the quality of the image.
Summary
[02] Implementations generally relate to data-charge phase data compression and related to compression of image pixel data that is captured by an area analogue-to-digital (AD) sensor based on a predetermined output area AD coding or capturing order. In some implementations, an apparatus includes one or more processors, and includes logic encoded in one or more tangible media for execution by the one or more processors and when executed operable to perform operations including: computing prediction values for image data, where the image data is data-charge phase data, and where the computing of prediction values is based on inter-block prediction; computing residual data based on the prediction values; quantizing the residual data; entropy encoding the quantized residual data; and refining an inverse quantized residual data based on one or more of the residual data and a number of left-over bit-budget after entropy encoding. In some
implementations, software is encoded in one or more non-transitory computer-readable media for execution by the one or more processors. When executed, the software is operable to perform operations including: computing prediction values for image data, where the image data is data-charge phase data, and where the computing of prediction values is based on inter-block prediction; computing residual data based on the prediction values; quantizing the residual data; entropy encoding the quantized residual data; and refining an inverse quantized residual data based on one or more of the residual data and a number of left-over bit-budget after entropy encoding.
[03] In some implementations, a method includes computing prediction values for image data, where the image data is data-charge phase data, and where the computing of prediction values is based on inter-block prediction. The method also includes computing residual data based on the prediction values. The method also includes quantizing the residual data. The method also includes entropy encoding the quantized residual data. The method also includes refining an inverse quantized residual data based on one or more of the residual data and a number of left-over bit-budget after entropy encoding.
[04] A further understanding of the nature and the advantages of particular embodiments disclosed herein may be realized by reference of the remaining portions of the specification and the attached drawings. Brief Description of the Drawings
[05] FIG. 1 illustrates a block diagram of an area analogue-to-digital converter (ADC), which may be used for implementations described herein.
[06] FIG. 2 illustrates an example graph diagram showing data from a correlated double sampling (CDS) based operation, according to some implementations.
[07] FIG. 3 is a block diagram of an example data compressor, according to some implementations .
[08] FIG. 4 illustrates an flow diagram of an example method for compressing image data, according to some implementations.
[09] FIG. 5 illustrates an example bitstream structure, according to some
implementations .
[10] FIG. 6A illustrates a block diagram of example D-phase data, according to some implementations .
[11] FIGS. 6B-6E illustrate block diagrams of example sub-sample coding blocks, according to some implementations.
[12] FIG. 7A illustrates a block diagram of an example input picture, according to some implementations.
[13] FIGS. 7B and 7C illustrate block diagrams of example sub-pictures, according to some implementations.
[14] FIG. 8A illustrates a block diagram of an example input picture, according to some implementations.
[15] FIGS. 8B and 8C illustrate block diagrams of example sub-pictures, according to some implementations.
[16] FIG. 9 illustrates a block diagram of an example input picture, where the coding order is a raster order, according to some implementations.
[17] FIG. 10 illustrates a block diagram of an example input picture, where the coding order is a zigzag order, according to some implementations.
[18] FIG. 11 illustrates a block diagram of an example input picture, according to some implementations.
[19] FIG. 12 illustrates a block diagram of an example computing system, which may be used for implementations described herein.
Detailed Description of Embodiments
[20] Implementations generally relate to data-charge phase data compression. As described in more detail below, implementations employ inter-block prediction based techniques for data compression of data-charge phase (D-phase) data such as that of an area analogue-to-digital (AD) sensor. As described in more detail below, inter-block prediction involves dependency between blocks. Some implementations may also employ intra-block prediction.
[21] In various implementations, an apparatus computes prediction values for image data, where the image data is data-charge phase data, and where the computing of prediction values is based on inter-block prediction. The apparatus also computes residual data based on the prediction values. The apparatus also quantizes the residual data. The apparatus also entropy encodes the quantized residual data. The apparatus also refines an inverse quantized residual data based on one or more of the residual data and a number of left-over bit-budget after entropy encoding. As a result, both subjective and objective performance improves compared to intra-prediction based techniques.
[22] FIG. 1 illustrates a block diagram of an area analogue-to-digital converter (ADC) 100, which may be used for implementations described herein. Area ADC 100 includes areas or blocks 102 of pixels, which generate pre-charge phase (P-phase) data and data-charge phase (P-phase) data.
[23] In various implementations, area ADC 100 performs parallel processing in the blocks 102. For ease of illustration, blocks 102 are shown as squares. In various implementations, blocks 102 may be any shape such as rectangles, circles, etc.
[24] In various implementations, the settling time for area ADC 100 may be significantly reduced, where the settling time may be the time for the sensor to reach a stable output once it is turned on. Such reduced settling times is beneficial for high-speed cameras.
[25] FIG. 2 illustrates an example graph diagram 200 showing data from a correlated double sampling (CDS) based operation, according to some implementations. In various implementations, CDS-based operations involve both P-phase data and D-phase data. Area ADC 100 performs CDS as a noise reduction technique, in which CDS data is generated from the difference between the D-phase data and the P-phase data. The CDS data reflects the canceling of noise in an image.
[26] From a codec point of view, area ADC 100 stores P-phase data, because the timing to obtain the P-phase data and the D-phase data is different. For example, area ADC 100 obtains and stores the P-phase data. When area ADC 100 obtains the D-phase data, ADC 100 may then fetch the P-phase data, and perform CDS in order to cancel noise. As described in more detail below, P-phase compression is desirable and is therefore performed.
[27] FIG. 3 is a block diagram of an example data compressor 300, according to some implementations. In various implementations, data compressor 300 may be a part of area ADC of FIG. 1. As shown, data compressor 300 includes a prediction unit 302, a quantizer 304, an entropy encoder 306, and a refinement unit 308. Example
implementations directed to operations of data compressor 300 are described in more detail below in connection with FIG. 4.
[28] While implementations described herein may be described in the context of image data, these implementations and others also apply to non-image data. For example, P-phase data is primarily non-image data or noise. Implementations described herein may apply to compression P-phase data consisting of non-image data or noise, as well as to compression of D-phase data consisting of image data.
[29] In various implementations, area ADC 100 may operate in two modes. The first mode, and simpler of the two modes, is referred to as pulse-code modulation (PCM). The second mode is referred to as differential pulse-code modulation (DPCM). In various implementations, a 1-bit signal (e.g., a 0 or 1 bit) indicates which mode to apply.
[30] With regard to PCM mode, area ADC 100 sends each bit-plane in an order from the most significant bit (MSB) to least significant bit (LSB), until all of the bit-budget is used. In some implementations, area ADC 100 may perform middle point reconstruction of un-coded bits. In some implementations, PCM mode may be applied to a random access (RA) pixel array, where each block is randomly accessed. In other words, each block may be accessed independently, and independently encoded and/or decoded, where there are no dependencies on surrounding blocks.
[31] In an example implementation, a given pixel block is an 8x1 block, or 8x1 pixels. The pixel block may be any size (e.g., 4x4, etc.). In this example, the block is 8x1 pixels. In this example, for each of 8 pixels, there is a 12-bit depth (bit plane). Each pixel is 12 bits and is to be compressed to 6 bits, but may be compressed to any other number of bits (e.g., 4 bits, etc.), depending on the particular implementation.
[32] If there is a need to code from 12 bit depth to 6 bit depth, 8x12 or 96 bits per sample becomes 8x6 or 48 bits per sample. As a result, an 8x1 block would be coded with 48 bits (instead of 96 bits). [33] In some implementations, area ADC 100 first computes the bit budget. With a 48-bit budget, one bit is used for indicating PCM mode. As such, in this example, there remain 47 bits. Area ADC 100 then removes the header bit. Area ADC 100 then codes each block from MSB to LSB until the bit budget is used up. In some implementations, area ADC 100 then performs middle-point reconstruction in order to minimize errors.
[34] Example implementations of the DPCM mode are described in more detail below in connection with FIG. 4.
[35] FIG. 4 illustrates an flow diagram of an example method for compressing image data, according to some implementations. Referring to both FIGS. 3 and 4, the method is initiated in block 402, where data compressor 300 computes prediction values for image data. In various implementations, the image data is data-charge phase data. The image data is partitioned into blocks, and data compressor 300 computes the prediction values for each block of the image data, and computes prediction values based on inter-block prediction. In other words, data compressor 300 computes the prediction values based on comparing samples of a first block to samples of a second block. In various
implementations, inter-block prediction has an advantage of higher performance and may have a tradeoff of requiring larger random access units. In some implementations, data compressor 300 may computes prediction values based on intra-block prediction. In various implementations, intra-block prediction has an advantage of requiring smaller random access units and may have a tradeoff of lower performance.
[36] As described in more detail below, in various implementations, the computing of prediction values is based on sample data in a same position in a different block. For example, the computing of prediction values may be based on sample data in a same position in a previous block.
[37] Referring again to FIG. 4, in block 404, data compressor 300 computes residual data based on the prediction values. [38] In block 406, data compressor 300 quantizes the residual data. In various implementations, data compressor 300 quantizes the residual image data based on a quantization value. In various implementations, data compressor 300 determines the quantization parameter (qn) at the mode decision process sent in the bitstream. In some implementations, there are two possible techniques, which are described in more detail below.
[39] With regard to the first technique, in some implementations, data compressor 300 selects a predetermined quantization value (qn). Various implementations are possible. For example, in some implementations the quantization value may be based on a predetermined bit-depth. In some implementations, data compressor 300 may select the quantization value from a set of quantization values ranging from a predetermined number (e.g., 0, 1, 2, etc.) to a predetermined bit-depth. For example, if the
predetermined bit depth is 12, the quantization value can be 0 to 11 (12 possible quantization values). In another example implementation, the quantization value may be based on a Q step size of 2n, where n = 1, 2, 3, 4, etc. In some implementations, there may be 4 bits to signal the quantization value in the header. Data compressor 300 tests for different quantization values.
[40] With regard to the second technique, in some implementations, data compressor 300 selects the quantization value from one or more predetermined quantization values. Such quantization values may change depending on the particular implementation. The mode decision may compare these 2 predetermined quantization values. As such, this technique simplifies the mode decision process. Also, only 1 bit is needed to signal the quantization value in the header.
[41] In block 408, data compressor 300 entropy encodes the quantized residual data. In some implementations, data compressor 300 performs entropy encoding based on Golomb coding (e.g., exponential-Golomb coding, progressive-Golomb coding, etc.). Other entropy coding may be used, depending on the particular implementation. [42] In block 410, data compressor 300 refines an inverse of the quantized residual data. In some implementations, data compressor 300 refines the inverse quantized residual data based on one or more of the residual data and a number of left-over bit- budget after entropy encoding.
[43] FIG. 5 illustrates an example bitstream structure 500, according to some implementations. In various implementations, bitstream structure 500 is used for DPCM mode, where various values described above are sent in the bitstream in bitstream structure 500. As shown, a mode field 502 (labeled "Mode") stores a 1-bit signal (e.g., 0 or 1) that indicates whether to use PCM or DPCM. A quantization field 504 (labeled "Qn") stores a quantization value. An entropy encoding field 506 stores entropy encoding values. A residual field 508 (labeled "Residual") stores residual data.
[44] In various implementations, bitstream structure 500 is based on sub-sampled image data. Also, scan order may be variable, and a uniform scan is desirable between all areas.
[45] While data compressor 300 is described as performing the steps as described in the implementations herein, any suitable component or combination of components of area ADC 100 or any suitable processor or processors associated with area ADC 100, and/or data compressor 300, and/or any suitable system may perform the steps described.
[46] FIG. 6A illustrates a block diagram of example D-phase data 600, according to some implementations. In various implementations, D-phase data 600 is D-phase image data from an area ADC such as area ADC 100.
[47] In various implementations, D-phase data 600 is basically the same as a raw Bayer image. Unlike P-phase data, D-phase data has a much higher dynamic range and there exists stronger spatial correlation. The primary differences of D-phase data 600 from raw Bayer include the following. Each block does not consist of consecutively neighboring samples of Bayer image. Also, as described in more detail below, each sample of a block belongs to different area. For example, an area the size may be 4x4, and the block size may be 4x2.
[48] As shown, D-phase data 600 is partitioned into 4x4 areas 602, 604, 606, 608, 610, 612, 614, 616, 618, 620, 622, 624, 626, 628, 630, and 632 (indicated with bold lines). In this particular example, the size of each area is 4x4. Other sizes are possible (e.g., 6x6, 8x8, 10x10, etc.), depending on the particular implementation.
[49] Each 4x4 area includes a coding block such as coding block 640 (indicated with dotted lines). In this particular example, the coding blocks are 4x2. Coding blocks of 4x2 are typical for Bayer image coding. Other sizes are possible (e.g., 4x4, 4x3, 4x1, etc.), depending on the particular implementation.
[50] Each coding block has a predetermined number of samples. In this example, there are 8 samples. The particular number of samples per coding block will depend on the particular implementation.
[51] Each coding block has associated sample positions. For example, coding block 640 has sample positions Al, A2, A3, A4, A5, A6, A7, and A8.
[52] In various implementations, data compressor 100 separates D-phase data 600 into different sub-sample coding blocks based on the location of each sample in the original coding blocks. In this particular example, the sub-sample coding blocks are 4x2. Other sizes are possible (e.g., 4x4, 4x3, 4x1, etc.), depending on the particular implementation.
[53] FIGS. 6B-6E illustrate block diagrams of example sub-sample coding blocks 650, 660, 670, and 680, according to some implementations.
[54] As shown, in this particular implementation, there are four 4x2 sub-sample coding blocks 650, 660, 670, and 680. As shown, sub-sample coding block 650 includes the samples corresponding to Al, Bl, CI, Dl, El, Fl, Gl, and HI. Because each sample of a given sub-sample block belongs to different areas of input picture 600, there is less correlation. Even though there is less correlation, data compressor 300 may still compress the D-phase data based on inter-block predictions.
[55] As a result, the computing of prediction values is based on sample data in a same position in a different coding block. For example, the computing of prediction values may be based on sample data in a same position in a previous block.
[56] As shown, sub-sample coding block 660 includes the samples corresponding to A2, B2, C2, D2, E2, F2, G2, and H2.
[57] As shown, sub-sample coding block 670 includes the samples corresponding to
11, Jl, Kl, LI, Ml, Nl, 01, and PI.
[58] As shown, sub-sample coding block 680 includes the samples corresponding to
12, J2, K2, L2, M2, N2, 02, and P2.
[59] In various implementations, data compressor 300 encodes all of the pixels in a given sub-sample block independently. For example, data compressor 300 may encode all of the pixels in sub-sample block 650, then sub-sample block 660, then sub-sample block 670, and then sub-sample block 680. Then actual order may vary and will depend on the particular implementation.
[60] Because Area ADC coding block consists of sub-sampled data, spatial correlation is much less within a block compared to conventional coding block with consecutive samples. Thus, this makes the DPCM mode less effective if intra-prediction (samples predicted from samples within the same block) is used.
[61] As indicated above, implementations described herein support both PCM mode and DPCM mode. Implementations enable access to neighboring blocks for the prediction for DPCM mode. In some implementations, some of these techniques may require extra memory to store other blocks. Also, a random access unit may become bigger.
[62] FIG. 7A illustrates a block diagram of an example input picture 700, according to some implementations. Input picture 700 includes D-phase data. As shown, input picture 700 has a picture size of 36x24, an area ADC size of 3x3, and an embedded block coding (EBC) block size of 4x1. In some implementations, there is a single color format (not R, Gr, Gb, B colors to simplify the illustration).
[63] FIGS. 7B and 7C illustrate block diagrams of example sub-pictures 710 and 720, according to some implementations. Each sub-picture 710 and 720 has 24 blocks having a block size of 4x1. In various implementations, the samples each of sub-picture 710 and 720 are based on the positions in input picture 700
[64] FIG. 8A illustrates a block diagram of an example input picture 800, according to some implementations. In this example implementation, input picture 800 is the same as input picture 700 of FIG. 7A.
[65] FIGS. 8B and 8C illustrate block diagrams of example sub-pictures 810 and 820, according to some implementations. In this example implementation, sub-pictures 810 and 820 are the same as sub-pictures 710 and 720 of FIG. 7B.
[66] Also shown is a sub-picture 830. In various implementations, the predictor of a current sample is the sample at the same position in previous block or sub-picture. In other words, inter-block prediction is used, where the predictor is from a different block. For example, as shown in FIG. 8B, sub-picture 720 is used to predict sub-picture 710. Also, as shown in FIG. 8C, sub-picture 720 is used to predict sub-picture 730. In some implementations, extra memory may be used to store sub-pictures. [67] In various implementations, the coding order may be fixed, block, or frame level adaptive. Various coding orders are possible depending on the particular implementations .
[68] FIG. 9 illustrates a block diagram of an example input picture 900 where the coding order is a raster order, according to some implementations. As shown, in some implementations, the coding order is a raster order. In this example implementation, the random access unit size is smaller (indicated with a bold rectangle). Also, sub-picture 0, sub-picture 3, sub-picture 6 (indicated with bold arrows) are all intra-prediction coded (yellow samples).
[69] FIG. 10 illustrates a block diagram of an example input picture 1000 where the coding order is a zigzag order, according to some implementations. As shown, in some implementations, the coding order is a zigzag order. In this example implementation, the random access unit size is bigger (indicated with a bold rectangle). Also, only sub- Picture 0 is all intra-prediction coded (yellow samples). Both objective and subjective performance are better than raster order.
[70] FIG. 11 illustrates a block diagram of an example sub-picture 1100, according to some implementations.
[71] FIG. 12 is a block diagram of an example computing system 1200, which may be used for implementations described herein. For example, computing system 1200 may be used to implement area ADC 100 of FIG. 1 or any other associated devices. In some implementations, computing system 1200 may include a processor 1202, an operating system 1204, a memory 1206, an input/output (I/O) interface 1210, and sensor 1212. A software application 1208 may be stored on memory 1206 or on any other suitable storage location or computer-readable medium. Software application 1208 provides instructions that enable processor 1202 to perform the functions described herein. In other embodiments, computing system 1200 may not have all of the components listed and/or may have other components instead of, or in addition to, those listed above. The components of computing system 1200 may be implemented by one or more processors or any combination of hardware devices, as well as any combination of hardware, software, firmware, etc.
[72] Although the description has been described with respect to particular embodiments thereof, these particular embodiments are merely illustrative, and not restrictive.
[73] Any suitable programming language can be used to implement the routines of particular embodiments including C, C++, Java, assembly language, etc. Different programming techniques can be employed such as procedural or object oriented. The routines can execute on a single processing device or multiple processors. Although the steps, operations, or computations may be presented in a specific order, this order may be changed in different particular embodiments. In some particular embodiments, multiple steps shown as sequential in this specification can be performed at the same time.
[74] Particular embodiments may be implemented in a non-transitory computer- readable storage medium for use by or in connection with the instruction execution system, apparatus, system, or device. Particular embodiments can be implemented in the form of control logic in software or hardware or a combination of both. The control logic, when executed by one or more processors, may be operable to perform that which is described in particular embodiments.
[75] Particular embodiments may be implemented by using a programmed general purpose digital computer, by using application specific integrated circuits, programmable logic devices, field programmable gate arrays, optical, chemical, biological, quantum or nanoengineered systems, components and mechanisms may be used. In general, the functions of particular embodiments can be achieved by any means as is known in the art. Distributed, networked systems, components, and/or circuits can be used. Communication, or transfer, of data may be wired, wireless, or by any other means.
[76] It will also be appreciated that one or more of the elements depicted in the drawings/figures can also be implemented in a more separated or integrated manner, or even removed or rendered as inoperable in certain cases, as is useful in accordance with a particular application. It is also within the spirit and scope to implement a program or code that can be stored in a machine -readable medium to permit a computer to perform any of the methods described above.
[77] A "processor" includes any suitable hardware and/or software system, mechanism or component that processes data, signals or other information. A processor can include a system with a general-purpose central processing unit, multiple processing units, dedicated circuitry for achieving functionality, or other systems. Processing need not be limited to a geographic location, or have temporal limitations. For example, a processor can perform its functions in "real time," "offline," in a "batch mode," etc. Portions of processing can be performed at different times and at different locations, by different (or the same) processing systems. Examples of processing systems can include servers, clients, end user devices, routers, switches, networked storage, etc. A computer may be any processor in communication with a memory. The memory may be any suitable processor-readable storage medium, such as random-access memory (RAM), read-only memory (ROM), magnetic or optical disk, or other tangible media suitable for storing instructions for execution by the processor.
[78] As used in the description herein and throughout the claims that follow, "a", "an", and "the" includes plural references unless the context clearly dictates otherwise. Also, as used in the description herein and throughout the claims that follow, the meaning of "in" includes "in" and "on" unless the context clearly dictates otherwise.
[79] Thus, while particular embodiments have been described herein, latitudes of modification, various changes, and substitutions are intended in the foregoing disclosures, and it will be appreciated that in some instances some features of particular embodiments will be employed without a corresponding use of other features without departing from the scope and spirit as set forth. Therefore, many modifications may be made to adapt a particular situation or material to the essential scope and spirit.

Claims

Claims We claim:
1. An apparatus comprising:
one or more processors; and
logic encoded in one or more tangible media for execution by the one or more processors and when executed operable to perform operations including:
computing a compressed bitstream for image data, wherein the image data is data-charge phase data or image data that is captured by an area analogue-to-digital (AD) sensor and based on a predetermined coding order, and wherein the computing of the compressed bitstream is based on inter-block prediction.
2. The apparatus of claim 1, wherein the computing of the compressed bitstream is based on sample data in a same position in a different block.
3. The apparatus of claim 1, wherein the computing of the compressed bitstream is based on sample data in a same position in a previous block.
4. The apparatus of claim 1, wherein the computing of the compressed bitstream comprises:
partitioning the image data into blocks; and
computing the compressed bitstream based on comparing samples of a first block to samples of a second block.
5. The apparatus of claim 1, wherein the predetermined coding order is fixed.
6. The apparatus of claim 1, wherein the predetermined coding order is a raster order.
7. The apparatus of claim 1, wherein the predetermined coding order is a zigzag order.
8. Software encoded in one or more non-transitory computer-readable media for execution by the one or more processors and when executed operable to perform operations comprising:
computing a compressed bitstream for image data, wherein the image data is data-charge phase data or image data that is captured by an area analogue-to-digital (AD) sensor and based on a predetermined coding order, and wherein the computing of the compressed bitstream is based on inter-block prediction;
computing residual data based on the compressed bitstream; quantizing the residual data; and
entropy encoding the quantized residual data.
9. The software of claim 8, wherein the computing of the compressed bitstream is based on sample data in a same position in a different block.
10. The software of claim 8, wherein the computing of the compressed bitstream is based on sample data in a same position in a previous block.
11. The software of claim 8, wherein the computing of the compressed bitstream comprises:
partitioning the image data into blocks; and
computing the compressed bitstream based on comparing samples of a first block to samples of a second block.
12. The software of claim 8, wherein the predetermined coding order is fixed.
13. The software of claim 8, wherein the predetermined coding order is a raster order.
14. The software of claim 8, wherein the predetermined coding order is a zigzag order.
15. A computer-implemented method comprising:
computing a compressed bitstream for image data, wherein the image data is data-charge phase data or image data that is captured by an area analogue-to-digital (AD) sensor and based on a predetermined coding order, and wherein the computing of the compressed bitstream is based on inter-block prediction;
computing residual data based on the compressed bitstream; quantizing the residual data;
entropy encoding the quantized residual data; and
refining an inverse quantized residual data based on one or more of the residual data and a number of left-over bit-budget after entropy encoding.
16. The method of claim 15, wherein the computing of the compressed bitstream is based on sample data in a same position in a different block.
17. The method of claim 15, wherein the computing of the compressed bitstream is based on sample data in a same position in a previous block.
18. The method of claim 15, wherein the computing of the compressed bitstream comprises:
partitioning the image data into blocks; and
computing the compressed bitstream based on comparing samples of a first block to samples of a second block.
19. The method of claim 15, wherein the predetermined coding order is fixed.
20. The method of claim 15, wherein the predetermined coding order is a raster order.
PCT/US2016/036767 2015-06-11 2016-06-09 Data-charge phase data compression architecture WO2016201145A1 (en)

Priority Applications (3)

Application Number Priority Date Filing Date Title
EP16808320.2A EP3295670A4 (en) 2015-06-11 2016-06-09 Data-charge phase data compression architecture
JP2017564411A JP6614420B2 (en) 2015-06-11 2016-06-09 Data charge phase data compression architecture
CN201680032410.1A CN107637079B (en) 2015-06-11 2016-06-09 Data loading phase data compression architecture

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US14/736,631 2015-06-11
US14/736,631 US10091506B2 (en) 2015-06-11 2015-06-11 Data-charge phase data compression architecture

Publications (1)

Publication Number Publication Date
WO2016201145A1 true WO2016201145A1 (en) 2016-12-15

Family

ID=57504366

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2016/036767 WO2016201145A1 (en) 2015-06-11 2016-06-09 Data-charge phase data compression architecture

Country Status (5)

Country Link
US (1) US10091506B2 (en)
EP (1) EP3295670A4 (en)
JP (1) JP6614420B2 (en)
CN (1) CN107637079B (en)
WO (1) WO2016201145A1 (en)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10057577B2 (en) * 2015-06-11 2018-08-21 Sony Corporation Pre-charge phase data compression
US10091506B2 (en) * 2015-06-11 2018-10-02 Sony Corporation Data-charge phase data compression architecture
US10027968B2 (en) * 2015-06-12 2018-07-17 Sony Corporation Data-charge phase data compression tool

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5068726A (en) * 1989-11-28 1991-11-26 Sony Corporation Coding apparatus that temporally interpolates block data and selects transmission mode
US6563953B2 (en) * 1998-11-30 2003-05-13 Microsoft Corporation Predictive image compression using a single variable length code for both the luminance and chrominance blocks for each macroblock
US6763139B1 (en) * 1998-07-03 2004-07-13 Canon Kabushiki Kaisha Image coding method and apparatus for localized decoding at multiple resolutions
US20130010864A1 (en) * 2011-07-05 2013-01-10 Qualcomm Incorporated Image data compression
US20140355675A1 (en) * 2013-05-29 2014-12-04 Research In Motion Limited Lossy data compression with conditional reconstruction refinement
WO2015044660A1 (en) * 2013-09-25 2015-04-02 Sony Corporation Data encoding and decoding

Family Cites Families (28)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP3392949B2 (en) * 1993-07-15 2003-03-31 ペンタックス株式会社 Image compression device and image decompression device
US6529633B1 (en) * 1998-09-16 2003-03-04 Texas Instruments Incorporated Parallel difference coding method for lossless compression and real time decompression
JP3902948B2 (en) * 2001-12-04 2007-04-11 株式会社Kddi研究所 Video encoding device
KR100518849B1 (en) * 2004-03-02 2005-09-30 엘지전자 주식회사 Image coding and decoding method
AU2011205017B2 (en) * 2006-10-23 2013-09-12 Vidyo, Inc. System and method for scalable video coding using telescopic mode flags
US7358880B1 (en) * 2007-02-07 2008-04-15 Cirrus Logic, Inc. Magnetic field feedback delta-sigma modulator sensor circuit
US8345968B2 (en) * 2007-06-28 2013-01-01 Mitsubishi Electric Corporation Image encoding device, image decoding device, image encoding method and image decoding method
JP4900200B2 (en) * 2007-11-15 2012-03-21 ソニー株式会社 Solid-state imaging device and camera system
US8223236B2 (en) 2008-07-03 2012-07-17 Sony Ericsson Mobile Communications Ab Apparatus and method for image recording
US8761531B2 (en) 2009-07-09 2014-06-24 Qualcomm Incorporated Image data compression involving sub-sampling of luma and chroma values
US9288495B2 (en) * 2009-11-24 2016-03-15 Sk Telecom Co., Ltd. Adaptive secondary prediction-based image encoding/decoding method, device and recording medium
CN101866421B (en) * 2010-01-08 2013-05-01 苏州市职业大学 Method for extracting characteristic of natural image based on dispersion-constrained non-negative sparse coding
US9225961B2 (en) 2010-05-13 2015-12-29 Qualcomm Incorporated Frame packing for asymmetric stereo video
CA2803272A1 (en) * 2010-07-05 2012-01-12 Nippon Telegraph And Telephone Corporation Encoding method, decoding method, device, program, and recording medium
JP5539592B2 (en) * 2010-09-14 2014-07-02 サムスン エレクトロニクス カンパニー リミテッド Multi-layer image encoding and decoding apparatus and method
CN106851320B (en) * 2010-11-04 2020-06-02 Ge视频压缩有限责任公司 Digital storage medium, method of decoding bit stream
US20120183234A1 (en) * 2011-01-14 2012-07-19 Sony Corporation Methods for parallelizing fixed-length bitstream codecs
US8634668B2 (en) * 2011-02-25 2014-01-21 Sony Corporation Method of compression of digital images using a fixed number of bits per block
KR20120140181A (en) * 2011-06-20 2012-12-28 한국전자통신연구원 Method and apparatus for encoding and decoding using filtering for prediction block boundary
US9143781B2 (en) * 2012-04-03 2015-09-22 Qualcomm Incorporated Weighted prediction parameter coding
TWI583195B (en) * 2012-07-06 2017-05-11 新力股份有限公司 A solid-state imaging device and a solid-state imaging device, and an electronic device
US8824812B2 (en) * 2012-10-02 2014-09-02 Mediatek Inc Method and apparatus for data compression using error plane coding
CN103037219B (en) * 2012-10-22 2016-08-10 北京航空航天大学 A kind of based on fractal and video compress H.264 and decompression method
US9412385B2 (en) * 2013-05-28 2016-08-09 Qualcomm Incorporated Performing spatial masking with respect to spherical harmonic coefficients
WO2015054811A1 (en) * 2013-10-14 2015-04-23 Microsoft Corporation Features of intra block copy prediction mode for video and image coding and decoding
US10091506B2 (en) * 2015-06-11 2018-10-02 Sony Corporation Data-charge phase data compression architecture
US10057577B2 (en) * 2015-06-11 2018-08-21 Sony Corporation Pre-charge phase data compression
US10027968B2 (en) * 2015-06-12 2018-07-17 Sony Corporation Data-charge phase data compression tool

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5068726A (en) * 1989-11-28 1991-11-26 Sony Corporation Coding apparatus that temporally interpolates block data and selects transmission mode
US6763139B1 (en) * 1998-07-03 2004-07-13 Canon Kabushiki Kaisha Image coding method and apparatus for localized decoding at multiple resolutions
US6563953B2 (en) * 1998-11-30 2003-05-13 Microsoft Corporation Predictive image compression using a single variable length code for both the luminance and chrominance blocks for each macroblock
US20130010864A1 (en) * 2011-07-05 2013-01-10 Qualcomm Incorporated Image data compression
US20140355675A1 (en) * 2013-05-29 2014-12-04 Research In Motion Limited Lossy data compression with conditional reconstruction refinement
WO2015044660A1 (en) * 2013-09-25 2015-04-02 Sony Corporation Data encoding and decoding

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP3295670A4 *

Also Published As

Publication number Publication date
US20160366411A1 (en) 2016-12-15
JP2018527774A (en) 2018-09-20
EP3295670A1 (en) 2018-03-21
CN107637079A (en) 2018-01-26
EP3295670A4 (en) 2019-07-24
CN107637079B (en) 2020-06-30
JP6614420B2 (en) 2019-12-04
US10091506B2 (en) 2018-10-02

Similar Documents

Publication Publication Date Title
RU2665237C1 (en) Moving image encoding device, moving image decoding device, moving image encoding method and moving image decoding method
US11496768B2 (en) GOLOMB-RICE/EG coding technique for CABAC in HEVC
RU2510945C1 (en) Method and apparatus for image encoding and decoding using large transformation unit
EP3295671A1 (en) Data-charge phase data compression tool
JP5061122B2 (en) Adaptive coding of prediction errors in hybrid video coding.
JP5547301B2 (en) Separate video encoder, video decoder, video encoding method and video decoding method for each color plane
CA3123520C (en) Image coding device, image decoding device, image coding method, and image decoding method
JP2017513311A (en) Quantization process for residual differential pulse code modulation
JP7047777B2 (en) Image data coding and decoding
US11310514B2 (en) Encoding method and apparatus using non-encoding region, block-based encoding region, and pixel-based encoding region
JP2011188368A (en) Encoding apparatus, decoding apparatus and program
TW201517600A (en) Data encoding and decoding
US10091506B2 (en) Data-charge phase data compression architecture
US10057577B2 (en) Pre-charge phase data compression
JP2024053019A (en) Coding device, decoding device and program
WO2013160277A1 (en) A method, device, computer program, and information storage means for encoding and decoding an image comprising blocks of pixels
KR20110051497A (en) Encoding system and method of conversion block for raising video compressive efficiency
GB2585044A (en) Image data encoding and decoding

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16808320

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2017564411

Country of ref document: JP

Kind code of ref document: A

WWE Wipo information: entry into national phase

Ref document number: 2016808320

Country of ref document: EP

NENP Non-entry into the national phase

Ref country code: DE