US9271009B2 - Image processing apparatus and image processing method - Google Patents
Image processing apparatus and image processing method Download PDFInfo
- Publication number
- US9271009B2 US9271009B2 US13/443,801 US201213443801A US9271009B2 US 9271009 B2 US9271009 B2 US 9271009B2 US 201213443801 A US201213443801 A US 201213443801A US 9271009 B2 US9271009 B2 US 9271009B2
- Authority
- US
- United States
- Prior art keywords
- block
- length
- variable
- semi
- fixed
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related, expires
Links
- 238000012545 processing Methods 0.000 title claims description 54
- 238000003672 processing method Methods 0.000 title claims description 3
- 230000009466 transformation Effects 0.000 claims description 35
- 238000004590 computer program Methods 0.000 claims description 2
- 238000000034 method Methods 0.000 description 39
- 230000008569 process Effects 0.000 description 15
- 230000003044 adaptive effect Effects 0.000 description 9
- 238000010586 diagram Methods 0.000 description 9
- 230000006870 function Effects 0.000 description 7
- 238000004364 calculation method Methods 0.000 description 4
- 238000011161 development Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000008901 benefit Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000013139 quantization Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
- H04N19/436—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation using parallelised computational arrangements
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/44—Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/90—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using coding techniques not provided for in groups H04N19/10-H04N19/85, e.g. fractals
- H04N19/91—Entropy coding, e.g. variable length coding [VLC] or arithmetic coding
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/129—Scanning of coding units, e.g. zig-zag scan of transform coefficients or flexible macroblock ordering [FMO]
Definitions
- the present invention relates to an image decoding technique.
- JPEG encoding image data is divided into a plurality of blocks each having a predetermined size. These blocks become the unit of encoding and data included in respective blocks undergo orthogonal transform. Then transformation coefficients after this orthogonal transform undergo quantization and coefficient prediction in turn. Furthermore, the data after the coefficient prediction undergo scan, and are converted from two-dimensional data into one-dimensional data including significant coefficients (nonzero data: levels) and a zero runlength of insignificant coefficients (zero data: runs). From the one-dimensional data, which is converted in this way, a plurality of syntax elements are generated, and are entropy-encoded. Finally, codes generated by the entropy encoding are coupled in a predetermined order, thus generating an encoded stream.
- JPEG XR JPEG XR encoding method
- JPEG XR JPEG XR encoding method
- FIG. 4 a prediction error after coefficient prediction is divided into a sign bit (flag) and absolute value.
- This absolute value is further divided into upper data and lower data, which are respectively encoded by different methods.
- the transformation coefficient after orthogonal transform consists of DC, Low-pass (LP), and High-pass (HP) coefficients.
- the adaptive fixed-length coded part of the lower data of DC or lowpass (LP) coefficient is called FLC refinement and that of highpass (HP) coefficient is called flexbits.
- FLC refinement and flexbits are referred to as semi-fixed-length data.
- the lower data is coupled to the encoded stream as semi-fixed-length data.
- a decoding apparatus described in patent literature 1 has a plurality of variable-length code decoders, and a single Flexbits decoder, and speeds up decoding processing by parallelly decoding a variable-length code and Flexbits.
- each variable-length code decoder (to be simply referred to as a variable-length decoder hereinafter) requires a large circuit scale.
- the processing performance improvement and a very large increase in circuit scale are required at the same time.
- a decoding apparatus described in patent literature 2 has a variable-length decoder and Flexbits decoder. Furthermore, a start code of a tile header included in an encoded stream is identified, and the start position of the variable-length code and that of the Flexbits are specified based on this information. Then, the variable-length code and Flexbits are parallelly decoded, thereby speeding up the decoding processing.
- this technique is premised on decoding of the stream encoded in a frequency mode of the JPEG XR, which has two types of encoding modes (spatial mode and frequency mode). Since an encoded stream encoded in the spatial mode does not include any header required to identify the start positions of the variable-length code and Flexbits, the start positions of respective codes cannot be specified.
- the present invention has been made in consideration of the aforementioned problems, and provides a technique for decoding an encoded stream, to which different encoding methods are applied together, more quickly without increasing any circuit scale.
- an image processing apparatus which entropy-decodes an input encoded stream in some streams of which a variable-length code stream is followed by a semi-fixed-length data stream, comprising: a first decoding unit that reads out a variable-length code stream of interest from a position designated as a start position of the variable-length code stream of interest in the input encoded stream, and decodes the readout variable-length code stream of interest; a second decoding unit that reads out a semi-fixed-length data stream of interest which follows the variable-length code stream of interest from the input encoded stream and decodes the readout semi-fixed-length data stream of interest after the decoding of the variable-length code stream of interest; a determination unit that determines a data length of the semi-fixed-length data stream of interest using a result of the decoding of the variable-length code stream of interest after the decoding of the variable-length code stream of interest; and a control unit that designates a position the data length behind the start position
- an image processing method to be executed by an image processing apparatus which entropy-decodes an input encoded stream in some streams of which a variable-length code stream is followed by a semi-fixed-length data stream, comprising: a first decoding step of reading out a variable-length code stream of interest from a position designated as a start position of the variable-length code stream of interest in the input encoded stream, and decoding the readout variable-length code stream of interest; a second decoding step of reading out a semi-fixed-length data stream of interest which follows the variable-length code stream of interest from the input encoded stream, and decoding the readout semi-fixed-length data stream of interest after the decoding of the variable-length code stream of interest; a determination step of determining a data length of the semi-fixed-length data stream of interest using a result of the decoding of the variable-length code stream of interest after the decoding of the variable-length code stream of interest; and a control step of designating a position the data
- FIG. 1 is a block diagram showing an example of the functional arrangement of an image processing apparatus according to the first embodiment
- FIG. 2 is a timing chart of processing according to the first embodiment
- FIG. 3 is a timing chart of processing according to a modification of the first embodiment
- FIG. 4 is a view showing a configuration example of a coefficient prediction error
- FIG. 5 is a view showing the configuration of an encoded stream in a spatial mode
- FIGS. 6A to 6D are views showing examples of the configurations of blocks in the JPEG XR;
- FIG. 7 is a view showing an example of a scan order (ScanOrder) of adaptive scan
- FIG. 8 is a view showing a semi-fixed-length data stream
- FIG. 9 is a block diagram showing an example of the functional arrangement of an image processing apparatus according to the second embodiment.
- FIG. 10 is a timing chart of processing according to the second embodiment
- FIG. 11 is a block diagram showing an example of the functional arrangement of an image processing apparatus according to the third embodiment.
- FIG. 12 is a flowchart of decoding processing according to the fourth embodiment.
- FIG. 13 is a timing chart of processing according to the fourth embodiment.
- FIG. 14 is a block diagram showing an example of the arrangement of a digital camera.
- FIG. 15 is a block diagram showing an example of the functional arrangement of a decoding unit 1413 .
- the image sensor 143 outputs an electrical signal (image signal) corresponding to an intensity of the received light to a development processor 1411 .
- the development processor 1411 applies known image processing such as image interpolation processing and color conversion processing to this image signal, and outputs the processed signal to an encoding unit 1412 as image data.
- the encoding unit 1412 encodes this image data to generate an encoded stream. Then, the encoding unit 1412 outputs this generated encoded stream to a memory controller 1416 or external storage interface 1417 via an internal bus 1418 .
- the memory controller 1416 controls data read/write accesses to a main storage unit 145 . Upon reception of the encoded stream from the encoding unit 1412 , the memory controller 1416 writes this encoded stream in the main storage unit 145 .
- the external storage interface 1417 serves as an interface with an external storage unit 146 . Upon reception of the encoded stream from the encoding unit 1412 , the external storage interface 1417 writes this encoded stream in the external storage unit 146 .
- the memory controller 1416 reads out the designated encoded stream, and transfers the readout stream to a decoding unit 1413 via the internal bus 1418 .
- the decoding unit 1413 decodes the encoded stream received from the memory controller 1416 or external storage interface 1417 , and outputs a decoded image to an image display controller 1414 .
- the image display controller 1414 converts the decoded image received from the decoding unit 1413 into an analog signal, and outputs the converted analog signal to the image display unit 144 .
- the image display unit 144 includes a liquid crystal panel or the like, and displays an image according to the analog signal received from the image display controller 1414 , that is, an image decoded by this apparatus.
- a main CPU 1415 executes operation control of the aforementioned units. Of course, processes (or some of them) of some functional units of the aforementioned units may be executed by the main CPU 1415 .
- an encoded stream is input to the decoding unit 1413 via the internal bus 1418 .
- This encoded stream is input to an entropy decoder 151 .
- the entropy decoder 151 entropy-decodes this input encoded stream, and inputs transformation coefficient prediction errors obtained as a result of the entropy decoding to an inverse scan unit 152 .
- the inverse scan unit 52 applies known inverse scan to the transformation coefficient prediction errors input from the entropy decoder 151 , and inputs the scan result to a coefficient prediction decoder 153 as transformation coefficient prediction errors.
- the coefficient prediction decoder 153 generates quantized transformation coefficients by adding prediction errors to the transformation coefficient prediction errors input from the inverse scan unit 152 , and inputs the generated quantized transformation coefficients to a dequantizer 154 .
- the dequantizer 154 dequantizes the quantized transformation coefficients input from the coefficient prediction decoder 153 , and inputs transformation coefficients obtained as a result of the dequantization to an inverse orthogonal transformer 155 .
- the inverse orthogonal transformer 155 applies inverse orthogonal transform to the transformation coefficients input from the dequantizer 154 , thereby generating an image. In this way, an image can be decoded from the encoded stream by the respective units included in the decoding unit 1413 .
- the output of the entropy decoder 151 corresponds to quantized transformation coefficients. Furthermore, when the dequantization is also skipped, the output from the entropy decoder 151 corresponds to transformation coefficients. In other words, the entropy decoder 151 decodes data corresponding to subsequent processing and outputs it.
- the following embodiments will explain an image processing apparatus applicable to this entropy decoder 151 . Also, the following embodiments will explain a case in which an encoded stream obtained from the main storage unit 145 , external storage unit 146 , encoding unit 1412 , or the like is to be decoded.
- FIG. 1 An example of the functional arrangement of an image processing apparatus according to this embodiment will be described first using the block diagram shown in FIG. 1 .
- a read-out unit 101 an input encoded stream, which is encoded according to the JPEG XR encoding method, is input.
- the read-out unit 101 reads out a variable-length code stream as upper data for each transformation coefficient prediction error from a position in the input encoded stream, which position is designated as a start position of an encoded stream of a block of interest, and outputs the readout stream to a variable-length decoder 102 (read-out unit: output 1 ). Then, upon completion of decoding of the variable-length code stream of the block of interest, the read-out unit 101 reads out, from the input encoded stream, a semi-fixed-length data stream as lower data for each transformation coefficient prediction error, which stream neighbors and follows the variable-length code stream. Then, the read-out unit 101 outputs the readout semi-fixed-length data stream to a semi-fixed-length data decoder 103 (read-out unit: output 2 ).
- the read-out unit 101 may have an arrangement having a read port and a write port, or that having two read ports and two write ports. That is, the read-out unit 101 may adopt arbitrary arrangements as long as it can read out and output a variable-length code stream and semi-fixed-length data stream for each block.
- variable-length decoder 102 decodes the variable-length code stream as the upper data (that in FIG. 4 ) for each transformation coefficient prediction error of the block of interest, which stream is output from the read-out unit 101 (first decoding). Then, the variable-length decoder 102 outputs the decoded upper data for each transformation coefficient prediction error, and also outputs a code length of the decoded upper data for each transformation coefficient prediction error in association with the block of interest to the read-out unit 101 .
- the decoder 102 may output the code length every time it decodes the upper data or may output a total code length after it has completed decoding of all upper data of the block of interest.
- variable-length decoder 102 Upon completion of decoding of the variable-length code stream of the block of interest, the variable-length decoder 102 outputs the number m of upper data, and the number n of significant upper data of those of the transformation coefficient prediction errors (number information), which are obtained by decoding, to a semi-fixed-length data stream length calculator 104 .
- the number of significant upper data of the transformation coefficient prediction errors may be counted after completion of decoding of all upper data of the block of interest, or may be counted every time significant upper data appears. Also, when the variable-length decoder 102 decodes significant upper data, it outputs position information of that significant upper data in the block of interest to the semi-fixed-length data decoder 103 . When the number m of upper data obtained by decoding the variable-length code stream of the block of interest is given or fixed, it need not be output to the semi-fixed-length data decoder 103 .
- the semi-fixed-length data stream length calculator 104 calculates the following equations using the numbers m and n from the variable-length decoder 102 and the number Modelbits (0 ⁇ Modelbits ⁇ 15) ( FIG. 4 ) of bits of lower data of one transformation coefficient prediction error.
- L 0 . . .
- Modelbits 0
- L Modelbits ⁇ m +( m ⁇ n ) . . . Modelbits ⁇ 0
- a data length L of the semi-fixed-length data stream of the block of interest, which stream is output from the read-out unit 101 to the semi-fixed-length data decoder 103 can be calculated.
- the semi-fixed-length data stream length calculator 104 outputs this calculated data length (semi-fixed-length data stream length) L to the read-out unit 101 .
- the read-out unit 101 can designate a position the data length L behind the start position of the semi-fixed-length data stream of the block of interest as a start position of an encoded block of the next block to be decoded. Therefore, the read-out unit 101 reads out the input encoded stream from this designated position, and outputs a variable-length code stream to the variable-length decoder 102 to control the variable-length decoder 102 to start decoding of the variable-length code stream of the next block.
- the semi-fixed-length data decoder 103 will be described below.
- the operation (second decoding) of the semi-fixed-length data decoder 103 when the configuration of an input encoded stream is that of an encoded stream in the spatial mode will be described below.
- FIG. 5 shows the configuration of an encoded stream in the spatial mode.
- an image is divided into a plurality of tiles, and this tile is divided into a plurality of macroblocks (MBs).
- An encoded stream of each MB (MB 1 , MB 2 , MB 3 , . . . ) further includes those of DC, LP (Lowpass), and HP (Highpass) components, which are obtained by applying the orthogonal transform to the image.
- the HP component includes encoded streams for respective blocks (Block 1 , Block 2 , Block 3 , . . . )
- An encoded stream of each block includes a variable-length code stream including a variable-length code group obtained by variable-length encoding upper data, and an encoded stream of a lower data group.
- a CBP Coded Block Pattern
- a CBP is information required for each block to discriminate the presence/absence of significant coefficients of upper data.
- a variable-length code stream of that block does not exist. For this reason, variable-length code streams of which blocks an encoded stream includes are discriminated based on the CBP.
- An encoded stream of lower data has a configuration of an encoded stream in which semi-fixed-length data (coefficient values in FIG. 5 ) and corresponding sign bits are alternately coupled.
- semi-fixed-length data coefficient values in FIG. 5
- sign bit is assigned to encoded variable-length data.
- this sign bit is assigned to only semi-fixed-length data corresponding to insignificant upper data.
- the semi-fixed-length data decoder 103 determines that upper data is insignificant based on a Modelbits value and significant coefficient position information of upper data, it decodes both semi-fixed-length data and a sign flag. When upper data is significant, the decoder 103 decodes only semi-fixed-length data. The decoder 103 outputs a decoded data length to the read-out unit 101 every time it decodes data.
- the each block shown in FIG. 1 is controlled by a control unit (not shown).
- the read-out unit 101 begins to read out a VLC (Variable Length Code) # 1 in a block # 1 , and outputs the readout VLC to the variable-length decoder 102 as [read-out unit: output 1 ]. Then, the variable-length decoder 102 begins to decode this VLC # 1 .
- VLC Very Length Code
- variable-length decoder 102 begins to output decoded upper data.
- the variable-length decoder 102 completes decoding and outputting of all upper data of the VLC # 1 .
- the variable-length decoder 102 outputs the number of upper data and the number of significant upper data (significant coefficients) of those upper data, which are obtained by decoding of the VLC # 1 , to the semi-fixed-length data stream length calculator 104 .
- the semi-fixed-length data stream length calculator 104 calculates a semi-fixed-length data stream length L of the block # 1 , as described above, and outputs it to the read-out unit 101 as [stream length calculator: output].
- the read-out unit 101 begins to read out semi-fixed-length data # 1 in the block # 1 , and outputs the readout semi-fixed-length data # 1 to the semi-fixed-length data decoder 103 as [read-out unit: output 2 ]. Then, the semi-fixed-length data decoder 103 begins to decode the semi-fixed-length data # 1 . Also, the read-out unit 101 designates a position the semi-fixed-length data stream length L behind the start position of the semi-fixed-length data # 1 of the block # 1 as a start position of a VLC # 2 of a block # 2 .
- the read-out unit 101 begins to read out the VLC # 2 of the block # 2 from this designated position, and outputs the readout VLC to the variable-length decoder 102 as [read-out unit: output 1 ].
- the variable-length decoder 102 begins to decode this VLC # 2 .
- the semi-fixed-length data decoder 103 begins to output the decoded semi-fixed-length data # 1 .
- the variable-length decoder 102 begins to output decoded upper data of the VLC # 2 .
- N is an integer equal to or larger than 1.
- the semi-fixed-length data decoder 103 may be substantially stopped, and only a variable-length code stream may be decoded.
- the timing chart shown in FIG. 3 shows the operations of the respective units shown in FIG. 1 when only a variable-length code stream is to be decoded.
- the semi-fixed-length data stream length calculator 104 calculates a semi-fixed-length data stream length L of the block # 1 . Then, the read-out unit 101 designates a position the semi-fixed-length data stream length L behind the start position of the semi-fixed-length data # 1 of the block # 1 as a start position of the VLC # 2 of the block # 2 . The variable-length decoder 102 decodes the VLC # 2 of the block # 2 from this designated position.
- the read-out unit 101 after the read-out unit 101 reads out the VLC # 1 of the block # 1 , it skips a read access of the semi-fixed-length data # 1 , and reads out the VLC # 2 of the block # 2 .
- a mode in which the read-out unit 101 reads out semi-fixed-length data and outputs the readout semi-fixed-length data to the semi-fixed-length data decoder 103 , but the semi-fixed-length data decoder 103 does not operate may be adopted.
- the semi-fixed-length data decoder 103 has to execute decoding corresponding to this adaptive scan, but it can execute decoding at the same timings as those shown in FIG. 2 .
- a scan order determination method executed at the time of encoding is described in detail in the JPEG XR recommendation. For this reason, a description thereof will not be given, and how the semi-fixed-length data decoder 103 executes decoding based on information of the scan order for each block will be described.
- FIG. 6A shows an example of the configuration of a block in the JPEG XR.
- one block includes 15 transformation coefficients.
- Significant coefficient position information is information indicating the ordinal number of an address at which a significant coefficient exists in the block shown in FIG. 6A .
- the significant coefficient position information is generated using an order of adaptive scan (not shown) to be applied to upper data as the decoding result of the variable-length decoder 102 .
- FIG. 7 shows an example of the scan order (ScanOrder) of the adaptive scan.
- the scan order in decoding corresponds to coefficient decoding order information.
- FIG. 6C shows information in which “1” is assigned to significant coefficient addresses, and “0” is assigned to insignificant coefficient addresses in FIG. 6B .
- the information in FIG. 6C is sorted based on the scan order shown in FIG. 7 , thereby generating significant coefficient position information shown in FIG. 6D .
- the series of processes can be executed parallel to decoding of a VLC of the subsequent block #(N+1). That is, significant/insignificant coefficient addresses are sequentially determined in turn based on decoded upper data, and “1” and “0” can be set for addresses in consideration of the scan order.
- the function of generating significant coefficient position information may be independently provided to a unit outside the variable-length decoder 102 , or may be provided to the semi-fixed-length data decoder 103 .
- the semi-fixed-length data decoder 103 decodes semi-fixed-length data based on the Modelbits value and significant coefficient position information. Upon referring to the significant coefficient position information shown in FIG. 6D , a semi-fixed-length data stream shown in FIG. 8 is to be decoded. In FIG. 6D , as for addresses corresponding to insignificant upper data (addresses 2 , 7 , 8 , 10 , 11 , 13 , 14 , and 15 ), semi-fixed-length data with sign bits are decoded. As for other addresses corresponding to significant upper data, semi-fixed-length data without any sign bit are decoded.
- the first embodiment is applicable independently of the presence/absence of the adaptive scan. As can also be seen from the above description, even when several cycles are spent from completion of decoding of each block until generation and outputting of the significant coefficient position information, the effects of the first embodiment will not be reduced.
- the semi-fixed-length data stream length calculator 104 is independently provided. However, the function equivalent to the semi-fixed-length data stream length calculator 104 may be provided to the variable-length decoder 102 , semi-fixed-length data decoder 103 , or read-out unit 101 .
- the arrangement shown in FIG. 1 is may be adopted for only HP (including flexbits) decoding or for only DC/LP (including FLC refinement) decoding. Needless to say, the arrangement may be adopted for both decoding.
- the image processing apparatus according to this embodiment is different from that according to the first embodiment in that k ( ⁇ 2) decoders used to decode a semi-fixed-length data stream are included, and a holding unit used to store position information of significant upper data of transformation coefficient prediction errors is included. Since the k decoders used to decode a semi-fixed-length data stream are included, a read-out unit 901 executes switching control to which decoder a semi-fixed-length data stream is to be output.
- variable-length decoder 902 and semi-fixed-length data stream length calculator 904 perform the same operations as the variable-length decoder 102 and semi-fixed-length data stream length decoder 104 , and a description thereof will not be repeated.
- a significant coefficient position information holding unit 905 is used to store position information indicating positions of significant upper data of transformation coefficient prediction errors in a block when the variable-length decoder 902 has completed decoding of that block. Assume that this significant coefficient position information holding unit 905 has a capacity that can store pieces of position information for k blocks.
- a semi-fixed-length data decoder 903 includes semi-fixed-length data decoders 1 to k.
- the semi-fixed-length data decoders 1 to k perform the same operation as the semi-fixed-length data decoder 103 , and can operate parallelly.
- variable-length decoding ends earlier.
- decoding of a semi-fixed-length data stream of the block #(N+1) is started without waiting for completion of the decoding of the semi-fixed-length data stream of the block #N.
- the semi-fixed-length data decoder 903 includes semi-fixed-length data decoders 1 and 2 will be described below using the timing chart shown in FIG. 10 . Note that processes different from the first embodiment will be mainly explained below, but a description of the same processes will not be repeated.
- variable-length decoder 902 has completed decoding and outputting of a VLC # 1 of a block # 1 , and the semi-fixed-length data stream length calculator 904 calculates a data length L of a semi-fixed-length data stream # 1 of the block # 1 . Then, at time T 1 , the read-out unit 901 begins to output the semi-fixed-length data stream # 1 to the semi-fixed-length data decoder 1 .
- the read-out unit 901 begins to output a VLC # 2 of a block # 2 to the variable-length decoder 902 .
- the decoding result of the semi-fixed-length data stream # 1 whose decoding has begun at time T 1 by the semi-fixed-length data decoder 1 is output from time T 3 .
- variable-length decoder 902 begins to output the decoding result of the VLC # 2 from time T 4 , and completes this outputting at time T 5 . At this timing, decoding and outputting of the semi-fixed-length data stream # 1 are not complete yet.
- the read-out unit 901 outputs a semi-fixed-length data stream # 2 to the semi-fixed-length data decoder 2 as an inactive decoder. Then, at time T 7 , the semi-fixed-length data decoder 2 begins to output the decoding result of the semi-fixed-length data stream # 2 . Parallel to decoding of the semi-fixed-length data streams of the blocks # 1 and # 2 , a VLC of a block # 3 is also decoded.
- decoding of semi-fixed-length data can be started without waiting for the decoding result of the variable-length decoder 902 . This is because if no VLC is included, since all significant coefficients of upper data are “0”, a data length of a semi-fixed-length data stream is allowed to be calculated.
- each of the semi-fixed-length data decoders 1 to k may include the significant coefficient position information holding unit 905 .
- the present invention is not limited to the encoding format of the JPEG XR encoding.
- An example of the functional arrangement of an image processing apparatus according to this embodiment will be described below using the block diagram shown in FIG. 11 .
- an encoded stream which is generated according to an encoding method other than the JPEG XR encoding, is to be decoded.
- an encoded stream in which a block #N is fully variable-length encoded, and a subsequent block #(N+1) is fully encoded as semi-fixed-length data is to be decoded.
- information required to calculate a stream length (data length) of a semi-fixed-length data stream (semi-fixed-length data stream length calculation information in FIG.
- semi-fixed-length data stream length calculation information obtained by decoding the block #N may directly indicate a data length of the semi-fixed-length data of the subsequent block #(N+1).
- this embodiment is applicable to an encoding format having a relationship in which based on the decoding result of a VLC, a data length of semi-fixed-length data that follow that VLC can be derived.
- a read-out unit 1101 and variable-length decoder 1102 perform the same operations as the read-out unit 101 and variable-length decoder 102 , but the variable-length decoder 1102 does not output any information about the number and positions of significant transformation coefficient prediction errors.
- a semi-fixed-length data decoder 1103 decodes a semi-fixed-length data stream in the same manner as the semi-fixed-length data decoder 103 . In this case, the semi-fixed-length data decoder 1103 does not use any position information of significant transformation coefficient prediction errors.
- a semi-fixed-length data stream length calculator 1104 operates in the same manner as the semi-fixed-length data stream length calculator 104 , and calculates each individual semi-fixed-length data stream length.
- the decoding results of the variable-length decoder 1102 and semi-fixed-length data decoder 1103 are transformation coefficient prediction errors, but they need not always be transformation coefficient prediction errors. These decoding results may be those according to the processes of the subsequent stages. Also, semi-fixed-length data stream length calculation information may be supplied from an external unit (not shown) in place of the variable-length decoder 1102 .
- this encoded stream is that, which is generated according to an encoding method other than the JPEG XR encoding method.
- the aforementioned efficient entropy decoding processing can be attained as long as an input encoded stream is as follows independently of its encoding method. That is, the aforementioned efficient entropy decoding processing can be attained as long as an input encoded stream includes a semi-fixed-length data stream which follows a variable-length code stream in at least some streams. Alternatively, the aforementioned efficient entropy decoding processing can be attained as long as an input encoded stream alternately includes variable-length code streams and semi-fixed-length data streams in at least some streams.
- An image processing apparatus which applies entropy decoding processing to such input encoded stream operates as follows.
- variable-length code stream of interest From a position designated as a start position of a variable-length code stream of interest in an input encoded stream, that variable-length code stream of interest is read out and decoded. Upon completion of this decoding, a semi-fixed-length data stream of interest which follows the variable-length code stream of interest is read out from the input encoded data stream, and is decoded. Furthermore, upon completion of the decoding of the VLC stream of interest, a data length of the semi-fixed-length data stream of interest is determined using that decoding result. For example, the data length of the semi-fixed-length data stream of interest is calculated using the number of transformation coefficients obtained by this decoding, the number of significant transformation coefficients of the obtained transformation coefficients, and the number of bits of one transformation coefficient.
- a position the data length behind the start position of the semi-fixed-length data stream of interest is designated as a start position of a next variable-length code stream to be decoded after the variable-length code stream of interest, and decoding of the next variable-length code stream is started.
- This embodiment will explain a case in which the functions of the image processing apparatus described in the above embodiments are implemented as software (JPEG XR decoder) which runs on a PC having a multi-core CPU or a multi-core CPU for an embedded apparatus.
- the main CPU 1415 shown in FIG. 14 may execute this software.
- FIG. 12 shows the flowchart of that processing. Note that a main body of processes in respective steps shown in FIG. 12 is a CPU of this apparatus.
- step S 102 If an encoded stream of a block #N is ready to be acquired, the process advances to step S 102 via step S 101 .
- the CPU activates a decoding processing thread of a VLC #N in step S 102 , so as to read out the VLC #N from the encoded stream of the block #N and to decode the readout VLC #N in step S 103 .
- the CPU continues the process in step S 103 until this decoding is completed.
- step S 105 Upon completion of this decoding, the process advances to step S 105 via step S 104 .
- step S 105 the CPU calculates a semi-fixed-length data stream length of a semi-fixed-length data stream #N.
- step S 106 the CPU quits the thread activated in step S 102 .
- step S 106 the CPU executes the process in step S 107 , and those in step S 108 and subsequent steps.
- step S 107 the CPU increments a variable N by “1” to apply the processes in step S 101 and subsequent steps to the next block.
- step S 108 the CPU compares the number of available threads assigned from, for example, an OS (Operating system), and the number of currently running threads. Then, if all the available threads are used, the CPU stands by in step S 108 . If the available threads include one or more inactive threads, the process advances to step S 109 via step S 108 .
- OS Operating system
- step S 109 the CPU activates a decoding processing thread of the semi-fixed-length data stream #N.
- step S 110 the CPU reads out the semi-fixed-length data stream #N from the block #N and decodes the readout stream. The CPU continues the process in step S 110 until this decoding is completed. Upon completion of this decoding, the process advances to step S 112 via step S 111 . In step S 112 , the CPU quits the thread activated in step S 109 .
- the semi-fixed-length data of the block #N and a VLC of a block #(N+1) can be parallelly decoded.
- the semi-fixed-length data of the block #N and semi-fixed-length data of a block after the block #(N+1) can also be parallelly decoded.
- FIG. 13 is a timing chart showing the processing according to the flowchart shown in FIG. 12 .
- the CPU activates the decoding processing thread of a VLC #N. Then, the CPU reads out the VLC #N from the encoded stream of the block #N, decodes the readout VLC #N, and calculates a semi-fixed-length data stream length of a semi-fixed-length data stream #N. This thread quits at time T 1 (step S 106 ).
- step S 109 the CPU activates the decoding processing thread of the semi-fixed-length data stream #N, reads out the semi-fixed-length data stream #N from the encoded stream of the block #N, and decodes the readout semi-fixed-length data stream #N.
- step S 102 if an encoded stream of the block #(N+1) is ready to be acquired, the CPU activates the decoding processing thread of a VLC #(N+1).
- the CPU then reads out the VLC #(N+1) from the encoded stream of the block #(N+1), decodes the readout VLC #(N+1), and calculates a semi-fixed-length data stream length of a semi-fixed-length data stream #(N+1).
- This thread quits at time T 4 (step S 106 ). At this time, the decoding processing of the semi-fixed-length data stream #N is not complete yet.
- step S 109 the CPU activates the decoding processing thread of the semi-fixed-length data stream #(N+1), reads out the semi-fixed-length data stream #(N+1) from the encoded stream of the block #(N+1), and decodes the readout semi-fixed-length data stream #(N+1).
- this processing is executed only when it is determined in step S 108 that the available threads include one or more inactive threads, and the semi-fixed-length data stream #(N+1) is ready to be read out.
- step S 102 if an encoded stream of a block #(N+2) is ready to be acquired, the CPU activates the decoding processing thread of a VLC #(N+2). Then, the CPU reads out the VLC #(N+2) from the encoded stream of the block #(N+2), decodes the readout VLC #(N+2), and calculates a semi-fixed-length data stream length of a semi-fixed-length data stream #(N+2).
- the semi-fixed-length data of the blocks #N and #(N+1) and the VLC of the block #(N+2) are parallelly decoded.
- the number of blocks of the semi-fixed-length data to be parallelly decoded is not limited to two.
- the semi-fixed-length data stream length calculation method and the significant coefficient position information generation method required at the time of decoding of semi-fixed-length data of the first embodiment are applicable.
- Some or all of the units shown in FIGS. 1 , 9 , and 11 except for the significant coefficient position information holding unit 905 may be implemented as software (computer program). Especially, in case of software implementation, VLC decoding and semi-fixed-length data decoding can be executed as multi-threads, as described in the fourth embodiment.
- aspects of the present invention can also be realized by a computer of a system or apparatus (or devices such as a CPU or MPU) that reads out and executes a program recorded on a memory device to perform the functions of the above-described embodiment(s), and by a method, the steps of which are performed by a computer of a system or apparatus by, for example, reading out and executing a program recorded on a memory device to perform the functions of the above-described embodiment(s).
- the program is provided to the computer for example via a network or from a recording medium of various types serving as the memory device (for example, computer-readable medium).
Abstract
Description
L=0 . . . Modelbits=0
L=Modelbits×m+(m−n) . . .
Claims (10)
L=0 . . . Modelbits=0
L=Modelbits×m+(m−n) . . . Modelbits≠0.
Applications Claiming Priority (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2011093512 | 2011-04-19 | ||
JP2011-093512 | 2011-04-19 | ||
JP2012-074889 | 2012-03-28 | ||
JP2012074889A JP6087511B2 (en) | 2011-04-19 | 2012-03-28 | Image processing apparatus and image processing method |
Publications (2)
Publication Number | Publication Date |
---|---|
US20120269448A1 US20120269448A1 (en) | 2012-10-25 |
US9271009B2 true US9271009B2 (en) | 2016-02-23 |
Family
ID=47021393
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US13/443,801 Expired - Fee Related US9271009B2 (en) | 2011-04-19 | 2012-04-10 | Image processing apparatus and image processing method |
Country Status (2)
Country | Link |
---|---|
US (1) | US9271009B2 (en) |
JP (1) | JP6087511B2 (en) |
Families Citing this family (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP6081862B2 (en) * | 2013-05-15 | 2017-02-15 | 株式会社シキノハイテック | Image encoding device |
JP6081869B2 (en) * | 2013-06-20 | 2017-02-15 | 株式会社シキノハイテック | Image decoding device |
US9432688B2 (en) * | 2013-08-26 | 2016-08-30 | Broadcom Corporation | Parallel symbol decoding |
US10283091B2 (en) * | 2014-10-13 | 2019-05-07 | Microsoft Technology Licensing, Llc | Buffer optimization |
Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20090245672A1 (en) * | 2008-03-25 | 2009-10-01 | Novatek Microelectronics Corp. | Entropy encoding circuit and encoding method thereof |
US20090245674A1 (en) * | 2008-03-27 | 2009-10-01 | Megachips Corporation | Image processor |
US20090274382A1 (en) * | 2008-05-02 | 2009-11-05 | Novateck Microelectronics Corp. | Entropy decoding circuit, entropy decoding method, and entropy decoding method using pipeline manner |
US20100086223A1 (en) * | 2008-10-08 | 2010-04-08 | Megachips Corporation | Image processor |
US20100111430A1 (en) * | 2008-11-04 | 2010-05-06 | Megachips Corporation | Image processor |
US20110262052A1 (en) | 2010-04-27 | 2011-10-27 | Canon Kabushiki Kaisha | Image coding apparatus, control method thereof, and storage medium |
Family Cites Families (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2008306450A (en) * | 2007-06-07 | 2008-12-18 | Panasonic Corp | Image encoder and image decoder |
-
2012
- 2012-03-28 JP JP2012074889A patent/JP6087511B2/en active Active
- 2012-04-10 US US13/443,801 patent/US9271009B2/en not_active Expired - Fee Related
Patent Citations (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20090245672A1 (en) * | 2008-03-25 | 2009-10-01 | Novatek Microelectronics Corp. | Entropy encoding circuit and encoding method thereof |
US20090245674A1 (en) * | 2008-03-27 | 2009-10-01 | Megachips Corporation | Image processor |
US20090274382A1 (en) * | 2008-05-02 | 2009-11-05 | Novateck Microelectronics Corp. | Entropy decoding circuit, entropy decoding method, and entropy decoding method using pipeline manner |
JP2009273106A (en) | 2008-05-02 | 2009-11-19 | Renei Kagi Kofun Yugenkoshi | Entropy decoding circuit, entropy decoding method, and entropy decoding method using pipeline manner |
US20100086223A1 (en) * | 2008-10-08 | 2010-04-08 | Megachips Corporation | Image processor |
JP2010093522A (en) | 2008-10-08 | 2010-04-22 | Mega Chips Corp | Image processor |
US20100111430A1 (en) * | 2008-11-04 | 2010-05-06 | Megachips Corporation | Image processor |
US20110262052A1 (en) | 2010-04-27 | 2011-10-27 | Canon Kabushiki Kaisha | Image coding apparatus, control method thereof, and storage medium |
Also Published As
Publication number | Publication date |
---|---|
JP6087511B2 (en) | 2017-03-01 |
US20120269448A1 (en) | 2012-10-25 |
JP2012235452A (en) | 2012-11-29 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
KR101260073B1 (en) | adaptive coefficient scan order | |
US8509547B2 (en) | Scan conversion apparatus, image encoding apparatus, and control method therefor | |
JP5502988B2 (en) | Apparatus and method for rotating an image | |
US8098947B2 (en) | Method and apparatus for processing image data by rearranging wavelet transform data | |
US9300984B1 (en) | Independent processing of data streams in codec | |
US20100316123A1 (en) | Moving image coding device, imaging device and moving image coding method | |
US8457428B2 (en) | Image coding apparatus, control method thereof, and storage medium | |
JP2007097145A (en) | Image coding device and image coding method | |
EP1689187A1 (en) | Method and system for video compression and decompression (CODEC) in a microprocessor | |
US9271009B2 (en) | Image processing apparatus and image processing method | |
US9055182B2 (en) | Image coding apparatus, image coding method, and program | |
US8787686B2 (en) | Image processing device and image processing method | |
US8233729B2 (en) | Method and apparatus for generating coded block pattern for highpass coefficients | |
US10728557B2 (en) | Embedded codec circuitry for sub-block based entropy coding of quantized-transformed residual levels | |
US10250899B1 (en) | Storing and retrieving high bit depth image data | |
CN109151484B (en) | Method and apparatus for decoding video bitstream | |
KR20170040892A (en) | Method for operating of encoder, and method for operating system on chip including encoder | |
KR101037624B1 (en) | Apparatus and method of rotating image without using memory | |
US20090245672A1 (en) | Entropy encoding circuit and encoding method thereof | |
US20160234529A1 (en) | Method and apparatus for entropy encoding and decoding | |
US10728555B1 (en) | Embedded codec (EBC) circuitry for position dependent entropy coding of residual level data | |
US20070192393A1 (en) | Method and system for hardware and software shareable DCT/IDCT control interface | |
US8369634B2 (en) | Image decoder for HD photo | |
US10652543B2 (en) | Embedded codec circuitry and method for frequency-dependent coding of transform coefficients | |
JP6223012B2 (en) | Image coding apparatus and image coding method |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: CANON KABUSHIKI KAISHA, JAPAN Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:KIMURA, MAKOTO;REEL/FRAME:028525/0746 Effective date: 20120404 |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |
|
MAFP | Maintenance fee payment |
Free format text: PAYMENT OF MAINTENANCE FEE, 4TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1551); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY Year of fee payment: 4 |
|
FEPP | Fee payment procedure |
Free format text: MAINTENANCE FEE REMINDER MAILED (ORIGINAL EVENT CODE: REM.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
LAPS | Lapse for failure to pay maintenance fees |
Free format text: PATENT EXPIRED FOR FAILURE TO PAY MAINTENANCE FEES (ORIGINAL EVENT CODE: EXP.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |