WO2011162299A1 - 情報処理装置と情報処理方法 - Google Patents
情報処理装置と情報処理方法 Download PDFInfo
- Publication number
- WO2011162299A1 WO2011162299A1 PCT/JP2011/064290 JP2011064290W WO2011162299A1 WO 2011162299 A1 WO2011162299 A1 WO 2011162299A1 JP 2011064290 W JP2011064290 W JP 2011064290W WO 2011162299 A1 WO2011162299 A1 WO 2011162299A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- information
- anchor
- anchor information
- unit
- block
- Prior art date
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/103—Selection of coding mode or of prediction mode
- H04N19/105—Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
- H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/44—Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
- H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
- H04N19/51—Motion estimation or motion compensation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
- H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
- H04N19/51—Motion estimation or motion compensation
- H04N19/513—Processing of motion vectors
- H04N19/517—Processing of motion vectors by encoding
- H04N19/52—Processing of motion vectors by encoding by predictive encoding
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
- H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
- H04N19/51—Motion estimation or motion compensation
- H04N19/573—Motion compensation with multiple frame prediction using two or more reference frames in a given prediction direction
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
- H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
- H04N19/51—Motion estimation or motion compensation
- H04N19/577—Motion compensation with bidirectional frame interpolation, i.e. using B-pictures
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
- H04N19/61—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding in combination with predictive coding
Definitions
- the present invention relates to an information processing apparatus and an information processing method. Specifically, it is an object to provide an information processing apparatus and an information processing method that can efficiently use anchor information.
- MPEG2 (ISO / IEC13818-2) is defined as a general-purpose image encoding method, and is currently widely used in a wide range of applications for professional use and consumer use.
- H.26L ITU-T Q6 / 16 VCEG
- MPEG2 and MPEG4 the standardization of a standard called H.26L
- MPEG4 the standardization of a standard called H.26L
- H.264 Standardization that realizes higher encoding efficiency based on H.26L is performed as Joint ⁇ ⁇ ⁇ Model of Enhanced-Compression Video Coding. It became an international standard under the names of H.264 and MPEG-4 Part 10 (hereinafter referred to as “H.264 / AVC (Advanced Video Coding)”).
- a prediction mode using an anchor picture for example, a skip mode or a direct mode (hereinafter referred to as “skip / direct mode”) is defined when deriving a motion vector of a target block.
- skip / direct mode a prediction mode using an anchor picture
- Patent Document 1 describes that inter prediction processing is performed using such an anchor picture.
- an anchor picture is a picture to which a picture to be decoded refers, and a picture decoded at a certain point in time may become an anchor picture of a picture decoded later than that.
- anchor information is generated and stored in a memory for a picture that may be referred to as an anchor picture, and in the skip / direct mode, the anchor information is read and decoded.
- the anchor information has a motion vector of the anchor block in the anchor picture and a reference index for identifying the anchor block in the anchor picture.
- the present invention provides an information processing apparatus and an information processing method that can efficiently use anchor information.
- 1st aspect of this invention is when the anchor information storage part which memorize
- the anchor information of the anchor block corresponding to the block to be decoded is acquired from the anchor information storage unit, and when the same condition is satisfied, the anchor information used in the previous block is continuously used and the acquired
- the information processing apparatus includes an image decoding unit that performs decoding processing using anchor information or the anchor information to be continuously used.
- the anchor information used in the decoding process of the decoding target block satisfies the same condition as the anchor information used in the previous block, for example, based on the identity identification information, acquisition of anchor information or the previous block
- the continued use of the anchor information is determined. That is, based on the identity identification information, when it is determined that the same condition is not satisfied, the anchor information of the anchor block corresponding to the decoding target block is acquired from the anchor information storage unit. When it is determined that the same condition is satisfied, the anchor information used in the previous block is used continuously. Decoding processing is performed using the acquired anchor information or the anchor information that has been continuously used.
- the identity identification information is information generated based on the anchor information generated for each block of the picture for the picture that has been decoded by the image decoding unit used as the anchor picture, or the decoding target block Information generated based on the anchor information used at the time of encoding and the anchor information used in the encoding of the previous block, for example, an identity flag indicating whether the previous block and the anchor information can be regarded as the same, or an anchor This is an identity count value that is information indicating the number of consecutive blocks that can be regarded as having the same information.
- the generated identity flag is stored in a memory provided separately from the anchor information storage unit.
- the generated identity flag is included in the encoded stream.
- the generated identity count value is stored in the anchor information storage unit together with anchor information whose continuity is indicated by the identity count value.
- the anchor information storage unit stores the anchor information.
- a step of acquiring anchor information of an anchor block corresponding to the block to be decoded, a step of continuously using anchor information used in the previous block when the same condition is satisfied, and the acquired anchor information or the continuation And a step of performing a decoding process using the anchor information to be used.
- the anchor block corresponding to the decoding target block from the anchor information storage unit Anchor information of is acquired.
- the anchor information used in the previous block is continuously used. Decoding processing is performed using the acquired anchor information or anchor information to be used continuously. For this reason, since it is not necessary to acquire the anchor information of the corresponding anchor block from the anchor information storage unit for each decoding target block, the anchor information can be used efficiently.
- the inter prediction mode in the skip / direct mode, decoding processing is performed using the anchor information of the anchor block corresponding to the decoding target block. For this reason, as the number of blocks in the skip / direct mode increases, access to anchor information increases.
- the motion vector of an anchor block is often equal between adjacent anchor blocks. For example, each anchor block located in the image of one moving object has the same motion vector.
- the anchor information used in the decoding process of the decoding target block is the same as the anchor information used in the block (previous block) decoded immediately before.
- the anchor information that has already been acquired is continuously used to perform the decoding process, thereby reducing the frequency of access to the memory so that the anchor information can be used efficiently.
- the present invention also relates to H.264.
- the present invention can be applied not only to the H.264 / AVC system but also to a new system for expanding the macroblock size. The description will be given in the following order. 1.
- Decryption process using identity of anchor information 2. When determining the identity of anchor information at the time of decoding 3. When determining the identity of anchor information at the time of encoding 4. Comparison when anchor information is identical in decoding and encoding In the case of software processing Example applied to electronic equipment
- FIG. 1 shows the configuration of the image decoding apparatus 10.
- the image decoding apparatus 10 is an information processing apparatus that performs a decoding process.
- the image decoding apparatus 10 performs a decoding process using a stream (encoded stream) generated by performing an encoding process of image data, and performs an encoding process before the encoding process. Generate image data.
- the image decoding apparatus 10 changes the anchor information storage unit from the anchor information storage unit to the decoding target block when the anchor information used in the decoding process of the decoding target block does not satisfy the same condition as the anchor information used in the previous block.
- the anchor information used in the previous block is assumed to be used continuously, and the decoding process is performed using the acquired anchor information or the anchor information determined to be used continuously. Do.
- the image decoding apparatus 10 includes an accumulation buffer 11, a lossless decoding unit 12, an inverse quantization unit 13, an inverse orthogonal transform unit 14, an addition unit 15, a deblocking filter 16, and a screen rearrangement buffer 17. Furthermore, the image decoding apparatus 10 includes a frame memory 21, selectors 22 and 26, an intra prediction unit 23, and a motion compensation unit 24. In addition, an anchor information storage unit 25 that stores anchor information is provided.
- the encoded stream generated by encoding the input image is supplied to the storage buffer 11 of the image decoding device 10 via a predetermined transmission path, recording medium, or the like.
- the accumulation buffer 11 accumulates the transmitted encoded stream.
- the lossless decoding unit 12 decodes the encoded stream supplied from the accumulation buffer 11.
- the lossless decoding unit 12 performs processing such as variable length decoding and arithmetic decoding on the encoded stream supplied from the accumulation buffer 11, and converts the quantized orthogonal transform coefficient to the inverse quantization unit 13. Output. Further, the lossless decoding unit 12 outputs prediction mode information such as a motion vector obtained by decoding the header information of the encoded stream to the intra prediction unit 23 and the motion compensation unit 24.
- the inverse quantization unit 13 inversely quantizes the quantized data decoded by the lossless decoding unit 12 by a method corresponding to the quantization method used in the image encoding device.
- the inverse orthogonal transform unit 14 performs inverse orthogonal transform on the output of the inverse quantization unit 13 by a method corresponding to the orthogonal transform method used in the image coding apparatus, and outputs the result to the adder unit 15.
- the addition unit 15 adds the data after inverse orthogonal transformation and the predicted image data supplied from the selector 26 to generate decoded image data, and outputs the decoded image data to the deblocking filter 16 and the frame memory 21.
- the deblocking filter 16 performs a filtering process on the decoded image data supplied from the adder 15, removes block distortion, supplies the frame memory 21, accumulates it, and outputs it to the screen rearrangement buffer 17.
- the screen rearrangement buffer 17 rearranges images.
- the screen rearrangement buffer 17 rearranges the order of frames rearranged in the encoding order by the image encoding device into the original display order, and outputs the rearranged order to the D / A conversion unit 18.
- the D / A conversion unit 18 D / A converts the image data supplied from the screen rearrangement buffer 17 and outputs it to a display (not shown) to display the image.
- the frame memory 21 holds the decoded image data before the filtering process supplied from the adding unit 15 and the decoded image data after the filtering process supplied from the deblocking filter 16.
- the selector 22 Based on the prediction mode information supplied from the lossless decoding unit 12, the selector 22 decodes the decoded image data before the filter processing read from the frame memory 21 when the prediction block subjected to the intra prediction is decoded. Is supplied to the intra prediction unit 23. In addition, the selector 22 performs decoding after the filtering process read out from the frame memory 21 when the prediction block subjected to the inter prediction is decoded based on the prediction mode information supplied from the lossless decoding unit 12. The image data is supplied to the motion compensation unit 24.
- the intra prediction unit 23 performs intra prediction processing in the prediction mode indicated by the prediction mode information supplied from the lossless decoding unit 12, and generates predicted image data.
- the intra prediction unit 23 outputs the generated predicted image data to the selector 26.
- the motion compensation unit 24 performs inter prediction processing based on the prediction mode information supplied from the lossless decoding unit 12 to generate predicted image data.
- the motion compensation unit 24 calculates a motion vector of the decoding target block based on the prediction mode information. Further, the motion compensation unit 24 uses the decoded image data indicated by the reference picture information included in the prediction mode information from the decoded image data stored in the frame memory 21. Furthermore, the motion compensation unit 24 performs motion compensation using the decoded image data based on the calculated motion vector and the prediction mode indicated by the prediction mode information, and generates predicted image data.
- the motion compensation unit 24 outputs the generated predicted image data to the selector 26.
- the anchor information storage unit 25 stores anchor information required when the motion compensation unit 24 performs the decoding process on the decoding target block in the skip / direct mode.
- the anchor information is information generated by the motion compensation unit 24 in the decoding process of a picture that may be referred to as an anchor picture.
- the selector 26 supplies the prediction image data generated by the intra prediction unit 23 to the addition unit 15.
- the selector 26 also supplies the predicted image data generated by the motion compensation unit 24 to the adding unit 15.
- FIG. 2 is a flowchart showing an image decoding processing operation performed in the image decoding apparatus 10.
- step ST1 the accumulation buffer 11 accumulates the transmitted encoded stream.
- step ST2 the lossless decoding unit 12 performs a lossless decoding process.
- the lossless decoding unit 12 decodes the encoded stream supplied from the accumulation buffer 11.
- the lossless decoding unit 12 performs processing such as variable length decoding and arithmetic decoding on the encoded stream, and outputs the obtained quantized data to the inverse quantization unit 13.
- the lossless decoding unit 12 outputs prediction mode information obtained by decoding the header information of the encoded stream to the intra prediction unit 23 and the motion compensation unit 24.
- the prediction mode information includes not only prediction modes in intra prediction and inter prediction, but also information on motion vectors and reference pictures used in inter prediction.
- step ST3 the inverse quantization unit 13 performs an inverse quantization process.
- the inverse quantization unit 13 performs an inverse quantization process on the quantized data supplied from the lossless decoding unit 12 and outputs the obtained transform coefficient data to the inverse orthogonal transform unit 14.
- dequantization performs the process which returns to the transformation coefficient data before quantization in an image coding process.
- step ST4 the inverse orthogonal transform unit 14 performs an inverse orthogonal transform process.
- the inverse orthogonal transform unit 14 performs an inverse orthogonal transform process on the transform coefficient data supplied from the inverse quantization unit 13 and outputs the obtained image data to the addition unit 15.
- inverse orthogonal transformation performs the process which returns to the image data before orthogonal transformation in an image coding process.
- step ST5 the adding unit 15 generates decoded image data.
- the adding unit 15 adds the data obtained by performing the inverse orthogonal transform process and the predicted image data selected in step ST9 described later to generate decoded image data. As a result, the original image is decoded.
- step ST6 the deblocking filter 16 performs filter processing.
- the deblocking filter 16 performs a filtering process on the decoded image data output from the adding unit 15, and removes block distortion included in the decoded image.
- step ST7 the frame memory 21 performs storage processing of the decoded image data.
- step ST8 the intra prediction unit 23 and the motion compensation unit 24 perform a prediction process.
- the intra prediction unit 23 and the motion compensation unit 24 perform prediction processing according to the prediction mode information supplied from the lossless decoding unit 12.
- the intra prediction unit 23 performs intra prediction processing in the prediction mode indicated by the prediction mode information, and generates predicted image data.
- the motion compensation unit 24 performs motion compensation based on the prediction mode indicated by the prediction mode information, information on motion vectors, reference pictures, and the like. To generate predicted image data.
- step ST9 the selector 26 selects predicted image data. That is, the selector 26 selects the prediction image supplied from the intra prediction unit 23 and the prediction image data generated by the motion compensation unit 24 and supplies the selected prediction image data to the addition unit 15, and as described above, in step ST5, the inverse orthogonality is performed. The output is added to the output of the conversion unit 14.
- step ST10 the screen rearrangement buffer 17 performs image rearrangement. That is, the screen rearrangement buffer 17 rearranges the order of frames rearranged for encoding to the original display order.
- step ST11 the D / A converter 18 D / A converts the image data from the screen rearrangement buffer 17. This image is output to a display (not shown), and the image is displayed.
- FIG. 3 is a flowchart showing a prediction process performed by the motion compensation unit 24. Note that the inter prediction mode or the intra prediction mode can be set in units of pictures or slices, and FIG. 3 shows a case in which the inter prediction mode is set in units of slices.
- step ST21 the motion compensation unit 24 starts the inter prediction process of the decoding target block and proceeds to step ST22.
- step ST22 the motion compensation unit 24 determines the prediction mode of the decoding target block.
- the motion compensation unit 24 determines the prediction mode based on the prediction mode information supplied from the lossless decoding unit 12, and proceeds to step ST23.
- step ST23 the motion compensation unit 24 determines whether the prediction mode is a mode using anchor information.
- the motion compensation unit 24 proceeds to step ST24 when the prediction mode determined at step ST22 is a mode using anchor information, that is, a skip / direct mode, and proceeds to step ST27 when it is another mode.
- step ST24 the motion compensation unit 24 determines whether the same anchor condition is satisfied.
- the motion compensation unit 24 determines in step ST25. move on. Further, when it cannot be regarded as the same, and when the previous block is in the prediction mode not using anchor information, the process proceeds to step ST26.
- step ST25 the motion compensation unit 24 continues to use the anchor information of the previous block.
- the motion compensation unit 24 continues using the anchor information used in the previous block as the anchor information of the block to be decoded, and proceeds to step ST27. In this way, by continuously using the already read anchor information, the motion compensation unit 24 does not need to read the anchor information from the anchor information storage unit 25.
- step ST26 the motion compensation unit 24 acquires anchor information of the corresponding anchor block.
- the motion compensation unit 24 reads the anchor information generated by the anchor block corresponding to the decoding target block from the anchor information storage unit 25, and proceeds to step ST27.
- the motion compensation unit 24 calculates a motion vector.
- the prediction mode is a mode that uses anchor information
- the motion compensation unit 24 uses the motion vector indicated by the anchor information used in the previous block or the anchor information read from the anchor information storage unit 25 to perform decoding. Calculate the motion vector of the block.
- the prediction mode is a mode that does not use anchor information
- the median of the motion vector of the adjacent block is used as a prediction motion vector, and the prediction motion vector is added to the difference motion vector indicated by the prediction mode information to perform decoding.
- the motion vector of the target block is used.
- the motion vector is calculated according to the prediction mode, and the process proceeds to step ST28.
- step ST28 the motion compensation unit 24 generates predicted image data.
- the motion compensation unit 24 performs motion compensation on the image data of the reference image stored in the frame memory based on the motion vector calculated in step ST27, generates predicted image data, and proceeds to step ST29.
- step ST29 the motion compensation unit 24 determines whether the slice is completed. When it is not the end of the slice, the motion compensation unit 24 returns to step ST21 and processes the next block. In addition, when the slice is finished, the motion compensation unit 24 finishes the inter prediction process for the slice.
- FIG. 4 is a diagram illustrating an anchor information acquisition operation performed by the motion compensation unit 24.
- (A) of FIG. 4 is a figure for demonstrating the acquisition operation
- FIG. 4B shows a conventional anchor information acquisition operation that does not use the identity of anchor information.
- blocks MB0, MB2, MB3, MB6, MB7, MB8, MB10, MB11, and MB14 in the decoding target picture are blocks in the skip / direct mode using anchor information.
- Blocks MB1, MB4, MB5, MB9, MB12, and MB13 in parentheses are other prediction modes that do not use anchor information.
- Anchor information Anc0 in the anchor picture is anchor information of an anchor block corresponding to the block MB0.
- the anchor information Anc1 to 15 is anchor information of anchor blocks corresponding to the blocks MB1 to MB15.
- the anchor information Anc3 is information that can be regarded as the same as the anchor information Anc2.
- the anchor information Anc7 and Anc8 are information that can be regarded as the same as the anchor information Anc6.
- the motion compensation unit 24 determines the corresponding anchor block from the anchor picture. Get anchor information.
- the motion compensation unit 24 continues the already acquired anchor information if the anchor information used in the previous block can be regarded as the same as the anchor information of the block to be decoded. use.
- FIG. 5 shows a case where both consecutive blocks require anchor information in the decoding target picture.
- the decoding target block MBn in the decoding target picture and the previous block MB (n ⁇ 1) which is the immediately preceding block are prediction modes in which decoding processing is performed using anchor information.
- block MBAn is an anchor block corresponding to decoding target block MBn
- block MBA (n ⁇ 1) is an anchor block corresponding to decoding target block MBn.
- the motion compensation unit 24 uses the anchor information Anc0 of the corresponding anchor block in the anchor picture because the block MB0 is a skip / direct mode block. get.
- the motion compensation unit 24 calculates a motion vector of the block MB0 using the motion vector indicated by the acquired anchor information Anc0, performs motion compensation based on the calculated motion vector, and generates predicted image data.
- the motion compensation unit 24 calculates the motion vector according to the prediction mode without acquiring the anchor information Anc1, and predicts the predicted image data. Is generated.
- the motion compensation unit 24 acquires the anchor information Anc2 because the block MB2 is in the skip / direct mode and the previous block MB1, which is the immediately preceding block, is not in the skip / direct mode.
- the motion compensation unit 24 calculates the motion vector of the block MB2 from the motion vector indicated by the acquired anchor information Anc2, and generates predicted image data.
- the motion compensation unit 24 calculates the motion vector of the block MB3 from the motion vector indicated by the anchor information Anc2 that has been continuously used, and generates predicted image data.
- blocks MB6 to MB8 are continuous in skip / direct mode blocks, and anchor information Anc6 to Anc8 is information that can be regarded as the same. Therefore, the motion compensation unit 24 continues to use the anchor information Anc6 as information of the blocks MB7 and MB8.
- the motion compensation unit 24 generates predicted image data by calculating not only the block MB6 but also the blocks MB7 and MB8 from the motion vector indicated by the anchor information Anc6 that has been used continuously.
- the blocks MB10 and MB11 are continuous in skip / direct mode blocks. Further, the anchor information Anc10 and the anchor information Anc8 are not information that can be regarded as the same. Therefore, the motion compensation unit 24 calculates the motion vector of the block MB10 from the motion vector indicated by the anchor information Anc10 of the anchor block corresponding to the block MB10, and generates predicted image data. Also, the motion compensation unit 24 calculates a motion vector of the block MB11 from the motion vector indicated by the anchor information Anc11 of the anchor block corresponding to the block MB11, and generates predicted image data.
- the block to be decoded is the block MB3, even if the anchor information Anc2 and the anchor information Anc3 can be regarded as the same, it corresponds.
- Anchor information Anc3 of the anchor block is acquired.
- the motion vector of the block MB3 is calculated from the motion vector indicated by the acquired anchor information Anc3, and predicted image data is generated.
- the anchor information Anc6 to Anc8 can be regarded as the same, the acquisition of the anchor information Anc7 and Anc8 of the corresponding anchor block is performed.
- the anchor information acquisition operation is performed using the identity of the anchor information, it is not necessary to read the anchor information for each block in the block in which the anchor information that can be regarded as the same is continuous. The number of accesses to can be reduced.
- the motion compensation unit 24 generates anchor information when a picture that may be referred to as an anchor picture is decoded. Furthermore, the identity of the generated anchor information is determined, and identity identification information indicating the determination result is generated.
- the identity identification information may be information that can determine whether the anchor information used in the decoding process of the decoding target block satisfies the same condition as the anchor information used in the previous block. For example, a flag indicating whether the anchor information can be regarded as the same (hereinafter referred to as “identity flag”) can be used as the identity identification information. Further, as other identity identification information, a count value (hereinafter referred to as “identity count value”) indicating the number of consecutive blocks in which the anchor information can be regarded as the same may be used.
- FIG. 6 is a flowchart showing an operation when an identity flag (referred to as “first identity identification information”) is generated as identity identification information. Note that the operation shown in FIG. 6 is performed on a picture that may be referred to as an anchor picture.
- step ST31 the motion compensation unit 24 starts an inter prediction process for blocks, and proceeds to step ST32.
- the motion compensation unit 24 calculates a motion vector. For example, the motion compensation unit 24 calculates a median of motion vectors of adjacent blocks as a predicted motion vector. Furthermore, the motion compensation unit 24 adds the difference motion vector indicated by the prediction mode information supplied from the lossless decoding unit 12 to the prediction motion vector, and proceeds to step ST33 as the motion vector of the block.
- the motion compensation unit 24 adds the difference motion vector indicated by the prediction mode information supplied from the lossless decoding unit 12 to the prediction motion vector, and proceeds to step ST33 as the motion vector of the block.
- a motion vector is used to generate an identity flag. There is no need to recalculate the motion vector.
- step ST33 the motion compensation unit 24 determines whether the motion vectors can be regarded as the same. If the motion vector of the previous block and the motion vector calculated in step ST32 can be regarded as the same, the motion compensation unit 24 proceeds to step ST34. On the other hand, if the motion compensator 24 cannot consider the same, the process proceeds to step ST35. To determine whether or not they can be regarded as the same, the difference between the motion vector of the block and the motion vector of the previous block is compared with a preset threshold value, and if the difference between the motion vectors is less than or equal to the threshold value, it is determined that they can be regarded as the same . Note that the threshold will be described later together with a case where the identity of anchor information is determined at the time of encoding.
- step ST34 the motion compensation unit 24 sets the identity flag to the same state. For example, the motion compensation unit 24 sets the same flag to “1” and proceeds to step ST36.
- step ST35 the motion compensation unit 24 sets the identity flag to a non-identical state. For example, the motion compensation unit 24 sets the same flag to “0” and proceeds to step ST36.
- step ST36 the motion compensation unit 24 determines whether the processing has been completed up to the last block of the slice. When the process has not been completed up to the last block, the motion compensation unit 24 returns to step ST31 and performs the process for the next block. In addition, when the processing is completed for all slices of the picture, the motion compensation unit 24 ends the generation of the identity flag for the picture.
- FIG. 7 is a diagram illustrating an example of the identity flag generation result.
- the block MBA0 of the picture has no previous block. Therefore, the motion compensation unit 24 sets the identity flag FE of the block MBA0 to “0”.
- the motion compensation unit 24 sets the identity flag FE of the block MBA1 to “0” when the motion vector of the block MBA1 cannot be regarded as the same as the motion vector of the previous block MBA0 which is the immediately preceding block.
- the motion compensation unit 24 sets the identity flag FE of the block MBA2 to “0”.
- the motion compensation unit 24 sets the identity flag FE of the block MBA3 to “1”. Thereafter, by performing the same process, the first identity identification information can be generated.
- the first identity identification information in the case shown in FIG. 7 is “010100111100101”.
- the first identity identification information generated in this way needs to be read before the anchor information at the time of decoding. Therefore, the first identity identification information is stored in a memory (for example, SRAM) provided separately from the anchor information storage unit 25 and capable of high-speed reading.
- the first identity identification information has a data amount of 1 bit per block and a small data amount. Therefore, a memory capable of high-speed reading may have a low capacity.
- FIG. 8 shows an operation when the anchor information is read using the identity flag.
- the identity flag FE is “0”
- the motion compensation unit 24 reads the anchor information of the corresponding anchor block from the anchor information storage unit 25.
- the identity flag FE is “1”
- the anchor information used in the previous block is continuously used. Therefore, for example, in the block MB3, the anchor information Anc2 read in the block MB2 is continuously used.
- the anchor information Anc6 of the corresponding anchor block is read from the anchor information storage unit 25. Further, in the blocks MB7 and MB8, the anchor information Anc6 read in the block MB6 is continuously used.
- Second identity identification information an identity count value (referred to as “second identity identification information”) is used as the identity identification information.
- the identity count value is used when, for example, all blocks in a decoded picture are in the skip / direct mode in which anchor information is used and the anchor picture is not switched during the decoding process of the decoded picture. . This point will be described later.
- FIG. 9 is a flowchart showing an operation when an identity count value is generated as identity identification information. Note that the operation shown in FIG. 9 is performed on a picture that may be referred to as an anchor picture.
- step ST41 the motion compensation unit 24 resets the identity count value and proceeds to step ST42.
- step ST42 the motion compensation unit 24 starts the inter prediction process for the block and proceeds to step ST43.
- the motion compensation unit 24 calculates a motion vector. For example, the motion compensation unit 24 calculates a median of motion vectors of adjacent blocks as a predicted motion vector. Further, the difference motion vector indicated by the prediction mode information supplied from the lossless decoding unit 12 is added to the prediction motion vector to calculate the motion vector of the block, and the process proceeds to step ST44. Note that the motion vector generates an identity count value when the motion vector calculated for each block is used to generate predicted image data in the decoding process of a picture that may be referred to as an anchor picture. In addition, there is no need to recalculate the motion vector.
- step ST44 the motion compensation unit 24 determines whether the motion vectors can be regarded as the same. If the motion vector of the previous block and the motion vector calculated in step ST43 can be regarded as the same, the motion compensation unit 24 proceeds to step ST45. On the other hand, if the motion compensator 24 cannot be regarded as the same, it proceeds to step ST46.
- step ST45 the motion compensation unit 24 performs an information update process.
- the motion compensation unit 24 increments an identity count value indicating the number of consecutive blocks that can be regarded as having the same anchor information. Further, in order to make the anchor information of the previous block available, the previous anchor information is held and the process proceeds to step ST48.
- step ST46 the motion compensation unit 24 performs information storage processing. Since the motion compensation unit 24 is not a continuation of blocks in which the anchor information can be regarded as the same, the motion compensation unit 24 stores the identity count value in the anchor information storage unit 25 together with the anchor information holding, and proceeds to step ST47.
- step ST47 the motion compensation unit 24 performs information generation restart processing.
- the motion compensation unit 24 resets the identity count value. Further, the anchor information of blocks that cannot be regarded as the same is held, and the process proceeds to step ST48.
- step ST48 the motion compensation unit 24 determines whether the processing has been completed up to the last block in the picture. When the process has not been completed up to the last block, the motion compensation unit 24 returns to step ST42 and performs the process for the next block. In addition, when the processing to the last block is completed, the motion compensation unit 24 proceeds to step ST49.
- step ST49 the motion compensation unit 24 performs information storage processing. Since the motion compensator 24 determines the identity up to the last block of the picture, it stores the identity count value in the anchor information storage unit 25 together with the anchor information holding the same, and the same for the picture The generation of the sex count value is terminated.
- FIG. 10 is a diagram illustrating an example of the generation result of the identity count value.
- the anchor information Anc0 is held because the anchor information Anc0 of the block MBA0 of the picture and the anchor information Anc1 of the next block MBA1 can be regarded as the same.
- FIG. 11 shows a case where the anchor information is read using the identity count value.
- the motion compensation unit 24 reads the anchor information and the identity count value corresponding to the first block MB0.
- the anchor information Anc0 can be used for the block MB1
- the anchor information Anc0 is used for the block MB2. It can be determined that it is not possible. Therefore, the block MB0 performs the decoding process using the anchor information Anc0, and the block MB1 performs the decoding process using the anchor information Anc0 continuously.
- the anchor information Anc0 cannot be used in the block MB2
- the anchor information corresponding to the block MB2 and the identity count value are read out.
- the anchor information Anc2 can be used for the block MB3
- the anchor information Anc0 cannot be used for the block MB4. Can be determined. Therefore, the block MB2 performs the decoding process using the anchor information Anc2, and the block MB3 performs the decoding process using the anchor information Anc2 continuously.
- block MB4 since anchor information Anc2 cannot be used, the anchor information and identity count value corresponding to block MB4 are read.
- the anchor information Anc4 cannot be used in the block MB5
- the anchor information and the identity count value corresponding to the block MB5 are read out.
- the anchor information Anc5 can be used for the blocks MB6 to MB9
- the anchor information Anc5 is used for the block MB10. It can be determined that it is not possible. Accordingly, the block MB5 performs the decoding process using the anchor information Anc5, and the blocks MB6 to MB9 perform the decoding process using the anchor information Anc5 continuously.
- the anchor information storage unit 25 does not need to store anchor information for each block. Therefore, the capacity of the anchor information storage unit 25 can be reduced.
- the identity count value when used as the identity identification information, the identity count value and the held anchor information are stored in the anchor information storage unit 25. For this reason, unless a block that does not use anchor information in the decoding target picture is considered, the order of the decoding target block does not correspond to the order of the blocks based on the identity count value. For this reason, if all the decoding target pictures are blocks using anchor information, the decoding process can be easily performed because the decoding target block order corresponds to the block order based on the identity count value. Further, when the anchor picture is switched in the middle of the picture, there is no guarantee that the anchor information can be read in the block after the switching. Therefore, the identity count value can be used when the anchor picture is not switched in the middle of the picture.
- the identity identification information may be information that can determine whether the anchor information used in the decoding process of the decoding target block satisfies the same condition as the anchor information used in the previous block. Not only can be generated at the time of encoding.
- the identity identification information is generated at the time of encoding, the generated identity identification information is included in the encoded stream.
- the image decoding apparatus extracts the identity identification information from the encoded stream, and acquires the anchor information or continues to use the anchor information of the previous block based on the extracted identity identification information. Next, the case where the identity of anchor information is determined at the time of encoding and the same anchor identification information is generated will be described.
- FIG. 12 shows the configuration of the image encoding device 50.
- the image coding device 50 is an information processing device that performs coding processing, and includes an analog / digital conversion unit (A / D conversion unit) 51, a screen rearrangement buffer 52, a subtraction unit 53, an orthogonal transformation unit 54, and a quantization unit. 55, a lossless encoding unit 56, a storage buffer 57, and a rate control unit 58.
- the image encoding device 50 includes an inverse quantization unit 61, an inverse orthogonal transform unit 62, an addition unit 63, a deblocking filter 64, a frame memory 65, a selector 66, an intra prediction unit 71, a motion prediction / compensation unit 72, and a prediction.
- An image / optimum mode selection unit 73 is provided.
- the A / D converter 51 converts an analog image signal into digital image data and outputs it to the screen rearrangement buffer 52.
- the screen rearrangement buffer 52 rearranges the frames of the image data output from the A / D conversion unit 51.
- the screen rearrangement buffer 52 rearranges the frames according to the GOP (Group of Pictures) structure related to the encoding process, and subtracts the image data after the rearrangement, the intra prediction unit 71, and the motion prediction / compensation unit. 72.
- GOP Group of Pictures
- the subtraction unit 53 is supplied with the image data output from the screen rearrangement buffer 52 and the predicted image data selected by the predicted image / optimum mode selection unit 73 described later.
- the subtraction unit 53 calculates prediction error data that is a difference between the image data output from the screen rearrangement buffer 52 and the prediction image data supplied from the prediction image / optimum mode selection unit 73, and sends the prediction error data to the orthogonal transformation unit 54. Output.
- the orthogonal transform unit 54 performs orthogonal transform processing such as discrete cosine transform (DCT) and Karoonen-Loeve transform on the prediction error data output from the subtractor 53.
- the orthogonal transform unit 54 outputs transform coefficient data obtained by performing the orthogonal transform process to the quantization unit 55.
- the quantization unit 55 is supplied with transform coefficient data output from the orthogonal transform unit 54 and a rate control signal from a rate control unit 58 described later.
- the quantization unit 55 quantizes the transform coefficient data and outputs the quantized data to the lossless encoding unit 56 and the inverse quantization unit 61. Further, the quantization unit 55 changes the bit rate of the quantized data by switching the quantization parameter (quantization scale) based on the rate control signal from the rate control unit 58.
- the lossless encoding unit 56 is supplied with quantized data output from the quantization unit 55 and prediction mode information from an intra prediction unit 71, a motion prediction / compensation unit 72, and a predicted image / optimum mode selection unit 73, which will be described later.
- the prediction mode information includes a prediction mode (optimum prediction mode) in intra prediction or inter prediction, a motion vector of an encoding target block in inter prediction, reference picture information, and the like.
- the lossless encoding unit 56 performs lossless encoding processing on the quantized data by, for example, variable length encoding or arithmetic encoding, generates an encoded stream, and outputs the encoded stream to the accumulation buffer 57.
- the lossless encoding unit 56 performs lossless encoding of the prediction mode information and adds it to the header information of the encoded stream. Further, when the identity identification information is generated at the time of image encoding, the lossless encoding unit 56 includes the identity identification information generated by the motion prediction / compensation unit 72 in the encoded stream. Further, the lossless encoding unit 56 reduces the data amount of the prediction mode information by including the difference motion vector in the prediction mode information instead of the motion vector of the encoding target block calculated by the motion prediction / compensation unit 72. To do. In this case, the lossless encoding unit 56 calculates a median from motion vectors already calculated for blocks adjacent to the encoding target block, for example, and sets it as a predicted motion vector. The lossless encoding unit 56 calculates a difference between the predicted motion vector and the motion vector of the encoding target block calculated by the motion prediction / compensation unit 72 to obtain a difference motion vector.
- the accumulation buffer 57 accumulates the encoded stream from the lossless encoding unit 56.
- the accumulation buffer 57 outputs the accumulated encoded stream at a transmission rate corresponding to the transmission path.
- the rate control unit 58 monitors the free capacity of the storage buffer 57, generates a rate control signal according to the free capacity, and outputs it to the quantization unit 55.
- the rate control unit 58 acquires information indicating the free space from the accumulation buffer 57, for example.
- the rate control unit 58 reduces the bit rate of the quantized data by the rate control signal when the free space is low. Further, the rate control unit 58 increases the bit rate of the quantized data by the rate control signal when the free space of the accumulation buffer 57 is sufficiently large.
- the inverse quantization unit 61 performs an inverse quantization process on the quantized data supplied from the quantization unit 55.
- the inverse quantization unit 61 outputs transform coefficient data obtained by performing the inverse quantization process to the inverse orthogonal transform unit 62.
- the inverse orthogonal transform unit 62 outputs the data obtained by performing the inverse orthogonal transform process on the transform coefficient data supplied from the inverse quantization unit 61 to the addition unit 63.
- the adder 63 adds the data supplied from the inverse orthogonal transform unit 62 and the predicted image data supplied from the predicted image / optimum mode selection unit 73 to generate decoded image data, and the deblocking filter 64 and the frame memory Output to 65.
- the deblocking filter 64 performs a filter process for reducing block distortion that occurs during image coding.
- the deblocking filter 64 performs a filter process for removing block distortion from the decoded image data supplied from the adding unit 63, and outputs the decoded image data after the filter process to the frame memory 65.
- the frame memory 65 holds the decoded image data supplied from the adding unit 63 and the decoded image data after the filtering process supplied from the deblocking filter 64.
- the selector 66 supplies the decoded image data before filter processing read from the frame memory 65 to perform intra prediction to the intra prediction unit 71. Further, the selector 66 supplies the decoded image data after the filter processing read from the frame memory 65 to the inter prediction, to the motion prediction / compensation unit 72.
- the intra prediction unit 71 uses the image data of the encoding target image output from the screen rearrangement buffer 52 and the decoded image data before the filter process read from the frame memory 65, and performs intra prediction in all candidate intra prediction modes. Perform prediction processing. Further, the intra prediction unit 71 calculates a cost function value for each intra prediction mode, and optimizes the intra prediction mode in which the calculated cost function value is the minimum, that is, the intra prediction mode in which the coding efficiency is the best. Select as the intra prediction mode. The intra prediction unit 71 outputs the predicted image data generated in the optimal intra prediction mode, the prediction mode information regarding the optimal intra prediction mode, and the cost function value in the optimal intra prediction mode to the predicted image / optimum mode selection unit 73. In addition, the intra prediction unit 71 outputs information indicating the intra prediction mode to the lossless encoding unit 56 in the intra prediction process of each intra prediction mode in order to obtain the generated code amount used in the calculation of the cost function value.
- the motion prediction / compensation unit 72 uses the image data of the encoding target image output from the screen rearrangement buffer 52 and the decoded image data after the filtering process read out from the frame memory 65, as a candidate for all inter prediction modes.
- the inter prediction process is performed at Further, the motion prediction / compensation unit 72 calculates a cost function value for each inter prediction mode, and selects an inter prediction mode in which the calculated cost function value is minimum, that is, an inter prediction mode in which the coding efficiency is the best. And selected as the optimal intra prediction mode.
- the motion prediction / compensation unit 72 outputs the prediction image data generated in the optimal inter prediction mode, the prediction mode information regarding the optimal inter prediction mode, and the cost function value in the optimal inter prediction mode to the prediction image / optimum mode selection unit 73. To do.
- the motion prediction / compensation unit 72 outputs information related to the inter prediction mode to the lossless encoding unit 56 in the inter prediction process of each inter prediction mode in order to obtain the generated code amount used in the calculation of the cost function value. Further, when the identity identification information is generated at the time of image encoding, the motion prediction / compensation unit 72 generates the identity identification information and outputs it to the predicted image / optimum mode selection unit 73 or the lossless encoding unit 56.
- the predicted image / optimum mode selection unit 73 compares the cost function value supplied from the intra prediction unit 71 and the cost function value supplied from the motion prediction / compensation unit 72 in units of blocks, and has a smaller cost function value. Are selected as the optimum mode with the best coding efficiency. Further, the predicted image / optimum mode selection unit 73 outputs the predicted image data generated in the optimal mode to the subtraction unit 53 and the addition unit 63. Further, the predicted image / optimum mode selection unit 73 outputs the prediction mode information of the optimal mode to the lossless encoding unit 56. Further, when the identity identification information is supplied from the motion prediction / compensation unit 72, when the optimum inter prediction mode is selected as the optimum mode, the identity identification information is output to the lossless encoding unit 56. Note that the predicted image / optimum mode selection unit 73 performs intra prediction and inter prediction in units of pictures or slices.
- FIG. 13 exemplifies the components related to the generation of identity identification information in the motion prediction / compensation unit 72.
- the motion prediction / compensation unit 72 includes a motion vector detection unit 721, a prediction mode determination unit 722, a prediction mode storage unit 723, an anchor information generation / storage unit 724, and an information generation unit 725.
- the motion vector detection unit 721 detects a motion vector using the image data of the block in the encoding target image read from the screen rearrangement buffer 52 and the decoded image data after filter processing read from the frame memory 65. To do.
- the motion vector detection unit 721 supplies the detected motion vector to the prediction mode determination unit 722 and the anchor information generation / storage unit 724.
- the prediction mode determination unit 722 performs motion compensation processing on the decoded image data based on the supplied motion vector, and generates predicted image data. Moreover, the prediction mode determination part 722 calculates the cost function value when using the produced
- the prediction mode storage unit 723 stores the prediction mode determined in units of pictures and slices. Also, the prediction mode storage unit 723 supplies the stored prediction mode to the information generation unit 725.
- the anchor information generation / storage unit 724 generates anchor information using the motion vector detected by the motion vector detection unit 721. Further, the anchor information generation / storage unit 724 stores the generated anchor information.
- the information generation unit 725 is based on the optimal inter prediction mode determined by the prediction mode determination unit 722, the prediction mode stored in the prediction mode storage unit 723, and the anchor information stored in the anchor information generation / storage unit 724.
- the identity identification information is generated. That is, the information generation unit 725 determines whether the optimal inter prediction mode determined by the prediction mode determination unit 722 is a prediction mode using anchor information.
- the information generation unit 725 determines the prediction mode of the previous block stored in the prediction mode storage unit 723 when the prediction mode uses anchor information. When the prediction mode of the previous block is also a prediction mode using anchor information, the information generation unit 725 has the same anchor information for the block stored in the anchor information generation / storage unit 724 as the anchor information used in the previous block. Determine whether it can be considered.
- the information generation unit 725 sets the identity identification information as information that can be regarded as the same as the anchor information of the previous block. In other cases, the information generation unit 725 sets the identity identification information as information that cannot be regarded as the same as the anchor information of the previous block. In this manner, the information generation unit 725 generates identity identification information and supplies the identity identification information to the lossless encoding unit 56 via the lossless encoding unit 56 or the predicted image / optimum mode selection unit 73.
- FIG. 14 is a flowchart showing the image encoding processing operation.
- the A / D converter 51 performs A / D conversion on the input image signal.
- step ST52 the screen rearrangement buffer 52 performs image rearrangement.
- the screen rearrangement buffer 52 stores the image data supplied from the A / D conversion unit 51, and rearranges from the display order of each picture to the encoding order.
- step ST53 the subtraction unit 53 generates prediction error data.
- the subtracting unit 53 calculates the difference between the image data of the images rearranged in step ST52 and the predicted image data selected by the predicted image / optimum mode selecting unit 73 to generate prediction error data.
- the prediction error data has a smaller data amount than the original image data. Therefore, the data amount can be compressed as compared with the case where the image is encoded as it is.
- the orthogonal transform unit 54 performs an orthogonal transform process.
- the orthogonal transform unit 54 performs orthogonal transform on the prediction error data supplied from the subtraction unit 53. Specifically, orthogonal transformation such as discrete cosine transformation and Karhunen-Loeve transformation is performed on the prediction error data, and transformation coefficient data is output.
- step ST55 the quantization unit 55 performs a quantization process.
- the quantization unit 55 quantizes the transform coefficient data.
- rate control is performed as described in the process of step ST65 described later.
- step ST56 the inverse quantization unit 61 performs an inverse quantization process.
- the inverse quantization unit 61 inversely quantizes the transform coefficient data quantized by the quantization unit 55 with characteristics corresponding to the characteristics of the quantization unit 55.
- the inverse orthogonal transform unit 62 performs an inverse orthogonal transform process.
- the inverse orthogonal transform unit 62 performs inverse orthogonal transform on the transform coefficient data inversely quantized by the inverse quantization unit 61 with characteristics corresponding to the characteristics of the orthogonal transform unit 54.
- step ST58 the adding unit 63 generates decoded image data.
- the adder 63 adds the predicted image data supplied from the predicted image / optimum mode selection unit 73 and the predicted image data and the data after inverse orthogonal transformation of the decoding target block to generate decoded image data.
- step ST59 the deblocking filter 64 performs filter processing.
- the deblocking filter 64 filters the decoded image data output from the addition unit 63 to remove block distortion.
- step ST60 the frame memory 65 stores the decoded image data.
- the frame memory 65 stores the decoded image data before the filtering process and the decoded image data after the filtering process.
- the intra prediction unit 71 and the motion prediction / compensation unit 72 each perform a prediction process. That is, the intra prediction unit 71 performs intra prediction processing in the intra prediction mode, and the motion prediction / compensation unit 72 performs motion prediction / compensation processing in the inter prediction mode.
- the prediction process is performed in all candidate prediction modes, and the cost function value for each prediction mode is calculated. Then, based on the calculated cost function value, the optimal intra prediction mode and the optimal inter prediction mode are selected, and the prediction image generated in the selected prediction mode and its cost function and prediction mode information are predicted image / optimum mode. It is supplied to the selector 73.
- the predicted image / optimum mode selection unit 73 selects predicted image data.
- the predicted image / optimum mode selection unit 73 determines the optimal mode with the best coding efficiency based on the cost function values output from the intra prediction unit 71 and the motion prediction / compensation unit 72. Further, the predicted image / optimum mode selection unit 73 selects the predicted image data of the determined optimal mode and supplies it to the subtraction unit 53 and the addition unit 63. As described above, this predicted image is used for the calculation in step ST58. Note that prediction mode information corresponding to the selected predicted image data is output to the lossless encoding unit 56.
- the lossless encoding unit 56 performs lossless encoding processing.
- the lossless encoding unit 56 performs lossless encoding on the quantized data output from the quantization unit 55. That is, lossless encoding such as variable length encoding or arithmetic encoding is performed on the quantized data, and the data is compressed.
- the prediction mode information (for example, including the prediction mode, the difference motion vector, the reference picture information, and the like) input to the lossless encoding unit 56 in step ST62 described above is also losslessly encoded.
- lossless encoded data of prediction mode information is added to header information of an encoded stream generated by lossless encoding of quantized data.
- the lossless encoding unit 56 includes the identity identification information generated by the motion prediction / compensation unit 72 in the encoded stream.
- step ST64 the accumulation buffer 57 performs accumulation processing.
- the accumulation buffer 57 accumulates the encoded stream output from the lossless encoding unit 56.
- the encoded stream stored in the storage buffer 57 is appropriately read and transmitted to the decoding side via the transmission path.
- step ST65 the rate control unit 58 performs rate control.
- the rate control unit 58 controls the quantization operation rate of the quantization unit 55 so that overflow or underflow does not occur in the accumulation buffer 57 when the encoded buffer is accumulated in the accumulation buffer 57.
- the intra prediction unit 71 performs an intra prediction process.
- the intra prediction unit 71 performs intra prediction on the image of the block to be processed in all candidate intra prediction modes. Note that the decoded image data stored in the frame memory 65 without being filtered by the deblocking filter 64 is used as the image data of the decoded image referred to in the intra prediction.
- intra prediction is performed in all candidate intra prediction modes, and cost function values are calculated for all candidate intra prediction modes. Then, based on the calculated cost function value, one intra prediction mode with the best coding efficiency is selected from all intra prediction modes.
- H As the cost function value, H. As defined by JM (Joint Model), which is reference software in the H.264 / AVC format, it is performed based on either the High Complexity Mode or the Low Complexity Mode.
- Cost (Mode ⁇ ) D + ⁇ ⁇ R (1)
- ⁇ indicates the entire set of prediction modes that are candidates for encoding the block or macroblock.
- D indicates the differential energy (distortion) between the decoded image and the input image when encoding is performed in the prediction mode.
- R is a generated code amount including orthogonal transform coefficients and prediction mode information, and ⁇ is a Lagrange multiplier given as a function of the quantization parameter QP.
- Cost (Mode ⁇ ) D + QPtoQuant (QP) ⁇ Header_Bit (2) ⁇ indicates the entire set of prediction modes that are candidates for encoding the block or macroblock. D indicates the differential energy (distortion) between the decoded image and the input image when encoding is performed in the prediction mode. Header_Bit is a header bit for the prediction mode, and QPtoQuant is a function given as a function of the quantization parameter QP.
- the motion prediction / compensation unit 72 performs an inter prediction process.
- the motion prediction / compensation unit 72 uses the decoded image data after the filter processing stored in the frame memory 65 to perform inter prediction processing in all candidate inter prediction modes.
- the motion prediction / compensation unit 72 performs prediction processing in all candidate inter prediction modes, and calculates cost function values for all candidate inter prediction modes. Then, based on the calculated cost function value, one inter prediction mode with the best coding efficiency is selected from all the inter prediction modes.
- FIG. 15 is a flowchart showing an operation when an identity flag is generated as identity identification information.
- step ST71 the motion prediction / compensation unit 72 determines the prediction mode of the encoding target block. As described above, the motion prediction / compensation unit 72 performs prediction processing in all prediction modes that are candidates in the inter prediction mode, and calculates cost function values for all prediction modes that are candidates.
- step ST72 the motion prediction / compensation unit 72 determines the prediction mode. Based on the cost function value calculated in step ST71, the motion prediction / compensation unit 72 determines one prediction mode that provides the best coding efficiency, that is, the prediction mode that minimizes the cost function value, and proceeds to step ST73. move on.
- step ST73 the motion prediction / compensation unit 72 determines whether the prediction mode uses anchor information.
- the motion prediction / compensation unit 72 proceeds to step ST74 when the encoding target block is in the prediction mode using anchor information, that is, the skip / direct mode, and proceeds to step ST77 when it is in another mode.
- step ST74 the motion prediction / compensation unit 72 determines whether the previous block is a block using anchor information.
- the motion prediction / compensation unit 72 proceeds to step ST75 when the previous block is a block on which decoding processing is performed using anchor information. Also, the motion prediction / compensation unit 72 proceeds to step ST77 when the previous block is not a block on which decoding processing is performed using anchor information.
- step ST75 the motion prediction / compensation unit 72 determines whether the anchor information can be regarded as the same.
- the motion prediction / compensation unit 72 proceeds to step ST76 when the anchor information used in the encoding process of the block can be regarded as the same as the anchor information used in the previous block.
- the motion prediction / compensation unit 72 proceeds to step ST77.
- step ST76 the motion prediction / compensation unit 72 sets the identity flag to the same state. For example, the motion prediction / compensation unit 72 sets the same flag to “1” and proceeds to step ST78.
- step ST77 the motion prediction / compensation unit 72 sets the identity flag to a non-identical state.
- the motion prediction / compensation unit 72 sets the same flag to “0”, for example, and proceeds to step ST78.
- step ST78 the motion prediction / compensation unit 72 determines whether the slice is completed. When the block is not the end of the slice, the motion prediction / compensation unit 72 returns to step ST71 and performs processing for the next block. In addition, when the processing is completed for all slices of the encoding target picture, the motion prediction / compensation unit 72 ends the generation of the identity flag for the picture.
- FIG. 16 is a diagram illustrating an example of the identity flag generation result.
- the block MB0 of the picture to be encoded has no previous block. Therefore, the motion prediction / compensation unit 72 sets the identity flag FE of the block MB0 to “0”.
- the motion prediction / compensation unit 72 sets the identity flag FE of the block MB1 to “0” because the next block MB1 is not in the mode using the anchor information.
- the motion prediction / compensation unit 72 sets the identity flag FE of the block MB2 to “0” because the block MB2 is in the mode using anchor information and the previous block MB1 is not in the mode using anchor information.
- the motion prediction / compensation unit 72 is a mode in which the block MB3 and the previous block MB2 use anchor information, and the anchor information Anc3 used in the block MB3 and the anchor information Anc2 used in the previous block MB2 can be regarded as the same. Therefore, the motion prediction / compensation unit 72 sets the identity flag FE of the block MB3 to “1”. If similar processing is performed thereafter, the identity flag FE can be generated as shown in FIG.
- the anchor information is not read out for each decoding target block. Both can be decrypted.
- Table 1 shows a comparison result between the case where the identity identifying information is generated by the image encoding device and the case where the identity identifying information is generated by the image decoding device.
- the identity flag is generated by determining the identity of the anchor information at the time of decoding, it is necessary to read the identity flag before the anchor information at the time of decoding. The amount of data is small. Therefore, the identity flag is stored in the low capacity memory. Further, when the identity flag is generated at the time of encoding, it is not necessary to store the identity flag in the low-capacity memory because the information on the identity flag is included in the encoded stream. Further, since the identity count value is stored in the anchor information storage unit together with the anchor information whose continuity is indicated by the identity count value, it is not necessary to store the identity count value in the low-capacity memory.
- the identity flag when using the identity flag, it is necessary to read the anchor information of the anchor block corresponding to the decoding target block according to the identity flag. Therefore, it is necessary to store the anchor information of all anchor blocks in the anchor picture in the anchor information storage unit.
- the identity count value when used, the held anchor information is stored in the anchor information storage unit together with the identity count value. Therefore, only the identity count value and the anchor information of some blocks are stored in the anchor information storage unit.
- the identity identification information is generated by determining the identity of the anchor information at the time of decoding. That is, even when an encoded stream generated by a conventional image encoding device is used, reading of anchor information can be reduced. However, when the identity information is generated by determining the identity of the anchor information at the time of encoding, since the identity flag is included in the encoded stream, bits are added.
- anchor picture restrictions there is no anchor picture restriction when the identity flag is used.
- anchor information is not stored for each block. Therefore, if the anchor picture is switched in the middle of a block in which anchor information is continuous, correct anchor information cannot be acquired. For this reason, it is necessary to provide anchor picture restrictions.
- a criterion for determining whether or not the anchor information can be regarded as the same can be set in consideration of image quality and the like. For example, if the difference between motion vectors in the anchor information used for encoding the decoding target block and the anchor information used for encoding the previous block is equal to or less than a preset threshold, increase the threshold. When there is little deterioration in image quality, etc., they are regarded as the same even if the motion vector difference is large. In this case, it is possible to set more blocks that do not need to read anchor information with less influence on image quality.
- the identity identification information when generating the identity identification information by considering the difference when the motion vector difference is equal to or less than the threshold, when the image quality deterioration is increased by using the anchor information of the previous block, and the deterioration exceeds a predetermined level, Even if the difference between the motion vectors is less than or equal to the threshold, an identity flag is generated assuming that the same is not satisfied. In this way, it is possible to control the reading of anchor information from the anchor information storage unit so that image quality deterioration does not exceed a predetermined level.
- the identity identification information is generated at the time of decoding, for example, if the same is considered only when the motion vectors of the anchor information match, the image quality of the decoded image may be deteriorated due to an error with the anchor information used in the previous block. This can prevent the risk of being lost.
- FIG. 17 shows an example for explaining the data amount of the anchor information.
- the anchor block is composed of 4 ⁇ 4 blocks and DirectInferenceflag is set to “1”
- motion vectors and reference indexes of blocks at four corners are used as anchor information.
- the horizontal motion vector is 14 bits
- the vertical motion vector is 12 bits
- the reference index is 6 bits
- the anchor information of one anchor block is (14 + 12 + 6)
- ⁇ 4 128 bits (16 bytes). That is, if there are K blocks that do not need to read anchor information by continuously using the anchor information of the previous block, the reading of the anchor information from the anchor information storage unit 25 requires 16 ⁇ K bytes of data. Reading can be reduced.
- DirectInferenceflag is set to “0”
- the motion vectors and reference indexes of all 4 ⁇ 4 blocks are used as anchor information.
- FIG. 18 is a flowchart showing a schematic operation when a motion vector is calculated in the spatial direct mode.
- the motion compensation unit 24 determines whether the previous block and the anchor information can be regarded as the same.
- the motion compensation unit 24 proceeds to step ST82 when the anchor information cannot be regarded as the same as the previous block based on the identity identification information, and proceeds to step ST84 when it can be regarded as the same.
- step ST82 the motion compensation unit 24 acquires anchor information.
- the motion compensation unit 24 acquires the anchor information of the anchor block corresponding to the block to be decoded from the anchor information storage unit 25, and proceeds to step ST83.
- step ST83 the motion compensation unit 24 generates colZeroFlag.
- the motion compensation unit 24 generates colZeroFlag based on the acquired anchor information, and proceeds to step ST85.
- ColZeroFlag is H.264. In the H.264 / AVC standard, it is information defined for each block of a P picture, and indicates whether there is motion in the image of the block. The colZeroFlag is set to “1” when all of the following are “true”, and is set to “0” otherwise.
- the reference picture having the smallest reference picture number in L1 prediction is a short-term reference picture (short-term reference picture).
- the reference picture number of the reference picture for the anchor block is 0. That is, the anchor picture is a reference picture that is located closest to the rear in the display order with respect to the decoding target picture.
- Both the horizontal and vertical components of the motion vector of the anchor block are values between ⁇ 1 and 1.
- step ST84 the motion compensation unit 24 continues to use the anchor information.
- the motion compensation unit 24 continues using the anchor information of the previous block and proceeds to step ST85. That is, the motion compensation unit 24 continuously uses the colZeroFlag generated based on the anchor information of the previous block without generating the colZeroFlag as in step ST83 by continuously using the anchor information of the previous block. .
- step ST85 the motion compensation unit 24 determines whether or not the motion vector zero determination condition is satisfied. For example, when the colZeroFlag is “1”, the motion compensation unit 24 proceeds to step ST86 assuming that the zero determination condition is satisfied, and proceeds to step ST87 because the zero determination condition is not satisfied when colZeroFlag is “0”.
- step ST86 the motion compensation unit 24 sets the motion vector to “0”.
- the motion compensation unit 24 sets “0” for the decoding target block together with the horizontal and vertical components of the motion vector, and ends the calculation of the motion vector.
- the motion compensation unit 24 performs a motion vector calculation process.
- the motion compensation unit 24 performs median prediction, for example, and sets a median value of motion vectors of adjacent blocks as a predicted motion vector. Furthermore, the motion compensation unit 24 calculates the motion vector of the decoding target block by adding the difference motion vector to the predicted motion vector, and ends the calculation of the motion vector.
- FIG. 19 is a flowchart showing a schematic operation when a motion vector is calculated in the time direct mode.
- step ST91 the motion compensation unit 24 determines whether the previous block and the anchor information can be regarded as the same.
- the motion compensation unit 24 proceeds to step ST92 when the anchor information cannot be regarded as the same as the previous block based on the identity identification information, and proceeds to step ST94 when it can be regarded as the same.
- step ST92 the motion compensation unit 24 acquires anchor information.
- the motion compensation unit 24 acquires the anchor information of the anchor block corresponding to the block to be decoded from the anchor information storage unit 25, and proceeds to step ST93.
- the motion compensation unit 24 calculates a motion vector.
- the motion compensation unit 24 calculates a motion vector based on the acquired anchor information. That is, H.I. As shown in the H.264 / AVC standard, based on the reference index indicated by the anchor information, the time interval between the picture to be decoded and the picture to be referenced in the L0 prediction, the picture to be decoded and the picture to be referenced in the L1 prediction Find the time interval. Further, the motion vector of the block to be decoded is calculated from the motion vectors indicated by the two time intervals and the anchor information.
- step ST94 the motion compensation unit 24 continues to use the anchor information.
- the motion compensation unit 24 continuously uses the anchor information of the previous block. That is, the motion compensation unit 24 continues using the motion vector calculated based on the anchor information of the previous block without calculating the motion vector as in step ST93 by continuously using the anchor information of the previous block. use.
- the anchor information of the decoding target block can be regarded as the same as the anchor information of the previous block by the identity identification information, it is more effective to use the anchor information of the previous block in the following cases.
- the anchor information has a motion vector of “0” and a reference index of “ ⁇ 1”. Therefore, when the anchor picture is an I picture, it is not necessary to read anchor information.
- the anchor picture includes an I slice, a P slice, or the like, when the anchor information is read out in the first block of the slice and the slice is an I slice, it is not necessary to read out the anchor information thereafter.
- the macroblock size is expanded and the horizontal block size becomes longer. For example, when the horizontal direction of the macroblock size is set to a double block size and this block is used as an anchor block, this anchor block is a size in which two blocks whose horizontal direction is 1 time are continuously arranged.
- the anchor information of the block to be decoded and the previous block are equal, reading of the anchor information can be reduced.
- the motion vectors of blocks indicating the background image are equal. Therefore, in many cases, the anchor information of the previous block can be continuously used in the background block, and the reading of the anchor information can be reduced.
- the program can be recorded in advance on a hard disk or ROM (Read Only Memory) as a recording medium.
- the program can be temporarily or permanently stored on a removable recording medium such as a flexible disk, CD-ROM (Compact Disc Read Only Memory), MO (Magneto optical disc), DVD (Digital Versatile Disc), magnetic disk, or semiconductor memory. It can be stored (recorded).
- a removable recording medium can be provided as so-called package software.
- the program is wirelessly transferred from the download site to the computer, or is wired to the computer via a network such as a LAN (Local Area Network) or the Internet.
- the computer can receive the program transferred in this manner and install it on a recording medium such as a built-in hard disk.
- the step of describing the program includes not only the processing that is performed in time series in the described order but also the processing that is not necessarily performed in time series but is executed in parallel or individually.
- the present invention can be used when receiving via network media such as satellite broadcasting, cable TV (television), the Internet, and mobile phones, or on storage media such as light, magnetic disk, and flash memory.
- the present invention can be applied to an image encoding device and an image decoding device used for processing.
- the information processing apparatus described above can be applied to any electronic device. Examples thereof will be described below.
- FIG. 20 illustrates a schematic configuration of a television apparatus to which the present invention is applied.
- the television apparatus 90 includes an antenna 901, a tuner 902, a demultiplexer 903, a decoder 904, a video signal processing unit 905, a display unit 906, an audio signal processing unit 907, a speaker 908, and an external interface unit 909. Furthermore, the television apparatus 90 includes a control unit 910, a user interface unit 911, and the like.
- the tuner 902 selects a desired channel from the broadcast wave signal received by the antenna 901, demodulates it, and outputs the obtained encoded bit stream to the demultiplexer 903.
- the demultiplexer 903 extracts video and audio packets of the program to be viewed from the encoded bit stream, and outputs the extracted packet data to the decoder 904. Further, the demultiplexer 903 supplies a packet of data such as EPG (Electronic Program Guide) to the control unit 910. If scrambling is being performed, descrambling is performed by a demultiplexer or the like.
- EPG Electronic Program Guide
- the decoder 904 performs packet decoding processing, and outputs video data generated by the decoding processing to the video signal processing unit 905 and audio data to the audio signal processing unit 907.
- the video signal processing unit 905 performs noise removal, video processing according to user settings, and the like on the video data.
- the video signal processing unit 905 generates video data of a program to be displayed on the display unit 906, image data by processing based on an application supplied via a network, and the like.
- the video signal processing unit 905 generates video data for displaying a menu screen for selecting an item and the like, and superimposes the video data on the video data of the program.
- the video signal processing unit 905 generates a drive signal based on the video data generated in this way, and drives the display unit 906.
- the display unit 906 drives a display device (for example, a liquid crystal display element or the like) based on a drive signal from the video signal processing unit 905 to display a program video or the like.
- a display device for example, a liquid crystal display element or the like
- the audio signal processing unit 907 performs predetermined processing such as noise removal on the audio data, performs D / A conversion processing and amplification processing on the processed audio data, and outputs the audio data to the speaker 908.
- the external interface unit 909 is an interface for connecting to an external device or a network, and transmits and receives data such as video data and audio data.
- a user interface unit 911 is connected to the control unit 910.
- the user interface unit 911 includes an operation switch, a remote control signal receiving unit, and the like, and supplies an operation signal corresponding to a user operation to the control unit 910.
- the control unit 910 is configured using a CPU (Central Processing Unit), a memory, and the like.
- the memory stores a program executed by the CPU, various data necessary for the CPU to perform processing, EPG data, data acquired via a network, and the like.
- the program stored in the memory is read and executed by the CPU at a predetermined timing such as when the television device 90 is activated.
- the CPU controls each unit so that the television device 90 operates according to the user operation by executing the program.
- the television device 90 is provided with a bus 912 for connecting the tuner 902, the demultiplexer 903, the video signal processing unit 905, the audio signal processing unit 907, the external interface unit 909, and the control unit 910.
- the decoder 904 is provided with the function of the information processing apparatus (information processing method) of the present application. Therefore, when decoding the encoded stream and generating decoded image data, the decoding process can be performed using the anchor information efficiently.
- FIG. 21 exemplifies a schematic configuration of a mobile phone to which the present invention is applied.
- the cellular phone 92 includes a communication unit 922, an audio codec 923, a camera unit 926, an image processing unit 927, a demultiplexing unit 928, a recording / reproducing unit 929, a display unit 930, and a control unit 931. These are connected to each other via a bus 933.
- an antenna 921 is connected to the communication unit 922, and a speaker 924 and a microphone 925 are connected to the audio codec 923. Further, an operation unit 932 is connected to the control unit 931.
- the mobile phone 92 performs various operations such as transmission / reception of voice signals, transmission / reception of e-mail and image data, image shooting, and data recording in various modes such as a voice call mode and a data communication mode.
- the voice signal generated by the microphone 925 is converted into voice data and compressed by the voice codec 923 and supplied to the communication unit 922.
- the communication unit 922 performs audio data modulation processing, frequency conversion processing, and the like to generate a transmission signal.
- the communication unit 922 supplies a transmission signal to the antenna 921 and transmits it to a base station (not shown).
- the communication unit 922 performs amplification, frequency conversion processing, demodulation processing, and the like of the reception signal received by the antenna 921, and supplies the obtained audio data to the audio codec 923.
- the audio codec 923 performs data expansion of the audio data and conversion to an analog audio signal and outputs the result to the speaker 924.
- the control unit 931 receives character data input by operating the operation unit 932 and displays the input characters on the display unit 930.
- the control unit 931 generates mail data based on a user instruction or the like in the operation unit 932 and supplies the mail data to the communication unit 922.
- the communication unit 922 performs mail data modulation processing, frequency conversion processing, and the like, and transmits the obtained transmission signal from the antenna 921.
- the communication unit 922 performs amplification, frequency conversion processing, demodulation processing, and the like of the reception signal received by the antenna 921, and restores mail data. This mail data is supplied to the display unit 930 to display the mail contents.
- the mobile phone 92 can also store the received mail data in a storage medium by the recording / playback unit 929.
- the storage medium is any rewritable storage medium.
- the storage medium is a removable medium such as a semiconductor memory such as a RAM or a built-in flash memory, a hard disk, a magnetic disk, a magneto-optical disk, an optical disk, a USB memory, or a memory card.
- the image data generated by the camera unit 926 is supplied to the image processing unit 927.
- the image processing unit 927 performs encoding processing of image data and generates encoded data.
- the demultiplexing unit 928 multiplexes the encoded data generated by the image processing unit 927 and the audio data supplied from the audio codec 923 by a predetermined method, and supplies the multiplexed data to the communication unit 922.
- the communication unit 922 performs modulation processing and frequency conversion processing of multiplexed data, and transmits the obtained transmission signal from the antenna 921.
- the communication unit 922 performs amplification, frequency conversion processing, demodulation processing, and the like of the reception signal received by the antenna 921, and restores multiplexed data.
- the multiplexed data is supplied to the demultiplexing unit 928.
- the demultiplexing unit 928 performs demultiplexing of the multiplexed data, and supplies the encoded data to the image processing unit 927 and the audio data to the audio codec 923.
- the image processing unit 927 performs a decoding process on the encoded data to generate image data.
- the image data is supplied to the display unit 930 and the received image is displayed.
- the audio codec 923 converts the audio data into an analog audio signal, supplies the analog audio signal to the speaker 924, and outputs the received audio.
- the image processing unit 927 is provided with the function of the information processing device (information processing method) of the present application. For this reason, in decoding of image data, when decoding the encoded stream and generating decoded image data, the decoding process can be performed using the anchor information efficiently.
- FIG. 22 illustrates a schematic configuration of a recording / reproducing apparatus to which the present invention is applied.
- the recording / reproducing apparatus 94 records, for example, audio data and video data of a received broadcast program on a recording medium, and provides the recorded data to the user at a timing according to a user instruction.
- the recording / reproducing device 94 can also acquire audio data and video data from another device, for example, and record them on a recording medium.
- the recording / reproducing device 94 decodes and outputs the audio data and video data recorded on the recording medium, thereby enabling image display and audio output on the monitor device or the like.
- the recording / reproducing apparatus 94 includes a tuner 941, an external interface unit 942, an encoder 943, an HDD (Hard Disk Drive) unit 944, a disk drive 945, a selector 946, a decoder 947, an OSD (On-Screen Display) unit 948, a control unit 949, A user interface unit 950 is included.
- Tuner 941 selects a desired channel from a broadcast signal received by an antenna (not shown).
- the tuner 941 outputs an encoded bit stream obtained by demodulating the received signal of a desired channel to the selector 946.
- the external interface unit 942 includes at least one of an IEEE 1394 interface, a network interface unit, a USB interface, a flash memory interface, and the like.
- the external interface unit 942 is an interface for connecting to an external device, a network, a memory card, and the like, and receives data such as video data and audio data to be recorded.
- the encoder 943 performs encoding by a predetermined method when the video data and audio data supplied from the external interface unit 942 are not encoded, and outputs an encoded bit stream to the selector 946.
- the HDD unit 944 records content data such as video and audio, various programs, and other data on a built-in hard disk, and reads them from the hard disk during playback.
- the disk drive 945 records and reproduces signals with respect to the mounted optical disk.
- An optical disk such as a DVD disk (DVD-Video, DVD-RAM, DVD-R, DVD-RW, DVD + R, DVD + RW, etc.), Blu-ray disk, or the like.
- the selector 946 selects one of the encoded bit streams from the tuner 941 or the encoder 943 and supplies it to either the HDD unit 944 or the disk drive 945 when recording video or audio. Further, the selector 946 supplies the encoded bit stream output from the HDD unit 944 or the disk drive 945 to the decoder 947 at the time of reproduction of video and audio.
- the decoder 947 performs a decoding process on the encoded bit stream.
- the decoder 947 supplies the video data generated by performing the decoding process to the OSD unit 948.
- the decoder 947 outputs audio data generated by performing the decoding process.
- the OSD unit 948 generates video data for displaying a menu screen for selecting an item and the like, and superimposes it on the video data output from the decoder 947 and outputs the video data.
- a user interface unit 950 is connected to the control unit 949.
- the user interface unit 950 includes an operation switch, a remote control signal receiving unit, and the like, and supplies an operation signal corresponding to a user operation to the control unit 949.
- the control unit 949 is configured using a CPU, a memory, and the like.
- the memory stores programs executed by the CPU and various data necessary for the CPU to perform processing.
- the program stored in the memory is read and executed by the CPU at a predetermined timing such as when the recording / reproducing apparatus 94 is activated.
- the CPU executes the program to control each unit so that the recording / reproducing device 94 operates in accordance with the user operation.
- the encoder 943 is provided with the function of the information processing apparatus (information processing method) of the present application. Therefore, when decoding the encoded stream and generating decoded image data, the decoding process can be performed using the anchor information efficiently.
- FIG. 23 illustrates a schematic configuration of an imaging apparatus to which the present invention is applied.
- the imaging device 96 images a subject and displays an image of the subject on a display unit, or records it on a recording medium as image data.
- the imaging device 96 includes an optical block 961, an imaging unit 962, a camera signal processing unit 963, an image data processing unit 964, a display unit 965, an external interface unit 966, a memory unit 967, a media drive 968, an OSD unit 969, and a control unit 970. Have. In addition, a user interface unit 971 is connected to the control unit 970. Furthermore, the image data processing unit 964, the external interface unit 966, the memory unit 967, the media drive 968, the OSD unit 969, the control unit 970, and the like are connected via a bus 972.
- the optical block 961 is configured using a focus lens, a diaphragm mechanism, and the like.
- the optical block 961 forms an optical image of the subject on the imaging surface of the imaging unit 962.
- the imaging unit 962 is configured using a CCD or CMOS image sensor, generates an electrical signal corresponding to the optical image by photoelectric conversion, and supplies the electrical signal to the camera signal processing unit 963.
- the camera signal processing unit 963 performs various camera signal processing such as knee correction, gamma correction, and color correction on the electrical signal supplied from the imaging unit 962.
- the camera signal processing unit 963 supplies the image data after the camera signal processing to the image data processing unit 964.
- the image data processing unit 964 performs an encoding process on the image data supplied from the camera signal processing unit 963.
- the image data processing unit 964 supplies the encoded data generated by performing the encoding process to the external interface unit 966 and the media drive 968. Further, the image data processing unit 964 performs a decoding process on the encoded data supplied from the external interface unit 966 and the media drive 968.
- the image data processing unit 964 supplies the image data generated by performing the decoding process to the display unit 965. Further, the image data processing unit 964 superimposes the processing for supplying the image data supplied from the camera signal processing unit 963 to the display unit 965 and the display data acquired from the OSD unit 969 on the image data. To supply.
- the OSD unit 969 generates display data such as a menu screen and icons made up of symbols, characters, or figures and outputs them to the image data processing unit 964.
- the external interface unit 966 includes, for example, a USB input / output terminal, and is connected to a printer when printing an image.
- a drive is connected to the external interface unit 966 as necessary, a removable medium such as a magnetic disk or an optical disk is appropriately mounted, and a computer program read from them is installed as necessary.
- the external interface unit 966 has a network interface connected to a predetermined network such as a LAN or the Internet.
- the control unit 970 reads the encoded data from the memory unit 967 in accordance with an instruction from the user interface unit 971, and supplies the encoded data to the other device connected via the network from the external interface unit 966. it can.
- the control unit 970 may acquire encoded data and image data supplied from another device via the network via the external interface unit 966 and supply the acquired data to the image data processing unit 964. it can.
- any readable / writable removable medium such as a magnetic disk, a magneto-optical disk, an optical disk, or a semiconductor memory is used.
- the recording medium may be any type of removable medium, and may be a tape device, a disk, or a memory card. Of course, a non-contact IC card or the like may be used.
- media drive 968 and the recording medium may be integrated and configured by a non-portable storage medium such as a built-in hard disk drive or an SSD (Solid State Drive).
- a non-portable storage medium such as a built-in hard disk drive or an SSD (Solid State Drive).
- the control unit 970 is configured using a CPU, a memory, and the like.
- the memory stores programs executed by the CPU, various data necessary for the CPU to perform processing, and the like.
- the program stored in the memory is read and executed by the CPU at a predetermined timing such as when the imaging device 96 is activated.
- the CPU executes the program to control each unit so that the imaging device 96 operates according to the user operation.
- the function of the information processing apparatus (information processing method) of the present application is provided in the image data processing unit 964. For this reason, when decoding the encoded data recorded in the memory unit 967, the recording medium, or the like to generate decoded image data, the decoding process can be performed using the anchor information efficiently.
- the information processing apparatus and the information processing method when the anchor information used in the decoding process of the decoding target block does not satisfy the same condition as the anchor information used in the previous block, the information is decoded from the anchor information storage unit. Anchor information of an anchor block corresponding to the target block is acquired. In addition, when the same condition is satisfied, the anchor information used in the previous block is continuously used. Decoding processing is performed using the acquired anchor information or anchor information to be used continuously. For this reason, since it is not necessary to acquire the anchor information of the corresponding anchor block from the anchor information storage unit for each decoding target block, the anchor information can be used efficiently. Therefore, it is suitable for an electronic device that performs image data decoding processing.
- lossless encoding unit 58 ... rate control unit, 72 ... motion prediction Compensation unit, 73 ... predicted image / optimum mode selection unit, 90 ... television device, DESCRIPTION OF SYMBOLS 2 ... Mobile telephone, 94 ... Recording / reproducing apparatus, 96 ... Imaging apparatus, 721 ... Motion vector detection part, 722 ... Prediction mode determination part, 723 ... Prediction mode memory
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
Description
1.アンカー情報の同一性を利用した復号化処理
2.復号化時にアンカー情報の同一性を判定する場合について
3.符号化時にアンカー情報の同一性を判定する場合について
4.アンカー情報の同一性を復号化時と符号化時で行った場合の比較
5.ソフトウェア処理の場合
6.電子機器に適用した例
情報処理装置において、アンカー情報の同一性を利用した復号化処理を行う場合について説明する。
図1は、画像復号化装置10の構成を示している。画像復号化装置10は、復号化処理を行う情報処理装置であり、画像データの符号化処理を行って生成されたストリーム(符号化ストリーム)を用いて復号化処理を行い、符号化処理前の画像データを生成する。また、画像復号化装置10は、復号化対象ブロックの復号化処理で用いるアンカー情報が前ブロックで用いたアンカー情報との同一条件を満たしていない場合に、アンカー情報記憶部から復号化対象ブロックに対応するアンカーブロックのアンカー情報を取得して、同一条件を満たす場合に、前ブロックで用いたアンカー情報を継続使用として、取得したアンカー情報または継続使用とされたアンカー情報を用いて復号化処理を行う。
図2は、画像復号化装置10で行われる画像復号処理動作を示したフローチャートである。
次に、復号化時にアンカー情報の同一性を判定して、同一アンカー識別情報を生成する場合について説明する。
図6は、同一性識別情報として、同一性フラグ(「第1の同一性識別情報」という)を生成する場合の動作を示すフローチャートである。なお、図6に示す動作は、アンカーピクチャとして参照される可能性のあるピクチャに対して行われる。
次に、同一性識別情報として、同一性カウント値(「第2の同一性識別情報」という)を用いる場合について説明する。なお、同一性カウント値は、例えば、復号化ピクチャにおける全ブロックがアンカー情報を用いるスキップ/ダイレクトモードであって、復号化ピクチャの復号化処理の途中でアンカーピクチャの切り替えが行われない場合に用いる。この点については後述する。
上述したように、同一性識別情報は、復号化対象ブロックの復号化処理で用いるアンカー情報が前ブロックで用いたアンカー情報との同一条件を満たしているか判別できる情報であればよく、復号化時だけでなく符号化時に生成することもできる。符号化時に同一性識別情報を生成する場合、生成した同一性識別情報を符号化ストリームに含める。画像復号化装置は、符号化ストリームから同一性識別情報を抽出して、抽出した同一性識別情報に基づき、アンカー情報の取得または前ブロックのアンカー情報の継続使用を行う。次に、符号化時にアンカー情報の同一性を判定して、同一アンカー識別情報を生成する場合について説明する。
図12は、画像符号化装置50の構成を示している。画像符号化装置50は、符号化処理を行う情報処理装置であり、アナログ/ディジタル変換部(A/D変換部)51、画面並び替えバッファ52、減算部53、直交変換部54、量子化部55、可逆符号化部56、蓄積バッファ57、レート制御部58を備えている。さらに、画像符号化装置50は、逆量子化部61、逆直交変換部62、加算部63、デブロッキングフィルタ64、フレームメモリ65、セレクタ66、イントラ予測部71、動き予測・補償部72、予測画像・最適モード選択部73を備えている。
図13は、動き予測・補償部72において、同一性識別情報の生成に関する構成部分を例示している。動き予測・補償部72は、動きベクトル検出部721、予測モード決定部722、予測モード記憶部723、アンカー情報生成/記憶部724、情報生成部725を有している。
次に、画像符号化処理動作について説明する。図14は、画像符号化処理動作を示すフローチャートである。ステップST51において、A/D変換部51は入力された画像信号をA/D変換する。
Cost(Mode∈Ω)=D+λ・R ・・・(1)
Ωは、当該ブロック乃至マクロブロックを符号化するための候補となる予測モードの全体集合を示している。Dは、予測モードで符号化を行った場合の復号画像と入力画像との差分エネルギー(歪み)を示している。Rは、直交変換係数や予測モード情報等を含んだ発生符号量、λは、量子化パラメータQPの関数として与えられるラグランジュ乗数である。
Cost(Mode∈Ω)=D+QPtoQuant(QP)・Header_Bit ・・・(2)
Ωは、当該ブロック乃至マクロブロックを符号化するための候補となる予測モードの全体集合を示している。Dは、予測モードで符号化を行った場合の復号画像と入力画像との差分エネルギー(歪み)を示している。Header_Bitは、予測モードに対するヘッダビット、QPtoQuantは、量子化パラメータQPの関数として与えられる関数である。
図15は、同一性識別情報として、同一性フラグを生成する場合の動作を示すフローチャートである。
[4-1.同一性識別情報の生成動作の比較]
表1は、画像符号化装置で同一性識別情報を生成する場合と、画像復号化装置で同一性識別情報を生成する場合とを比較結果を示している。
図17は、アンカー情報のデータ量を説明するための例を示している。例えばアンカーブロックが4×4ブロックで構成されており、DirectInferenceflagが「1」に設定されると、4隅のブロック(斜線で示すブロック)の動きベクトルと参照インデックスがアンカー情報として用いられる。ここで、水平方向動きベクトルを14ビット、垂直方向動きベクトルを12ビット、参照インデックスを6ビットとすると、1アンカーブロックのアンカー情報は(14+12+6)×4=128ビット(16バイト)となる。すなわち、前ブロックのアンカー情報を継続使用することでアンカー情報の読み出しを行う必要のないブロックがKブロック生じると、アンカー情報記憶部25からのアンカー情報の読み出しでは、16×Kバイト分のデータの読み出しを削減できる。なお、DirectInferenceflagが「0」に設定されると、4×4の全てのブロックの動きベクトルと参照インデックスがアンカー情報として用いられる。
明細書中において説明した一連の処理はハードウェア、またはソフトウェア、または両者の複合構成によって実行することが可能である。ソフトウェアによる処理を実行する場合は、処理シーケンスを記録したプログラムを、専用のハードウェアに組み込まれたコンピュータ内のメモリにインストールして実行させる。または、各種処理が実行可能な汎用コンピュータにプログラムをインストールして実行させることも可能である。
また、本発明は、衛星放送、ケーブルTV(テレビジョン)、インターネット、および携帯電話機などのネットワークメディアを介して受信する際に、あるいは、光、磁気ディスク、およびフラッシュメモリのような記憶メディア上で処理する際に用いられる画像符号化装置および画像復号装置に適用することができる。
Claims (11)
- アンカー情報を記憶するアンカー情報記憶部と、
復号化対象ブロックの復号化処理で用いるアンカー情報が前ブロックで用いたアンカー情報との同一条件を満たしていない場合に、前記アンカー情報記憶部から前記復号化対象ブロックに対応するアンカーブロックのアンカー情報を取得して、前記同一条件を満たす場合に、前記前ブロックで用いたアンカー情報を継続使用として、前記取得したアンカー情報または前記継続使用とされたアンカー情報を用いて復号化処理を行う画像復号化部と
を有する情報処理装置。 - 前記画像復号化部は、前記同一条件を満たしているか判別する同一性識別情報に基づき、前記アンカー情報の取得または前記前ブロックのアンカー情報の継続使用を行う請求項1記載の情報処理装置。
- 前記同一性識別情報は、アンカーピクチャとして用いられる前記画像復号化部で復号化処理済みのピクチャを対象として、該ピクチャのブロック毎に生成したアンカー情報に基づいて生成された情報である請求項2記載の情報処理装置。
- 前記同一性識別情報は、前ブロックとアンカー情報が同一と見なせるか否かを示す同一性フラグである請求項3記載の情報処理装置。
- 前記画像復号化部は、前記アンカー情報記憶部と別個に設けた記憶部に前記同一性フラグを記憶して、前記アンカー情報記憶部に、前記アンカーピクチャとして用いられるピクチャにおいて生成したアンカー情報を記憶させる請求項4記載の情報処理装置。
- 前記同一性識別情報は、前記アンカー情報が同一と見なせるブロックの連続数を示す同一性カウント値である請求項3記載の情報処理装置。
- 前記画像復号化部は、ブロック順に前記同一性カウント値と該同一性カウント値で同一と見なされたアンカー情報を対応させて前記アンカー情報記憶部に記憶させる請求項6記載の情報処理装置。
- 前記同一性識別情報は、前記復号化対象ブロックの符号化時に用いたアンカー情報と前ブロックの符号化で用いたアンカー情報に基づいて生成された情報である請求項2記載の情報処理装置。
- 前記同一性識別情報は、前記復号化対象ブロックの符号化時に用いたアンカー情報と前ブロックの符号化で用いたアンカー情報において、動きベクトルの差が予め設定した閾値以下であるとき2つのアンカー情報を同一とみなして生成されており、前記符号化によって生じた画質の劣化が所定レベルを超えるとき、前記2つのアンカー情報を同一とみなさないで生成されている請求項8記載の情報処理装置。
- 前記画像復号化部は、画像データの符号化ストリームから前記同一性識別情報を抽出する請求項8記載の情報処理装置。
- 復号化対象ブロックの復号化処理で用いるアンカー情報が前ブロックで用いたアンカー情報との同一条件を満たしていない場合に、アンカー情報を記憶するアンカー情報記憶部から前記復号化対象ブロックに対応するアンカーブロックのアンカー情報を取得する工程と、
前記同一条件を満たす場合に、前記前ブロックで用いたアンカー情報を継続使用する工程と、
前記取得したアンカー情報または前記継続使用するアンカー情報を用いて復号化処理を行う工程と
を有する情報処理方法。
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US13/699,091 US20130089146A1 (en) | 2010-06-25 | 2011-06-22 | Information processing apparatus and information processing method |
CN201180030088.6A CN102986225B (zh) | 2010-06-25 | 2011-06-22 | 信息处理设备和信息处理方法 |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2010144907A JP5387520B2 (ja) | 2010-06-25 | 2010-06-25 | 情報処理装置と情報処理方法 |
JP2010-144907 | 2010-06-25 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2011162299A1 true WO2011162299A1 (ja) | 2011-12-29 |
Family
ID=45371475
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2011/064290 WO2011162299A1 (ja) | 2010-06-25 | 2011-06-22 | 情報処理装置と情報処理方法 |
Country Status (4)
Country | Link |
---|---|
US (1) | US20130089146A1 (ja) |
JP (1) | JP5387520B2 (ja) |
CN (1) | CN102986225B (ja) |
WO (1) | WO2011162299A1 (ja) |
Families Citing this family (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2017069617A (ja) * | 2015-09-28 | 2017-04-06 | ルネサスエレクトロニクス株式会社 | 半導体装置および画像符号化方法 |
CN108347602B (zh) * | 2017-01-22 | 2021-07-30 | 上海澜至半导体有限公司 | 用于无损压缩视频数据的方法和装置 |
US11388432B2 (en) * | 2018-12-10 | 2022-07-12 | Qualcomm Incorporated | Motion estimation through input perturbation |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2008141782A (ja) * | 2004-11-26 | 2008-06-19 | Matsushita Electric Ind Co Ltd | 復号化回路、復号化装置、及び復号化システム |
JP2009055519A (ja) * | 2007-08-29 | 2009-03-12 | Sony Corp | 符号化処理装置、符号化処理方法、復号処理装置、及び、復号処理方法 |
WO2010137086A1 (ja) * | 2009-05-26 | 2010-12-02 | パナソニック株式会社 | 動画像処理装置および動画像処理方法 |
Family Cites Families (12)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5508744A (en) * | 1993-03-12 | 1996-04-16 | Thomson Consumer Electronics, Inc. | Video signal compression with removal of non-correlated motion vectors |
US6163576A (en) * | 1998-04-13 | 2000-12-19 | Lsi Logic Corporation | Video encoder having reduced memory bandwidth requirements |
US7020201B2 (en) * | 2002-11-20 | 2006-03-28 | National Chiao Tung University | Method and apparatus for motion estimation with all binary representation |
CN1922884B (zh) * | 2004-02-20 | 2012-05-23 | 三叉微系统(远东)有限公司 | 视频解码方法 |
US7933336B2 (en) * | 2004-11-30 | 2011-04-26 | Humax Co., Ltd. | Image down-sampling transcoding method and device |
WO2007081177A1 (en) * | 2006-01-12 | 2007-07-19 | Lg Electronics Inc. | Processing multiview video |
US8270492B2 (en) * | 2006-05-12 | 2012-09-18 | Panasonic Corporation | Moving picture decoding device |
CN101455088B (zh) * | 2006-05-24 | 2011-05-11 | 松下电器产业株式会社 | 运动图像解码装置 |
JP4427086B2 (ja) * | 2006-12-27 | 2010-03-03 | パナソニック株式会社 | 動画像復号化装置 |
CN101828397A (zh) * | 2007-10-15 | 2010-09-08 | 汤姆森特许公司 | 编码和解码信号的装置和方法 |
US8619861B2 (en) * | 2008-02-26 | 2013-12-31 | Microsoft Corporation | Texture sensitive temporal filter based on motion estimation |
WO2010131903A2 (en) * | 2009-05-12 | 2010-11-18 | Lg Electronics Inc. | Method and apparatus for processing a video signal |
-
2010
- 2010-06-25 JP JP2010144907A patent/JP5387520B2/ja not_active Expired - Fee Related
-
2011
- 2011-06-22 CN CN201180030088.6A patent/CN102986225B/zh not_active Expired - Fee Related
- 2011-06-22 WO PCT/JP2011/064290 patent/WO2011162299A1/ja active Application Filing
- 2011-06-22 US US13/699,091 patent/US20130089146A1/en not_active Abandoned
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2008141782A (ja) * | 2004-11-26 | 2008-06-19 | Matsushita Electric Ind Co Ltd | 復号化回路、復号化装置、及び復号化システム |
JP2009055519A (ja) * | 2007-08-29 | 2009-03-12 | Sony Corp | 符号化処理装置、符号化処理方法、復号処理装置、及び、復号処理方法 |
WO2010137086A1 (ja) * | 2009-05-26 | 2010-12-02 | パナソニック株式会社 | 動画像処理装置および動画像処理方法 |
Also Published As
Publication number | Publication date |
---|---|
US20130089146A1 (en) | 2013-04-11 |
JP2012010147A (ja) | 2012-01-12 |
JP5387520B2 (ja) | 2014-01-15 |
CN102986225B (zh) | 2016-02-17 |
CN102986225A (zh) | 2013-03-20 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
KR101914896B1 (ko) | 화상 처리 장치와 화상 처리 방법 | |
WO2012017858A1 (ja) | 画像処理装置と画像処理方法 | |
WO2011155364A1 (ja) | 画像復号化装置と画像符号化装置およびその方法とプログラム | |
WO2012063878A1 (ja) | 画像処理装置と画像処理方法 | |
WO2013001945A1 (ja) | 画像処理装置と画像処理方法 | |
KR20140019388A (ko) | 화상 처리 장치 및 화상 처리 방법 | |
KR20170055554A (ko) | 화상 처리 장치, 화상 처리 방법 및 기록 매체 | |
WO2011155332A1 (ja) | 画像復号化装置と画像符号化装置およびその方法とプログラム | |
WO2012063604A1 (ja) | 画像処理装置と画像処理方法 | |
WO2010035732A1 (ja) | 画像処理装置および方法 | |
US20110229049A1 (en) | Image processing apparatus, image processing method, and program | |
WO2011155331A1 (ja) | 画像復号化装置と画像符号化装置およびその方法とプログラム | |
JP2013038623A (ja) | 画像符号化装置と画像符号化方法およびプログラム | |
JP2013150164A (ja) | 符号化装置および符号化方法、並びに、復号装置および復号方法 | |
WO2010038858A1 (ja) | 画像処理装置および方法 | |
JP5387520B2 (ja) | 情報処理装置と情報処理方法 | |
WO2018235405A1 (ja) | 画像処理装置と画像処理方法 | |
WO2012056924A1 (ja) | 画像処理装置と画像処理方法 | |
US20130182967A1 (en) | Image processing device and image processing method | |
WO2013065527A1 (ja) | 画像処理装置と画像処理方法 | |
WO2012077533A1 (ja) | 画像復号化装置と動きベクトル復号化方法、画像符号化装置と動きベクトル符号化方法 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
WWE | Wipo information: entry into national phase |
Ref document number: 201180030088.6 Country of ref document: CN |
|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 11798181 Country of ref document: EP Kind code of ref document: A1 |
|
WWE | Wipo information: entry into national phase |
Ref document number: 13699091 Country of ref document: US |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 11798181 Country of ref document: EP Kind code of ref document: A1 |