RU2792865C2 - Method for adaptive update of unavailable reference frame blocks using explicit and implicit signaling - Google Patents

Abstract

FIELD: video compression.

SUBSTANCE: invention relates to the field of video compression, in particular, it is directed to a method for adaptively updating blocks for an unavailable reference image. The technical result is to increase the efficiency of video coding. A decoding device is provided, comprising a circuit configured to receive a current block and to determine whether the current block is a block, to update an unavailable reference frame based on the current block, and update the unavailable key frame using the current block. The signaling information contains a sequence parameter set (SPS) indicating that a mode is available where a part of the reference picture can be modified.

EFFECT: increase in the efficiency of video coding.

3 cl, 9 dwg

Description

The field of technology to which the invention belongs

The present invention generally relates to the field of video compression. In particular, the present invention is directed to a method for adaptively updating blocks for an unavailable reference picture.

State of the art

A video codec may contain electronic circuitry or software that compresses or decompresses digital video. It can convert uncompressed video to compressed format or vice versa. In the context of video compression, a device that compresses the video (and/or performs some functions during compression) may commonly be referred to as an encoder, and a device that decompresses the video (and/or performs some functions during decompression) may be referred to as a decoder. .

The compressed data format may follow standard video compression specifications. This compression may be more lossy in the sense that some of the information present in the original video is missing from the compressed video. As a result, the decompressed video may be of lower quality than the original uncompressed video because the decompressed video will not contain enough information to accurately reconstruct the original video.

There can be complex relationships between video quality, amount of data used to represent the video (e.g. determined by bit rate), complexity of encoding and decoding algorithms, sensitivity to data loss and errors, ease of editing, random access, delay from one end of the line from the other and other similar characteristics.

The motion compensation procedure may be a prediction of a video frame or part thereof based on a reference frame, such as a previous and/or future frame, by taking into account the motion of the video camera and/or objects in the video. This can be used when encoding and decoding video data for video compression, such as encoding and decoding using Motion Picture Experts Group MPEG-2 standards (also called advanced video coding (AVC) standards. ) and H.264). Motion compensation may describe images in terms of transforming a reference image into a current image. The reference picture may precede the current picture, may be future relative to the current picture, or may be a long term reference (LTR) frame. When images can be accurately synthesized based on previously transmitted and/or stored images, compression efficiency can be improved.

Today's standards, such as the H.264 and H.265 standards, allow frames, such as long-term anchor frames, to be updated by signaling a newly decoded frame so that it can be stored and made available as a reference frame. Such updates are signaled from the encoder and update the entire frame. However, updating the entire frame can be costly, particularly when only a small portion of the static background has changed. Partial frame updates are possible, but complex and computationally expensive procedures are often used for such frame updates.

Disclosure of the essence of the invention

In one aspect, the decoder comprises circuitry configured to receive a data bitstream containing a current encoded block, determine the decoded current block, determine that an unavailable reference block update mode is activated in the data bitstream for the current encoded block, and update the unavailable reference frame. using the decoded current block.

According to another aspect, the method comprises receiving, by the decoder, a data bitstream containing the current encoded block. This method comprises determining the decoded current block. The method comprises determining that an unavailable reference block update mode is activated in the data bitstream for the current encoded block. This method comprises updating an unavailable reference frame using the decoded current block.

According to another aspect, the decoder comprises circuitry configured to receive a current block, mark the current block for updating an unavailable reference frame, and update the unavailable reference frame using said current block.

According to another aspect, the method comprises receiving, by the decoder, the current block. This method comprises marking the current block to update an unavailable key frame based on the current block. This method comprises updating an unavailable key frame using the current block.

Details of one or more embodiments of the subject matter of the present invention described herein are set forth in the accompanying drawings and in the description below. Other features and advantages of the subject matter described here will be apparent from the description and drawings, as well as from the claims.

Brief description of the drawings

For purposes of illustrating the present invention, these drawings show aspects of one or more embodiments of the present invention. However, it should be understood that the present invention is not limited to exactly the configurations and tools shown in the drawings, in which:

Fig. 1 is a logic diagram illustrating an example procedure for updating blocks of an unavailable reference (UR) frame, where this UR frame is partially updated by the decoder and based on the decoded current block;

Fig. 2 is a logic diagram illustrating an example of a UR frame block update procedure, where the UR frame is partially updated by the decoder and based on the decoded current block;

Fig. 3 is a block diagram illustrating an example of a frame containing a current block X and three adjacent blocks having associated motion vectors using a UR frame for motion estimation;

Fig. 4 is a block diagram illustrating an example of a frame containing blocks not using a UR frame as a reference and blocks using a UR frame as a reference;

Fig. 5 is a system block diagram illustrating an example of a decoder capable of decoding a data bit stream with block update of a UR frame;

Fig. 6 is a flow diagram illustrating an example of a video coding procedure with UR frame block updates in accordance with some aspects of the subject matter of the present invention;

Fig. 7 is a flowchart illustrating an example of a video coding procedure with UR frame block updates in accordance with some aspects of the subject matter of the present invention to reduce coding complexity while increasing compression efficiency;

Fig. 8 is a system block diagram illustrating an example of a video decoder, where the device is capable of signaling the decoder side to update blocks of a UR frame;

And

Fig. 9 is a block diagram of a computer system that may be used to implement any of one or more of the methods described herein and to implement any of one or more parts of those methods.

The drawings are not necessarily drawn to scale and may be illustrated with imaginary lines, schematic representations and partial views. In some cases, details that are not necessary to understand embodiments of the invention or display other details that are difficult to understand may be omitted. Like reference designators in the various drawings are assigned to like elements.

Implementation of the invention

Some embodiments of the subject matter of the present invention contain approaches to performing partial updates to an unavailable reference (UR) frame and/or picture that is used to create predictive frames and/or pictures in one or more groups of pictures (GOP) ), but which is not represented by itself in the video image. A frame marked as a UR frame in the video bitstream may be available for use as a reference frame until it is explicitly excluded by signaling in the data bitstream. The use of UR frames can improve the performance of prediction and compression in scenes that have a static background for a long period of time (for example, the background in the case of a video conference or in the case of video surveillance in a parking lot). However, over time, the background of the scene gradually changes (for example, cars parked in empty spaces become part of the background scene). Partial updates of UR frames may be signaled explicitly in the data bitstream by using an unavailable reference block update mode which, when enabled for a particular current block, may indicate that the decoded current block is being used to update spaced in that same the location of the pixels within the UR frame. Such partial updates of the UR frame can improve prediction without requiring the entire UR frame to be updated, and therefore reduce residual error and thereby improve compression performance. Moreover, by explicitly signaling UR block updates, such UR block updates can be implemented in a simple manner and without the use of inadequate and complicated procedures.

Alternatively or in addition, a partial update of the UR frame may be signaled implicitly, such that neither full nor reduced signaling is explicitly required compared to some alternative approaches. The implicit signaling approach may indicate that the decoded current block may be used to update the same location spatially located pixels in the UR frame. Such partial updates of the UR frame can improve prediction without requiring the entire UR frame to be updated, and therefore reduce the residual error and thereby improve compression performance. Moreover, by implicitly signaling UR block updates, header overhead can be reduced, thereby lowering the data bit rate and improving compression. In addition, in some embodiments, updates to the UR frame can be performed in a fairly simple manner and without the use of inadequate and complex procedures.

In some embodiments, implicit video block signaling identifies instances in which the context of the current and neighboring blocks is used to implicitly determine which blocks are updated in the UR frame (eg, which portions of the UR frame in question should be updated). Let us now turn to Fig. 1, which is a flowchart illustrating an example of a UR frame block update procedure 100 in which a decoder partially updates a UR frame based on the decoded current block. Such updates to UR blocks can improve prediction without requiring the entire UR frame to be updated, and therefore can reduce residual error and thereby improve compression performance.

At step 105, and still referring to FIG. 1, the decoder receives a data bitstream containing the current encoded block. The data bitstream may contain, for example, data found in the data bitstream input to the decoder during data compression. This data bitstream may contain information necessary to decode the video. The procedure for receiving the stream may comprise extracting and/or parsing the block in question and the associated signaling information from the data bitstream. In some embodiments, the current encoded block may contain a coding tree unit (CTU), a coding unit (CU), or a prediction unit (PU).

At step 110, continuing to refer to FIG. 1, the decoded current block can be determined. For example, the received current encoded block may be decoded using inter prediction, for example. Decoding by inter-frame prediction may be decoding using a previous frame, a future frame, or a UR frame as a reference to calculate a prediction result, which may be combined with a residual included in the data bit stream.

At step 115, continuing with reference to FIG. 1, it can be determined whether the unavailable reference block update mode in the data bit stream is activated for the current encoded block. An indication of the update mode of the UR-block may be included in the header of the data bit stream and/or in any field in this header. For example, it may be determined that the UR-block update mode field in the header of the data bitstream is enabled. Updated UR frame block signaling may be selectively activated using a header such as a picture parameter set (PPS) and/or a sequence parameter set (SPS). One of the fields, such as the UR_BLOCK_UPDATE field, may be true or false (eg, 0 or 1) without limitation.

Still referring to FIG. 1, if the UR block update mode is enabled (eg, if the UR_BLOCK_UPDATE field in the header is set), at step 120, the decoder may update the unavailable reference frame using the decoded current block. Such an update may be the implementation of updates to UR blocks that contain pixels of the UR frame located in space at the same place as the current block for which the update mode of the UR frame block is activated, so that these UR blocks are updated (for example, modified ) using the decoded current block. In some implementations, block update mechanisms may be explicit or implicit. For example, a portion (eg, block) of a UR frame may be updated by updating pixels (eg, luminance values) in the UR frame using pixel values of the decoded current block co-located with the UR frame pixels being updated; in other words, the procedure for updating the UR frame may comprise replacing the luminance values in the UR frame with luminance values located in the same place in space with them from the decoded current block. In some implementations, the UR frame may be updated according to another mechanism, such as updating a portion of the UR frame using the average of the co-located pixel values of the UR frame and the pixel values of the current decoded block. Those skilled in the art, after studying the present invention in its entirety, will be aware of a variety of other mechanisms that may be used. In some implementations, the update may comprise updating multiple blocks in a UR frame using multiple decoded blocks, such that the update may be performed using any of the methods described above for updating a UR frame using the decoded current block. The update procedure may comprise, without limitation, generating a block and/or frame of default chroma and/or luminance values and replacing and/or updating them from one or more decoded blocks using any of the procedure and/or procedure steps described above; the update procedure may comprise generating a UR frame, which may be done by creating a UR frame from default values and updating those default values as described above.

Continuing with reference to FIG. 1, for subsequent current blocks, the updated UR frame may be used as a reference frame for inter-frame prediction. For example, a second coded block may be received. It may be determined whether the inter-prediction mode is enabled for the second coded block. The second decoded block may be determined using the updated UR frame as the reference frame and in accordance with the inter prediction mode. For example, the procedure for decoding by inter-frame prediction may comprise using the updated UR frame as a reference to calculate a prediction result that may be combined with a residual included in the data bitstream.

Continuing with reference to FIG. 1, UR block updates may be available for each current block during the decoding process. In some embodiments, block updates may be available for each current block during the decoding procedure. In some implementations, UR block updates may be implicitly omitted for frames encoded with intra-prediction. For example, the data bitstream may contain a second coded block in a frame different from the frame in which the first coded block resides; the coding mode for the second coded block may comprise intra prediction. The update of the UR block may be skipped in response to determining that the second current coded block is an intra prediction coded block. Such an omission may include, for example, not updating the UR frame or determining if a header field, such as the UR_BLOCK_UPDATE field, is set.

In some embodiments, and referring again to FIG. 1, if the UR_BLOCK_UPDATE field is set in the header, the decoder can wait for explicit block update signaling bits. With this approach, a coded video block can be designated by signaling in a block header in the video data bitstream as the block to be updated in the UR frame. In this case, the block located in the same place in the UR frame may be replaced by the block reported by the signaling; this updated UR frame may be used for future motion estimation purposes until this UR frame is subsequently updated, eg.

On FIG. 2 is a flow diagram illustrating an example variant of procedure 200 using implicit signaling of UR frame block updates such that during this procedure the UR frame is partially updated by the decoder and based on the decoded current block. Implicit signaling may cover cases where the context of the current and adjacent blocks is used to implicitly determine which blocks to update in a UR frame, as opposed to explicit signaling regarding the update mode of a UR block in a data bitstream, such as as in the data bit stream header. Such updates to UR blocks using implicit signaling can improve prediction and reduce signaling overhead, leading to improved data bit rates and improved compression. Similarly, by performing the update of the UR blocks without requiring the entire UR frame to be updated, the residual error can be reduced, and thus the compression performance can be improved.

At step 205, and still referring to FIG. 2, the decoder receives the current block. This current block may be included in the data bitstream received by the decoder. The data bitstream may comprise, for example, data found in a bitstream input to a decoder when data compression is used. The data bitstream may contain information necessary to decode the video. The procedure for receiving the stream may comprise extracting and/or parsing the block in question and the associated signaling information from the data bit stream. In some implementations, the current encoded block may comprise a coding tree unit (CTU), a coding unit (CU), and/or a prediction unit (PU).

Continuing with reference to FIG. 2, in some implementations, the data bit stream for the current block may not explicitly signal the update mode of the unavailable reference block. Likewise, in some implementations, the decoder may determine, using the data bitstream, whether there is explicit signaling of an unavailable reference block update mode or not in the data bitstream for the current block. In some implementations, the decoder may determine if a field in the data bitstream header indicates that the unavailable reference block update mode is disabled or not present.

At 210, and continuing with FIG. 2, the current block may be marked to update the UR frame based on this current block; this current block may be marked based on or in response to determining that the current block is implicitly signaled for use in updating blocks of the UR frame. The marking procedure may include, without limitation, assigning a value to some variable indicating that the current block is to be used to update the UR frame; for example, a Boolean variable associated with the current block and configured to indicate whether the current block should be used to update the UR frame may be set to "true" or "1" to indicate a mark.

For example, still referring to FIG. 2, the current block may be marked for updating when the current block uses the UR frame as a reference frame for inter-prediction and contains the application of light compensation; marking may be done in response to and/or based on this determination. Light compensation can be used to improve prediction during video encoding. Light compensation can improve prediction when the block pixels change due to light changes (eg, a passing cloud reduces the light). A block that uses a UR frame as a reference frame and applies light compensation may be defined as an update block and marked accordingly to update the UR frame.

As another example, and continuing with reference to FIG. 2, the current block may be marked to update the UR frame if and based on which of the neighbors of that block is using the motion vector of the UR frame as a reference. In a representative (eg typical) video, a portion of the scene's background is updated over time. When such a change occurs, the changed area may be encoded as an intra prediction block if no similar block is detected. In such cases, all blocks except for the newly modified area are likely to use the UR frame as the reference frame. A block having multiple reference blocks with reference motion vectors of the UR frame may be marked for implicit update of the UR frame. Accordingly, the probability that the current block is an update UR block increases as the number of blocks adjacent to the current block using the UR frame increases; this probability can be calculated and compared with the threshold level, the fact that this probability exceeds the threshold level may indicate that the current block is an update for the UR block. If the current block is encoded using intra-frame prediction, a smaller number of neighboring blocks with a UR reference may signal an update for the UR block; for example, when compared with a threshold as above, a lower threshold level may be used. If the current block is encoded using inter-picture prediction, more neighboring blocks with a UR reference may implicitly signal an update for the UR block, for example, by using a larger threshold level. Accordingly, as another non-exhaustive example, the current block may be marked to update the UR frame in response to and/or based on a determination that two or more neighboring blocks use the motion vector of the UR frame for inter-frame prediction.

Still referring to FIG. 2, in some implementations, update candidate blocks may be determined after the entire frame has been decoded, so that all of the block's neighbors may be considered. For example, a non-UR-supported block surrounded by UR-supported blocks on two or more sides may be a candidate for UR frame updates. As another example, if the current intra-prediction coding block and several blocks adjacent to it use the UR frame as a reference and have a motion vector of (0,0) (e.g., zero motion vector), then the current block may be marked to update the UR frame. As yet another example, if the current block is encoded using intra-prediction and several neighboring blocks use the UR frame as a reference and have the same or similar motion vectors (for example, the difference between these motion vectors may be less than a predetermined value), then the current block can be marked to update the UR frame. As a further example, if the current block has at least three adjacent blocks using the UR frame as a reference frame for inter-prediction, the current block may be marked to update the UR frame.

In some embodiments, and continuing with reference to FIG. 2, an update for a UR frame may be defined at a coding unit (CU) level. If all neighboring CUs on at least three sides use the UR foot, all blocks of the current CU may be marked to update the UR frame. In some implementations, an update for a UR frame may be defined at the coding tree unit (CTU) level. If all adjacent CTUs on at least three sides use the UR foot, all units of the current CTU that do not use the UR foot can be marked for updating. On FIG. 3 is a block diagram illustrating an exemplary version of a frame 300 comprising a current block X and three neighboring blocks having associated motion vectors MV1, MV2, MV3 using a UR frame for motion estimation. In the illustrated example, the current block X may be marked to update the UR frame.

On FIG. 4 is a block diagram illustrating another example of a frame 400 variant comprising blocks labeled X and not using a UR frame as a reference, and L blocks using a UR frame as a reference; blocks marked as X may be marked to update the UR frame.

Referring again to Fig. 2, at step 215, the UR frame may be updated using the current block. The update procedure may comprise performing UR block updates where pixels of the UR frame co-located in space with the current block are updated (eg, modified) using the current block. In some implementations, block update mechanisms may be explicit or implicit. For example, a portion (eg, block) of a UR frame may be updated by updating (eg, replacing) pixels (eg, brightness values) in the UR frame with pixel values of the current block co-located with the updated UR frame pixels. In some embodiments, the UR frame may be updated according to another mechanism, such as updating a portion of the UR frame using an average of the co-located original UR frame pixel values and the current block pixel values. Those skilled in the art, after studying the present invention in its entirety, will be aware of various other mechanisms that may be used. In some implementations, the update may be an update of multiple blocks of a UR frame using multiple decoded blocks, which may be done using any of the methods described above for updating a UR frame using a single decoded current block. The update procedure may include, without limitation, generating a block and/or frame of default values for the chrominance and/or luminance component and replacing and/or updating them based on one or more decoded blocks using any procedure and/or steps of the procedure, described above; the update procedure may comprise generating a UR frame, which may be done by creating a UR frame from default values and updating those default values as described above.

For subsequent current blocks, the updated UR frame may be used as a reference frame for inter-frame prediction. For example, a second coded block may be received. It may be determined whether the inter-frame prediction mode is activated for this second coded block. Then, the second decoded block may be determined using the updated UR frame as the reference frame and in accordance with the inter prediction mode. For example, the procedure for decoding by inter-frame prediction may comprise using the updated UR frame as a reference for computing a prediction result, which may be combined with a residual contained in the data bitstream.

On FIG. 5 is a block diagram of an example system illustrating an example decoder 500 capable of decoding a data bit stream 504 with UR frame block updates. UR frame block updates may include, without limitation, explicitly signaled UR frame block updates, for example and without limitation as described above with reference to FIG. 1. UR frame block updates may include, without limitation, signaling implicitly signaled UR frame block updates (e.g., UR frame block updates determined from the context of the current and/or neighboring blocks), for example, as described. above with reference to Fig. 2-4. The decoder 500 includes an entropy decoding processor 508, an inverse quantization and inverse transform processor 512, a deblocking filter 516, a frame buffer 520, a motion compensation processor 524, and an intra prediction processor 528. In some implementations, data bit stream 504 may contain parameters (eg, a field in the data bit stream header) signaling the update mode of the UR frame block. Alternatively, or in addition, data bit stream 504 may not explicitly signal the block update mode of the UR frame for the current block; the data bitstream may contain other parameters. The motion compensation processor 524 may reconstruct the pixel information using the UR frame and update the UR frame according to the update mode of the UR frame block. For example, when explicitly signaling the block update mode of the UR frame for the current block, values (eg, luminance values) in the UR frame may be replaced by pixel values of the current block co-located with the values of the block being updated. When block updates are implicitly signaled, motion compensation processor 524 may mark the current block for UR frame update (eg, based on whether UR frame update is explicitly signaled) and update the UR frame accordingly. For example, when updating blocks of a UR frame for the current block is implicitly signaled, pixels (eg, luminance values) in the UR frame may be replaced with co-located pixel values from the current block.

During operation, the data bitstream 504 may be received by the decoder 500 and fed to an entropy decoding processor 508, which may entropy decode the data bitstream and convert it to quantized coefficients. The quantized coefficients may be provided to an inverse quantization and inverse transform processor 512, which may perform inverse quantization and inverse transformation to produce a residual signal, which may be added to the output of motion compensation processor 524 or intra prediction processor 528 in accordance with the processing mode. . The output of motion compensation processor 524 and intra prediction processor 528 may comprise predictive blocks based on a pre-decoded block or UR frame. The sum of the predicted block and the corresponding residual may be processed by a deblocking filter 516 and stored in a frame buffer 520 . For the particular block in question (eg, a CU or a PU), when the data bit stream 504 explicitly and/or implicitly signals that the UR frame block update mode is activated, the motion compensation processor 524 may update the UR frame, which may reside in the frame buffer 520 to update the pixels (eg, luminance values) in the UR frame with the same location pixel values of the current block.

In some implementations, decoder 500 may include a UR frame block update processor 532 that can generate a UR frame update based on the current block and generate UR frame pixel values for inter-frame prediction procedures; this can be implemented in accordance with any steps of the procedures described in the present invention. The processor 532 for updating the blocks of the UR frame can directly affect the motion compensation. Further, processor 532 for updating blocks of the UR frame may receive information from processor 528 for intra prediction, such as when the current block is an intra prediction block.

On FIG. 6 is a flow diagram illustrating an example variant of a procedure 600 for encoding video with block updates of a UR frame using explicit signaling in accordance with some aspects of the subject matter of the present invention, which can reduce coding complexity while increasing compression efficiency. At 605, the video frame may be subjected to an initial block segmentation, eg, using a tree structured macroblock partitioning scheme, which may include splitting the image frame into CTUs and CUs. At 610, a block may be selected to update a portion of the UR frame. This block may be selected, for example, based on the frequency content and/or measure of motion in the block in question compared to one or more blocks located at the same location that are close in time (for example, adjacent frames in time, for example, in a given number of frames relative to the current frame). The definition of a measure of frequency content may comprise a definition using a transformation matrix, such as, without limitation, a discrete cosine transformation matrix. The procedure for determining a measure of frequency content for a block may comprise determining using a generalized discrete cosine transform matrix. For example, when the blocks as described above are 4 x 4 pixel blocks, the generalized discrete cosine transform matrix may be a discrete cosine transform II matrix taking the form:

T=

, и c равно

.where a is ½, b is

, and c is equal to

.

In some implementations, an integer approximation of the transformation matrix can be used, which can be used to efficiently implement hardware and software. For example, when the blocks as described above are 4x4 pixel blocks, the generalized discrete cosine transform matrix may be a discrete cosine transform II matrix taking the form:

. _TINT =

.

For a block B _i , the frequency content of this block can be calculated using:

F _Bi =TxB _i xT',

where T' is the transposed cosine transform matrix T, B _i is a block represented as a matrix of numerical values corresponding to pixels in the block, such as a 4x4 matrix representing a 4 x 4 block as described above, and the operator x denotes matrix multiplication . The procedure for said selection may comprise identifying, according to the metric rule, that the block is to be used to update a portion of the UR frame at the decoder. This metric rule may comprise comparing a measure of texture and/or motion, including, without limitation, any measure of frequency content or the like described above, to a given and/or stored threshold level. As a non-limiting example, when the parameter FBi, as defined above, exceeds a certain value, an update procedure may be triggered, for example because very high frequencies may not be perceived. The selection may alternatively or in addition be made using any of the procedure and/or steps of the procedure described below with reference to FIG. 7. At 615, a block may be encoded and inserted into the data bit stream.

At 620, UR frame block update parameters may be explicitly determined and included in the data bitstream to signal that the UR frame block update mode is enabled for the current block. For example, a field in the PPS set or in the SPS set may be set (eg, enabled). For example, a field such as the UR_BLOCK_UPDATE field may be set to indicate that the block update mode of the UR frame is enabled for the current block.

On FIG. 7 is a flow diagram illustrating an exemplary embodiment of a video encoding procedure 700 using implicit signaling UR frame block update according to some aspects of the subject matter of the present invention, which can reduce encoding complexity while increasing compression efficiency. At 705, the video frame may be subjected to an initial block segmentation, eg, using a tree-structured partitioning scheme, which may comprise splitting the image frame into CTUs and CUs.

At 710, and still referring to FIG. 7, a block may be selected to update part of the UR frame; this block may be selected, for example, based on an explicit update condition of the UR frame. The selection procedure may comprise identifying, according to some condition, that a certain block is to be used to update a portion of the UR frame at the decoder. These conditions may include, for example, when the current block uses a UR frame as a reference for inter-frame prediction, as well as the application of light compensation. As another example, these conditions may relate to which of the block's neighbors use the UR frame's motion vector as a reference, as is the case when a block has multiple neighboring blocks with the UR frame's motion vector as a reference if the current the block is encoded using inter-prediction, and two or more blocks adjacent to the current block use the motion vector of the UR frame for inter-prediction. As another example, if a non-UR-reference box is surrounded by UR-reference boxes on two or more sides, this may mean that the box in question should be used to update part of the UR frame. As another example, if the current block is encoded using intra-prediction and several adjacent blocks use the UR frame as a reference and have a motion vector equal to (0,0) (e.g., zero motion vector), the current block may be identified as the block to be used to update part of the UR frame. As yet another example, if the current block is encoded using intra-prediction and several neighboring blocks use the UR frame as a reference and have the same or similar motion vector, (for example, the motion vector may be different from the vectors of other blocks less than a predetermined amount), the current block may be identified as the block to be used to update the frame UR.

As another example, and continuing with reference to FIG. 7, in some implementations, a block may contain a coding unit (CU), and the condition for determining may be a condition specifying that if all neighboring CUs on at least three sides use a UR reference, all blocks of the current CU units. As another example, in some implementations, a block may contain a coding tree unit (CTU), and the condition may be that if all adjacent CTUs on at least three sides use a UR-reference, all blocks of the current CTU, which do not use the UR foot can be selected as the blocks used to update the UR frame.

At 730, the selected block may be encoded and included in the data bit stream. The encoding procedure may include, for example, using an inter prediction mode and an intra prediction mode. In some implementations, the block update parameters of the UR frame may not be explicitly included in the data bit stream for the current block.

On FIG. 8 is a system block diagram illustrating an exemplary embodiment of a video encoder 800 capable of explicitly and/or implicitly signaling UR frame block updates to the decoder side. An example video encoder 800 receives input video 804, which may be initially segmented or split according to some processing scheme, such as a macroblock partitioning scheme according to a tree structure (eg, square tree plus binary tree). An example of a macroblock partitioning scheme according to a tree structure may comprise partitioning an image frame into larger block elements called coding tree units (CTUs). In some embodiments, each CTU may be further divided one or more times into a number of sub-blocks called coding units (CUs); the result of such a split may contain a group of sub-blocks, which may be referred to as prediction units (PUs). Transform units (TU) may also be used. The term "transformation unit" as used herein means a base unit to which a frequency transformation is applied, so that this base unit may be, without limitation, a sub-unit of a CU suitable for performing frequency transformation operations. As a non-exhaustive example, one TU may contain one transform block from luminance samples and two corresponding transform blocks from chrominance samples of the image, as well as syntax structures used to transform samples from these transform blocks, where "transform block" is defined as having the size MxN is a rectangular block of samples obtained as a result of the transformation during the decoding procedure.

Still referring to FIG. 8, an exemplary video encoder 800 includes an intra prediction processor 815, a motion estimation/compensation processor 812 (also referred to as an inter prediction processor) capable of supporting UR frame block updates in an explicit or implicit form in the decoder, a processor 816 for transform/quantizer, an inverse quantizer/inverse transform processor 820, an in-loop filter 824, a decoded image buffer 828, and an entropy coding processor 832. In some embodiments, motion estimation/compensation processor 812 may determine for the current block that the UR frame should be updated at the decoder, eg, in accordance with one or more of the conditions described above with reference to FIG. 6 and/or 7; the motion estimation/compensation processor 812 may set parameters to explicitly signal that the UR frame block update mode is activated. The data bit stream parameters signaling the update modes of the UR frame block may be input to the entropy encoding processor 832 for inclusion in the output data bit stream 836 . Alternatively, or in addition, the motion estimation/compensation processor 812 may update the UR frame at the encoder using the current block, so that no explicit signaling regarding updating the blocks of the UR frame may be included in the data bitstream. . Data bit stream parameters may be input to entropy encoding processor 832 for inclusion in output data bit stream 836 . The encoder-side portion of the UR frame may be updated for use by motion estimation/compensation processor 812 to encode additional blocks that may use the UR frame as a reference for inter-frame prediction.

During operation and continuing to refer to FIG. 8, for each block included in a frame of input video 804, it can be determined whether that block should be processed through intra-frame prediction or using motion estimation/compensation. The block may be directed to a processor 808 for intra prediction or to a processor 812 for motion estimation/compensation. If the block is to be processed by intra prediction, the intra prediction processor 808 may perform processing to output a prediction result. If the block is to be processed using motion estimation/compensation, the motion estimation/compensation processor 812 may perform processing including using the encoder-side UR frame as an inter prediction reference, if applicable.

Still referring to FIG. 8, the residual may be generated by subtracting the prediction result from the input video. The remainder may be received by a transform/quantization processor 816, which may perform a transform (eg, a discrete cosine transform (DCT)) to obtain coefficients that can be quantized. The quantized coefficients and any associated signaling information may be passed to an entropy encoding processor 832 to entropy encode and include data bits in the output stream 836 . An entropy encoding processor 832 may support encoding signaling information regarding update modes of a block of a UR frame. In addition, the quantized coefficients may be passed to an inverse quantization/inverse transform processor 820 capable of reproducing pixels that may be combined with the prediction result and processed by an in-loop filter 824, and the output of this filter may be stored in a decoded image buffer 828 for use by a motion estimation/compensation processor 812 capable of supporting updating blocks of the UR frame at the decoder.

While only a small number of variants have been described in detail above, other modifications or additions are also possible. For example, in some implementations, the current blocks may be any symmetrical blocks (8x8, 16x16, 32x32, 64x64, 128x128 and other similar sizes), as well as any asymmetric blocks (8x4, 16x8 and other similar sizes). ).

In some embodiments, a decision tree can be implemented which is a combination of a quadtree plus a binary decision tree (QTBT). In the case of a QTBT tree, at the unit level of the coding tree, the partitioning parameters for that QTBT tree can be dynamically inferred to adapt to local characteristics without transferring any overhead. Thereafter, at the coding unit level, the unified classification decision tree structure can eliminate unnecessary iterations and management risks due to false prediction. In some implementations, a UR frame block update mode may be available at each of the leaf nodes of the QTBT tree. The current decoded block, as described above, may be in the QTBT tree; for example, and without limitation, the current decoded block may be a non-leaf node of the QTBT tree.

In some implementations, additional syntax elements may be signaled in the data bitstream at different levels of the hierarchy. A UR frame block update, which may include, without limitation, an explicitly signaled UR frame block update and/or an implicitly signaled UR frame block update, may be activated for the entire sequence by inserting an encoded wake-up flag in sequence parameter set (SPS). Further, the CTU flag may be encoded at the coding tree units (CTU) level to indicate whether any coding units (CUs) are using the UR frame update mode. The CU flag may be encoded to indicate whether the current coding unit uses the UR frame update mode. The CTU flag may be encoded at the coding tree units (CTU) level to indicate whether any coding units (CUs) are explicitly signaling regarding the UR frame update. The CTU flag may be encoded at the coding tree units (CTU) level to indicate whether any coding units (CUs) use implicit signaling regarding the UR frame update. While the above options have been described in relation to updates to UR frames, the above options may alternatively or in addition be applied to other frames, pictures, including, without limitation, long-term key frames.

The subject invention described here provides numerous technical advantages. For example, some embodiments of the subject matter of the present invention may provide a block decoding method using a UR frame that can update only portions of the UR frame without having to update the entire UR frame. Such approaches can reduce complexity while increasing compression efficiency. Moreover, by updating blocks of the UR frame in response to implicit signaling, header overhead can be reduced, thereby reducing the data bit rate and improving compression.

It should be noted that any one or more of the aspects and options described herein can be successfully implemented using digital electronic circuits, integrated circuits, application specific integrated circuits (ASICs), field programmable gate arrays ( FPGA)) computer hardware, firmware, downloadable software, and/or combinations of these elements implemented in one or more machines (for example, one or more computing devices used as a user computing device for an electronic document, one or more server devices such as a document server, etc.) programmed in accordance with the present description, as will be clear even to ordinary computer technicians. These various aspects or features may comprise implementation in one or more computer programs and/or software executing and/or interpreted on a programmable system comprising at least one special purpose or general purpose processor coupled to receive data and instructions from and to transmit data and commands to the information storage system, at least one input device and at least one output device. Qualified programmers will be able to code appropriate software based on the teachings of the present invention quite easily, as will be obvious even to ordinary programmers. The above-described aspects and implementations using software and/or software modules may also include suitable hardware to facilitate the implementation of machine-executable software and/or software module instructions.

Such software may be a computer program product using a computer-readable medium for storing information. A computer-readable storage medium may be any medium capable of storing and/or encoding a sequence of instructions for execution by a machine (eg, a computing device) and to control the machine to implement any of the methods and/or options described herein. Examples of such computer-readable storage media include, but are not limited to, a magnetic disk, an optical disk (for example, CD, CD-R, DVD, DVD-R, etc.), a magneto-optical disk, read only memory (ROM (read -only memory “ROM”), random access memory (RAM (random access memory “RAM”)), magnetic card, optical card, solid-state memory, erasable programmable read-only memory (EPROM), electrically erasable programmable read only memory (EEPROM), programmable logic devices (Programmable Logic Device (PLD)) and/or any combination of such devices. The term "computer-readable medium" as used herein encompasses both a single medium and a collection of physically separate media such as, for example, a set of CDs or one or more hard disk drives in combination with a computer storage device. As used herein, the term "computer-readable storage medium" does not cover temporal forms of signal transmission.

Such software may also contain information (eg, data) transmitted as a data signal on a storage medium, such as a carrier wave. For example, machine-readable information may be represented as a data signal on a data carrier, on which the signal encodes a sequence of instructions, or a portion thereof, to be executed by a machine (e.g., a computing device) and any related information (e.g., data structures and data) that causes machine to carry out any of the methods and/or options described here.

Examples of a computing device include, but are not limited to, an e-book reader, a computer workstation, a terminal computer, a server computer, a hand-held device (such as a tablet computer, a smartphone, etc.), an Internet device, a network router , a network bridge, any machine capable of executing a sequence of instructions specifying the actions that the machines should perform, and any combination of these devices. In one example, the computing device may comprise and/or be embedded in a kiosk.

On FIG. 9 is a schematic representation of one embodiment of a computing device in the form of an exemplary computer system 900, where a variety of instructions can be executed, in accordance with which the control system implements one or more aspects and/or methods of the present invention. It is also contemplated that a plurality of computing devices may be used to implement a specially configured set of instructions in accordance with which one or more devices perform one or more aspects and/or methods of the present invention. Computer system 900 includes a processor 904 and a memory 908 that communicate with each other and with other components over a bus 912. Bus 912 may include any of a number of bus structures including, but not limited to, a memory bus, a memory controller, a peripheral bus. devices, a local bus, and any combination of these components using any of the various bus architectures.

The storage device 908 may include various components (eg, computer-readable media), including, but not limited to, a random access storage component, a persistent storage component, and any combination of such components. In one example, the storage device 908 may contain a basic input/output system (BIOS) 916 containing basic routines to help transfer information between elements in the computer system 900, such as during startup of that computer system. systems. Memory 908 may also contain (eg, stored on one or more computer-readable media) instructions (eg, software) 920 that implement any one or more aspects and/or methods of the present invention. In another example, storage device 908 may further contain any number of program modules, including, but not limited to, an operating system, one or more application programs, other program modules, program data, and any combination thereof.

The computer system 900 may also include a program and data store 924. Examples of a program and data store (e.g., program and data store 924) include, but are not limited to, a hard disk drive, a magnetic disk drive, an optical disk drive, an an optical disc in combination with an optical media, a solid state storage device, and any combination of such devices. The program and data store 924 may be connected to the bus 912 via a suitable interface (not shown). For example, such interfaces include, but are not limited to, small computer system interface, SCSI, advanced technology attachment (ATA), ATA serial interface, universal serial bus (USB), interface according to the standard IEEE 1394 (FIREWIRE) or any combination of these interfaces. In one example, program and data store 924 (or one or more components of such store) may be interfaced with computer system 900 such that it can be detached (eg, via an external port connector (not shown)). In particular, the program and data store 924 and its associated computer-readable medium 928 may create a non-volatile and/or volatile store of computer-readable instructions, data structures, program modules, and/or other data for the computer system 900. In one example, the software 920 may be resident, in whole or in part, on the computer-readable medium 928. In another example, software 920 may be resident, in whole or in part, in the processor 904.

Computer system 900 may also include an input device 932. In one example, a user of computer system 900 may enter commands and/or other information into computer system 900 through this input device 932. Examples of input device 932 include, but are not limited to, an alphanumeric input device (e.g., a keyboard), a pointing device, a joystick, a gamepad (game pad), an audio input device (e.g., a microphone, a voice response system, etc.). ), cursor control device (such as a mouse), touch pad, optical scanner, image capture device (such as a camera, camcorder), touch screen, or some combination of these devices. The input device 932 may interface with the bus 912 through any of a variety of interfaces (not shown), including but not limited to serial, parallel, game port, USB, FIREWIRE, direct to bus 912, and any combination of such devices. Device 932 may include a touch screen interface, which may be part of or separate from display 936, as discussed further below. An input device 932 may be used as a user selector to select one or more graphical representations, as discussed above.

The user may also enter commands and/or other information into the computer system 900 via the program and data stores 924 (eg, removable disk drive, flash drive, etc.) and/or the network interface device 940. Such a network interface device 940 may be used to connect computer system 900 to one or more networks, such as network 944, and one or more remote devices 948 connected thereto. Examples of network interface devices include, but are not limited to, a network interface card (eg, a mobile network interface card, a local area network (LAN) card), a modem, and some combination of these devices. Examples of a network include, but are not limited to, a wide area communications network (eg, the Internet, an enterprise network), a local communications network (eg, a network associated with an office, building, campus, or some relatively small geographical area), telephone network, network a data connection associated with a telephone/voice provider (eg, a data and/or voice network of a mobile provider), a direct connection between two computing devices, or some combination of these entities. A communications network, such as network 944, may use wired and/or wireless communications. In general, any network topology can be used. Information (eg, data, software 920, etc.) may be transferred to or from computer system 900 via network interface device 940.

The computer system 900 may further include a video display adapter 952 for transmitting the displayed image to a display device, such as a display device 936. Examples of a display device include, but are not limited to, liquid crystal display (LCD), cathode ray tube (cathode ray tube (CRT)), plasma display, light emitting diode (LED) display, and any combination of such displays. Display adapter 952 and display device 936 may be used in conjunction with processor 904 to generate graphical representations of aspects of the present invention. In addition to the display device, computer system 900 may include one or more other peripheral output devices, including, but not limited to, an audio speaker, a printer, and any combination of these devices. Such output peripherals may be connected to the bus 912 via a peripheral interface 956 . Examples of such peripheral interfaces include, but are not limited to, a serial port, a USB connection, a FIREWIRE connection, a parallel connection, or any combination of such connections.

The above is a detailed description of illustrative embodiments of the present invention. Various modifications and additions can be made here without deviating from the spirit and scope of the present invention. The features of each of the various options described herein may be appropriately combined with the features of the other options described to obtain multiple combinations of features, in associated new variants. Further, although a number of separate options have been discussed above, the description given here is merely an illustration of the application of the principles of the present invention. In addition, although the various methods provided herein may be illustrated and/or described as being performed in a certain specific order, this order may be varied within wide limits, accessible even to an ordinary specialist, to obtain various options, as described here. Accordingly, the present description is merely an example and does not otherwise limit the scope of the present invention.

In the above description and in the claims, the phrases "at least one of" or "one or more of" may be followed by an appendix list of elements or features. The term "and/or" may also occur in a list of two or more elements or features. Unless explicitly or implicitly contradicted by the context, such term shall mean any of the elements or features listed in the list individually, or any of the mentioned elements or features in combination with any other of the listed elements or features. For example, the phrases "at least one of A and B"; "one or more of A and B"; and "A and/or B" shall each mean "A alone, B alone, or A and B together". A similar interpretation is also intended for a list containing three or more entries. For example, the phrases "at least one of A, B and C"; "one or more of A, B and C"; and "A, B and/or C" shall each mean "A alone, B alone, C alone, A and B together, A and C together, B and C together, or A and B and C together." In addition, the use of the term "based on" above and in the claims must mean "at least partially based on" such that an unmentioned feature or element is also allowed.

The subject matter described herein may be implemented in systems, apparatus, methods, and/or articles, depending on the desired configuration. The embodiments described in the description above do not represent all embodiments consistent with the subject matter described herein. On the contrary, these are just a few examples consistent with aspects related to the described subject matter. Although only a small number of variants have been described in detail above, other modifications or additions are also possible. In particular, other features and/or variants may be created in addition to what is discussed here. For example, the embodiments described above may be directed to various combinations and subcombinations of the described features and/or combinations and subcombinations of a number of other features described above. In addition, the logic circuits depicted in the accompanying drawings and/or described herein do not need to be performed in the order shown here or in sequential order to achieve the desired results. Other implementation options may be within the scope of the following claims.

Claims (9)

1. A decoding device containing a circuit for: receiving a data bit stream generated by an encoder, comprising signaling information and a first encoded picture to be used as a reference and a second encoded picture, the second encoded picture comprising a first block and a second block, the signaling information comprising a Sequence Parameter Set (SPS) indicating that a mode is available in which part of the reference picture can be modified; decoding and storing the first picture as a reference picture; modifying, using said mode, only a portion of the reference picture in response to information generated by the decoding apparatus and contained in the data bit stream; using the modified part of the reference image as a predictor for the first block of the second image; decoding the first block by adding residual pixels to the predictor; And decoding the second block of the second picture, excluding the use of modified parts of any reference pictures. 2. The decoding apparatus according to claim 1, wherein the first block and the second block are NxN coding units. 3. The decoding apparatus according to claim 1, wherein the second block of the second image is an intra-frame coding block.

Images