JP6503101B2 - Block inversion and skip mode in intra block copy prediction - Google Patents

Description

  Engineers use compression (also called source coding) to reduce the bit rate of digital video. Compression reduces the cost of storing and transmitting video information by converting the video information to a lower bit rate format. Decompression (also called decoding) reconstructs a version of the original information from the compressed format. A "codec" is an encoder / decoder system.

  For the last 20 years, ITU-T H. H.261 standard, H. H. 262 (MPEG-2 or ISO / IEC 13818-2) standard; H.263 standard and H.264. H.264 (MPEG-4 AVC or ISO / IEC 14496-10) standard, and MPEG-1 (ISO / IEC 11172-2) standard, MPEG-4 Visual (ISO / IEC 14496-2) standard, and SMPTE 421 M (VC) -1) Various video codec standards have been adopted including the standard. More recently, H. The H. 265 / HEVC (ITU-T H. 265 or ISO / IEC 23008-2) standard has been approved. H. Extensions to the H.265 / HEVC standard (e.g. for scalable video coding / decoding, for video encoding / decoding with higher fidelity in terms of sample color depth or chroma sampling rate, for screen capture content, Or multi-view coding / decoding) is currently under development. The video codec standard usually defines an option for the syntax of the coded video bitstream, which details the parameters in the coded video bitstream when certain features are used in coding and decoding . In many cases, the video codec standard also provides details on the decoding operation that the decoder should perform to achieve matching results in decoding. Apart from the codec standard, various proprietary codec formats define other options for the syntax of the coded video bitstream and the corresponding decoding operations.

  Intra block copy ("BC") is based on H.264. This is a prediction mode being considered for the H.265 / HEVC extension. For intra BC prediction mode, the sample values of the current block in a picture are predicted using previously reconstructed sample values in the same picture. The block vector ("BV") indicates the displacement from the current block to the area in the picture, including previously reconstructed sample values used for prediction. The BV is included and signaled in the bitstream. Intra-BC prediction is a form of intra-picture prediction (in-picture prediction), and intra-BC prediction for blocks in a picture does not use any sample values other than sample values in the same picture.

  The intra BC prediction mode is currently H.264. Specified in the H. 265 / HEVC standard; Although implemented in some reference software for the H.265 / HEVC standard, intra BC prediction mode has some problems. In particular, coding of blocks with predictable BC displacements is not processed efficiently, and intra BC prediction for content with inverted patterns is not processed efficiently.

  In summary, the detailed description presents innovation in the area of block encoding or decoding using intra block copy ("BC") prediction. For example, some of the innovations relate to block inversion in which intra BC prediction blocks are inverted relative to a reference region that may be indicated by a block vector ("BV") value. Another innovation relates to the signaling in skip mode, where the current intra BC prediction block uses signaled BV differences but does not have residual data. In many situations, the present innovation improves intra BC prediction block coding efficiency.

  According to a first aspect of the innovation described herein, an image encoder or video encoder is configured to generate an intra BC prediction region for a current block (eg, coding unit, prediction unit) in a picture, a reference region in the picture Make a decision based on The intra BC prediction region is inverted relative to the reference region. For example, the intra BC prediction region is flipped horizontally to the reference region, flipped vertically to the reference region, or flipped both horizontally and vertically to the reference region It is done.

  The encoder uses the intra BC prediction region to encode the current block and outputs the data encoded in the bitstream. The encoded data includes an indication of whether the intra BC prediction region is inverted relative to the reference region. For example, an indication is one or more syntax elements in the bitstream that may be signaled for the current block or a larger block that contains the current block. The one or more syntax elements may be one or more flags, each flag indicating a decision on the direction of inversion. One or more syntax elements may be jointly coded with another syntax element or may be separately signaled by being included in a bitstream.

  The corresponding decoder receives the encoded data in the bitstream. The encoded data includes an indication of whether the intra BC prediction region for the current block (eg, coding unit, prediction unit) in the picture is inverted relative to the reference region in the picture. For example, an indication is one or more syntax elements in the bitstream that may be signaled for the current block or a larger block that contains the current block. The one or more syntax elements may be one or more flags, each flag indicating a decision on the direction of inversion. One or more syntax elements may be jointly encoded with another syntax element or may be included in the bitstream and signaled separately.

  The decoder determines the intra BC prediction region for the current block based on the reference region in the picture. The intra BC prediction region is inverted (eg, horizontally and / or vertically) with respect to the reference region. The decoder uses the intra BC prediction region to decode the current block.

  When the encoder or decoder determines an intra BC prediction region that is inverted relative to the reference region, the encoder or decoder (a) determines the reference region, (b) inverts the reference region, and then (c) ) Sample values at inverted reference region locations can be assigned to sample values at those locations of the intra BC prediction region. Alternatively, the encoder or decoder (a) determines the reference region, (b) assigns sample values at the location of the reference region to sample values at those locations of the intra BC prediction region, and then (c) intra BC prediction The area can be inverted. Alternatively, the encoder or decoder may (a) determine the reference region and then (b) assign sample values at the position of the reference region to sample values at the corresponding position of the intra BC prediction region, where: The corresponding position takes into account the reversal.

  In some exemplary implementations, the encoded data includes the BV value of the current block. The BV value indicates the displacement to the reference area in the picture. During encoding, the BV value may be a predicted BV value, or the BV value may be identified in the BV estimate and signaled together with the BV difference to the predicted BV value. During decoding, the BV value may be a predicted BV value, or the BV value may be reconstructed by adding the BV difference to the predicted BV value.

  According to another aspect of the innovation described herein, the image encoder or video encoder determines the BV value of the current block (eg, coding unit, prediction unit) in the picture. The BV value indicates the displacement to the reference area in the picture. The encoder uses the BV value of the current block and the BV predictor (predicted BV value) to determine the BV difference of the current block. The bitstream may include index values that indicate the selection of BV predictor candidates from the set of BV predictor candidates for use as a BV predictor. Alternatively, BV predictors may be selected in some other manner. The encoder encodes the current block using intra BC prediction using BV values. The encoder outputs encoded data including in the bitstream a flag indicating that the current block is encoded in skip mode using intra BC prediction. Since the current block is coded in skip mode using intra BC prediction, the bitstream contains the BV difference of the current block but does not include residual data of the current block.

  In some example implementations, if a given block (eg, the current block, a subsequent block) is not encoded in skip mode using intra BC prediction, another flag may be given for the given block It may indicate whether it is encoded in non-skip mode using intra BC prediction. If a given block is not coded in non-skip mode using intra BC prediction, then the given block may be intra spatial prediction mode or inter picture as indicated by one or more other syntax elements It may be encoded in another mode such as a mode.

  In some example implementations, a given block being intra BC-predicted in skip mode (e.g., current block, subsequent block) has a defined value for partitioning mode . This affects the signaling of syntax elements for split mode. If a given block is encoded in non-skip mode using intra BC prediction, the bitstream contains syntax elements that indicate the division mode of the given block. However, if the given block is encoded in skip mode using intra BC prediction, then the bitstream does not contain syntax elements indicating the splitting mode of the given block, and the splitting of the given block The mode has a defined value.

  In some exemplary implementations, a given block being intra BC-predicted in skip mode (eg, current block, subsequent block) does not have a flag indicating the presence or absence of residual data for the given block . Residual data for a given block is considered not to be in the bitstream. Also, if a given block is encoded in non-skip mode using intra BC prediction, and the split mode of the given block has a defined value, then the bitstream is of the given block. Does not include a flag indicating the presence or absence of residual data. In this case, the residual data of a given block is considered to be in the bitstream. Also, if the given block is encoded in non-skip mode using intra BC prediction, and the split mode of the given block does not have a defined value, then the bitstream is given It includes a flag indicating the presence or absence of residual data of the block.

  The corresponding decoder is encoded with a flag that indicates that the current block (eg, the encoding unit, the prediction unit) in the picture from the bitstream is encoded in skip mode using intra BC prediction Receive the received data. Since the current block is coded in skip mode using intra BC prediction, the bitstream contains the BV difference of the current block but does not include residual data of the current block. The decoder uses the BV difference of the current block and the BV predictor (predicted BV value) to determine the BV value of the current block. The bitstream may include index values that indicate the selection of BV predictor candidates from the set of BV predictor candidates for use as a BV predictor. Alternatively, BV predictors may be selected in some other manner. The BV value indicates the displacement to the reference area in the picture. The decoder decodes the current block using intra BC prediction using BV values.

  If the current block is coded in skip mode using intra BC prediction, the intra BC prediction region for the current block may be the one that is inverted relative to its reference region. Examples of inversion operation, direction of inversion, and signaling of whether inversion is used are summarized above.

  The innovation for intra-BC prediction may have stored computer-executable instructions for causing the computing device to perform the method, as part of a method, as part of a computing device adapted to perform the method, or It may be implemented as part of a computer readable medium of the body. Various innovations may be used in combination or separately. In particular, block inversion in intra BC prediction may be used in combination with the skip mode for intra BC prediction blocks.

  The foregoing and other objects, features and advantages of the present invention will become more apparent from the following detailed description, which proceeds with reference to the accompanying drawings.

FIG. 13 is a diagram of an example computing system that can implement some described embodiments. FIG. 1 is a diagram of an example network environment in which some described embodiments may be implemented. FIG. 1 is a diagram of an example network environment in which some described embodiments may be implemented. FIG. 13 is a diagram of an example encoder system that can be cooperatively implemented with some described embodiments. FIG. 13 is a diagram of an example decoder system that can be cooperatively implemented with some described embodiments. FIG. 10 illustrates an example video encoder that can be implemented cooperatively with some described embodiments. FIG. 10 illustrates an example video encoder that can be implemented cooperatively with some described embodiments. FIG. 10 illustrates an example video decoder that can be cooperatively implemented with some described embodiments. FIG. 10 shows intra BC prediction for blocks in a picture. FIG. 16 shows BV prediction for B and B blocks. FIG. 7 shows BV prediction for blocks in a picture. FIG. 7 shows the inversion of the reference area for the block. FIG. 7 shows the inversion of the reference area for the block. FIG. 7 shows the inversion of the reference area for the block. FIG. 7 shows the inversion of the reference area for the block. FIG. 7 shows the inversion of the reference area for the block. FIG. 7 shows the inversion of the reference area for the block. FIG. 7 shows the inversion of the reference area for the block. FIG. 7 shows the inversion of the reference area for the block. FIG. 7 shows the inversion of the reference area for the block. FIG. 7 shows the inversion of the reference area for the block. FIG. 7 is a flow chart illustrating a technique for encoding in which an intra BC prediction region is inverted with respect to a reference region. FIG. 16 is a flow chart illustrating a technique for decoding in which the intra BC prediction region is inverted relative to the reference region. 10 is a flow chart illustrating a technique for encoding including skip mode for intra BC prediction blocks. 10 is a flow chart illustrating a technique for encoding including skip mode for intra BC prediction blocks. 10 is a flow chart illustrating a technique for decoding that includes skip mode for intra BC prediction blocks. 10 is a flow chart illustrating a technique for decoding that includes skip mode for intra BC prediction blocks. Table showing syntax structure for a coding unit, according to the conventional approach. Table showing syntax structure for a coding unit, according to the conventional approach. FIG. 7 is a table showing a new syntax structure for a coding unit that may be coded as an intra BC prediction block in skip mode.

  The detailed description presents innovation in the area of block encoding or decoding using intra block copy ("BC") prediction. For example, some of the innovations relate to block inversion in which intra BC prediction blocks are inverted relative to a reference region that may be indicated by a block vector ("BV") value. Another innovation relates to the signaling in skip mode, where the current intra BC prediction block uses signaled BV differences but does not have residual data. In many situations, the present innovation improves intra BC prediction block coding efficiency.

  Although the operations described herein are described elsewhere as being performed by a video encoder or video decoder, in many cases such operations may be performed by another type of media processing tool (e.g. Image decoder).

  Some of the innovations described herein are: It is shown with reference to syntax elements and operations specific to the H.265 / HEVC standard. For example, H. Reference is made to the draft version JCTVC-P1005 ("High Efficiency Video Coding (HEVC) Range Extensions Text Specification: Draft 6", JCTVC-P1005_v1, February 2014) of the H.265 / HEVC standard. The innovations described herein can also be implemented for other standards or formats.

  Some of the innovations described herein (eg, block inversion) are described with reference to intra BC prediction. The innovation can also be applied in other contexts (eg, block inversion for reference regions in motion compensation).

  More generally, various alternatives to the examples described herein are possible. For example, some of the methods described herein may be performed by, for example, changing predetermined method operations by changing the order of method operations described, repeating predetermined method operations, or Changes can be made by omitting the method operations. Various aspects of the disclosed technology can be used in combination or separately. Various embodiments use one or more of the described innovations. Some of the innovations described herein address one or more of the issues noted in the background art. In general, a given technology / tool does not solve all such problems.

I. Exemplary Computing System FIG. 1 shows a generalized example of a suitable computing system (100) that can implement some of the described innovations. The computing system (100) is not intended to suggest any limitation as to the scope of use or functionality. This is because the innovation can be implemented on a variety of general purpose computing systems or dedicated computing systems.

  Referring to FIG. 1, a computing system (100) includes one or more processing units (110, 115) and memories (120, 125). The processing units (110, 115) execute computer-executable instructions. The processing unit may be a general purpose central processing unit ("CPU"), a processor in an application specific integrated circuit ("ASIC"), or any other type of processor. In a multi-processing system, multiple processing units execute computer-executable instructions to increase processing power. For example, FIG. 1 also shows, in addition to the central processing unit (110), a graphics processing unit or co-processing unit (115). The tangible memory (120, 125) may be volatile memory (eg, register, cache, RAM) accessible by one or more processing devices, or non-volatile memory (eg, ROM, EEPROM, etc.). It may be a flash memory or the like, or any combination of the two. The memory (120, 125) is software (one or more innovations for block inversion and / or skip mode in intra BC prediction) in the form of computer-executable instructions suitable for execution by one or more processing devices 180) is stored.

  The computing system can have additional features. For example, the computing system (100) includes storage (140), one or more input devices (150), one or more output devices (160), and one or more communication connections (170). An interconnection mechanism (not shown), such as a bus, controller, or network interconnects the components of the computing system (100). Operating system software (not shown) typically provides an operating environment for other software executing on the computing system (100) to coordinate the operation of the components of the computing system (100).

  The tangible storage (140), which may or may not be removable, may be used to store magnetic disk, magnetic tape, magnetic cassette, CD-ROM, DVD or information And any other medium that can be accessed within the computing system (100). The storage (140) stores instructions of the software (180) implementing one or more innovations for block inversion and / or skip mode in intra BC prediction.

  The one or more input devices (150) may be a keyboard, a touch input device such as a mouse, a pen, or a trackball, an audio input device, a scanning device or another device providing input to the computing system (100) be able to. With respect to video, one or more input devices (150) may be a camera, video card, TV tuner card, screen capture module, or similar device that accepts video input in analog or digital form, or a computing system (100 ) Can be used as a CD-ROM or a CD-RW. The one or more output devices (160) may be a display, a printer, speakers, a CD writer, or another device that provides output from the computing system (100).

  One or more communication connections (170) enable communication with another computing entity via a communication medium. The communication medium conveys information such as computer-executable instructions, audio input, video input, audio output, video output, or other data via the modulated data signal. A modulated data signal is a signal that has one or more of the characteristics set or changed to encode information in the signal. By way of example and not limitation, communication media may use electrical, optical, RF or other carriers.

  The innovation can be described in the general context of computer readable media. Computer readable media are any available tangible media that can be accessed within a computing environment. By way of example, and not limitation, in computing system (100), computer readable media may include memory (120, 125), storage (140), and any combination thereof.

  The innovation can be described in the general context of computer-executable instructions. Computer-executable instructions are included, for example, in program modules, and executed on a target real or virtual processor in a computing system. Generally, program modules include routines, programs, libraries, objects, classes, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The functionality of the program modules may be combined or split between program modules as desired in various embodiments. Computer executable instructions of program modules may be executed within a local computing system or a distributed computing system.

  The terms "system" and "device" are used interchangeably herein. These terms do not denote any limitation as to the type of computing system or computing device, unless the context clearly indicates otherwise. In general, computing systems or devices may be localized or distributed and may be dedicated and / or general purpose hardware and software that implements the functionality described herein It may include any combination.

  The disclosed methods can also be implemented using dedicated computing hardware configured to perform any of the disclosed methods. For example, the disclosed method may be an integrated circuit (eg, an ASIC (eg, an ASIC digital signal processor ("DSP") or the like), a graphics processing Or a programmable logic device ("PLD") or the like, such as a field programmable gate array ("FPGA").

  For the purpose of presentation, the detailed description uses terms such as "determine" and "use" to describe computer operations in a computing system. These terms are high-level abstract representations of computer-implemented operations and should not be confused with human-implemented operations. The actual computer operations that correspond to these terms will vary depending on the implementation.

II. Exemplary Network Environment FIGS. 2a and 2b illustrate an exemplary network environment (201, 202) that includes a video encoder (220) and a video decoder (270). The encoder (220) and the decoder (270) are connected via the network (250) using an appropriate communication protocol. The network (250) may include the Internet or another computer network.

  In the network environment (201) shown in FIG. 2a, each real-time communication ("RTC") tool (210) includes both an encoder (220) and a decoder (270) for bi-directional communication. A given encoder (220) is an H.323 encoder. A variant or extension of the H.265 / HEVC standard, an output conforming to the SMPTE 421M standard, the ISO / IEC 14496-10 standard (also known as H.264 or AVC), another standard or a proprietary format The corresponding decoder (270) can accept the encoded data from the encoder (220). Two-way communication may be part of a video conferencing, video calling, or other two party or multi party communication scenario. Although the network environment (201) of FIG. 2a includes two real time communication tools (210), the network environment (201) may include more than two real time communication tools (210) participating in multi-party communication.

  The real time communication tool (210) manages the encoding by the encoder (220). FIG. 3 shows an exemplary encoder system (300) that may be included in the real time communication tool (210). Alternatively, the real time communication tool (210) may use another encoder system. The real time communication tool (210) also manages decoding by the decoder (270). FIG. 4 shows an exemplary decoder system (400) that may be included in the real time communication tool (210). Alternatively, the real time communication tool (210) may use another decoder system.

  In the network environment (202) shown in FIG. 2b, the encoding tool (212) includes an encoder (220) that encodes video for delivery to a plurality of playback tools (214) including a decoder (270). Unidirectional communication is for video surveillance systems, webcam monitoring systems, screen capture modules, remote desktop conference presentations, or other scenarios where video is encoded and transmitted from one location to one or more other locations Can be provided. The network environment (202) of FIG. 2b includes two playback tools (214), but the network environment (202) may include more or less playback tools (214). In general, the playback tool (214) communicates with the encoding tool (212) to determine the stream of video that the playback tool (214) receives. The playback tool (214) receives the stream, buffers the received encoded data for an appropriate period of time, and starts decoding and playback.

  FIG. 3 shows an exemplary encoder system (300) that may be included in the encoding tool (212). Alternatively, the encoding tool (212) may use another encoder system. The coding tool (212) may also include server-side controller logic to manage the connection with one or more playback tools (214). FIG. 4 shows an exemplary decoder system (400) that may be included in the playback tool (214). Alternatively, the playback tool (214) may use another decoder system. The playback tool (214) may also include client-side controller logic to manage the connection with the coding tool (212).

III. Exemplary Encoder System FIG. 3 is a block diagram of an exemplary encoder system (300) that may be implemented cooperatively with some described embodiments. The encoder system (300) may generate multiple codes such as low delay coding mode for real time communication, transcoding mode, and higher delay coding mode to generate media for playback from a file or stream. It may be a general purpose coding tool that can operate in any of the two coding modes, or it may be a dedicated coding tool adapted for one such coding mode. The encoder system (300) may be adapted to encode particular types of content (eg, screen capture content). The encoder system (300) can be implemented as an operating system module, as part of an application library, or as a stand-alone application. In general, the encoder system (300) receives a series of source video frames (311) from a video source (310) and generates encoded data as output to a channel (390). The coded data output to the channel may include content coded using block inversion and / or skip modes in intra BC prediction, as described herein.

  The video source (310) may be a camera, tuner card, storage medium, screen capture module, or other digital video source. The video source (310) generates a series of video frames at a frame rate, eg, 30 frames per second. As used herein, the term "frame" generally refers to source encoded or reconstructed image data. For progressive scan video, the frame is a progressive scan video frame. For interlaced video, in an exemplary embodiment, interlaced video frames may be de-interlaced prior to encoding. Alternatively, two complementary interlaced video fields may be encoded together as one video frame or may be encoded as two separately encoded fields. The terms "frame" or "picture" refer to a single non-paired video field, a complementary pair of video fields, video at a given time, apart from indicating progressive scan video frame or interlaced scan video frame It may indicate a video object plane representing an object, or a region of interest in a larger image. The video object plane or region may be part of a larger image that includes multiple objects or regions of the scene.

  The incoming source frame (311) is stored in a source frame temporary memory storage area (320) that includes a plurality of frame buffer storage areas (321, 322, ..., 32n). The frame buffer (321, 322, etc.) holds one source frame in the source frame storage area (320). After one or more of the source frames (311) are stored in the frame buffer (321, 322, etc.), the frame selector (330) selects individual source frames from the source frame storage area (320). The order in which frames are selected by the frame selector (330) for input to the encoder (340) may be different than the order in which frames are generated by the video source (310). For example, some subsequent frames may be encoded first, which may delay the encoding of some frames in order to allow temporal backward prediction to be facilitated. Prior to the encoder (340), the encoder system (300) may include a pre-processor (not shown) that performs preprocessing (eg, filtering) of the selected frame (331) prior to encoding. Pre-processing is color space conversion to primary (eg luma) and secondary (eg chroma difference to red and blue) components for encoding and (eg to reduce spatial resolution of chroma component) It may include a resampling process. Usually, before encoding, the video is converted to a color space such as YUV. In YUV, the sample values of the luma (Y) component represent luminance values or lightness values, and the sample values of the chroma (U, V) components represent chrominance values. The exact definition of the color difference values (and the conversion operation from YUV color space to another color space such as RGB, conversion operation from another color space such as RGB to YUV color space) is implementation dependent. In general, as used herein, the term YUV refers to any color space having one luma (or luminance) component and one or more chroma (or chrominance) components, Y'UV, YIQ, In addition to Y'IQ and YDbDr, variants such as YCbCr and YCoCg are included. Chroma sample values may be subsampled to a lower chroma sampling rate (eg, to match YUV 4: 2: 0 format), or chroma sample values may be matched (eg, to match YUV 4: 4: 4 format) 2.) may have the same resolution as luma sample values. Alternatively, the video may be encoded in another format (eg, RGB 4: 4: 4 format).

  The encoder (340) encodes the selected frame (331) to generate an encoded frame (341), and a memory management control operation ("MMCO") signal (342) or a reference picture set ("RPS") ) Generate information. RPS is a set of frames that can be used for reference in motion compensation for the current frame or any subsequent frame. If the current frame is not the first frame encoded, the encoder (340) performs one or more previously encoded as stored in the decoded frame temporary memory storage area (360) when performing the encoding process. / A decoded frame (369) can be used. Such stored decoded frame (369) is used as a reference frame for inter-frame prediction (inter-frame prediction) of the content of the current source frame (331). The MMCO / RPS information (342) indicates to the decoder which reconstructed frame may be used as a reference frame and thus should be stored in the frame storage area.

  In general, the encoder (340) includes multiple coding modules that perform coding tasks such as division into tiles, intra prediction estimation and prediction, motion estimation and compensation, frequency transform, quantization, and entropy coding . The exact operation performed by the encoder (340) may vary depending on the compression format. The format of the encoded data to be output is H.1. H.265 / HEVC format variants or extensions, Windows® Media Video format, VC-1 format, MPEG-x format (eg, MPEG-1, MPEG-2, or MPEG-4), H.264, H.264, H.264, H.264, H.264, H.264, H.264, H.264, H.264, H.264, H.264, H.264, H.264, H.264, H.264, H.264, H.264, H.264, H.264, H.264, H. 26x format (e.g., H.261, H.262, H.263, H.264) or another format.

  The encoder (340) may divide the frame into multiple tiles of the same size or different sizes. For example, the encoder (340) divides the frame along tile rows and tile columns that use the frame boundaries to define the horizontal and vertical boundaries of tiles within the frame. Here, each tile is a rectangular area. Tiles are often used to provide options for parallel processing. A frame may also be organized as one or more slices. Here, a slice may be an entire frame or an area of a frame. Slices can be decoded independently of other slices in the frame, which improves error resilience. The content of the slice or tile is further divided into other sets of blocks or samples for encoding and decoding.

  H. For syntax according to the H.265 / HEVC standard, the encoder divides the content of a frame (or slice or tile) into coding tree units. A coding tree unit ("CTU") includes luma sample values organized as one luma coded tree block ("CTB") and corresponding chroma sample values organized as two chroma CTBs. The size of the CTU (and its CTB) is selected by the encoder and may be, for example, 64 × 64 sample values, 32 × 32 sample values, or 16 × 16 sample values. A CTU includes one or more coding units. The coding unit ("CU") has one luma coding block ("CB") and two corresponding chroma CBs. For example, a CTU with one 64 × 64 luma CTB and two 64 × 64 chroma CTBs (YUV 4: 4: 4 format) may be divided into four CUs. Here, each CU includes one 32 × 32 luma CB and two 32 × 32 chroma CBs, and each CU is further divided into smaller CUs, if possible. Alternatively, as another example, a CTU (YUV 4: 2: 0 format) having one 64 × 64 luma CTB and two 32 × 32 chroma CTBs may be divided into four CUs. Here, each CU includes one 32 × 32 luma CB and two 16 × 16 chroma CBs, and each CU is further divided into smaller CUs, if possible. The CU's minimum allowable size (eg, 8x8, 16x16) may be included in the bitstream and signaled.

  In general, a CU has a prediction mode such as inter or intra. The CU includes one or more prediction units for signaling and / or processing of prediction information (eg, prediction mode details, displacement values, etc.). The prediction unit ("PU") has one luma prediction block ("PB") and two chroma PBs. For intra-prediction CUs, the PU has the same size as the CU, except where the CU has the smallest size (eg, 8 × 8). In this case, the CU may be divided into four smaller PUs (e.g. 4x4 each if the minimum CU size is 8x8), or PUs are indicated by a syntax element for CU Can have a minimum CU size. The CU also has one or more transform units for residual coding / decoding. Here, the transform unit ("TU") has one luma transform block ("TB") and two chroma TBs. A PU in an intra prediction CU may include one TU (equal in size to a PU) or a plurality of TUs. The encoder determines how to divide the video into CTUs, CUs, PUs, TUs, etc.

  H. In the 265 / HEVC implementation, a slice may include one slice segment (independent slice segment) or may be divided into multiple slice segments (one independent slice segment and one or more dependent slice segments). A slice segment is an integral number of CTUs sequentially ordered in a tile scan, included in one network abstraction layer ("NAL") unit. For independent slice segments, the slice segment header contains the values of syntax elements applied for the independent slice segment. For a dependent slice segment, the truncated slice segment header contains some values of syntax elements that apply for that dependent slice segment, and the values of other syntax elements for that dependent slice segment are in decoding order Inferred from the values for the preceding independent slice segment.

  As used herein, the term "block" may, depending on context, be a macroblock, a prediction unit, a residual data unit, ie CB, PB or TB, or some other set of sample values Can be shown.

  Returning to FIG. 3, the encoder represents the intra-coded block of that source frame (331) in terms of prediction from other previously reconstructed sample values in the source frame (331). For intra BC prediction, the intra picture estimator estimates the displacement of the block relative to other previously reconstructed sample values (or, in some implementations, relative to the original sample values in frame (331)) . An intra frame prediction reference region is a region of samples in a frame that is used to generate a BC prediction value for a block. The reference region may be indicated by the BV value (as determined in block vector ("BV") estimation). The reference region may be inverted relative to the prediction region for the block, as described herein. For intra-spatial prediction for a block, an intra-picture estimator estimates the extrapolation of adjacent reconstructed sample values to the block. The intra picture estimation unit can output prediction information (BV value for intra BC prediction, prediction mode (direction) for intra spatial prediction, etc.), and this prediction information is entropy coded. The intra-frame prediction predictor applies prediction information to determine an intra prediction value.

  The encoder (340) represents inter-frame coded prediction blocks of the source frame (331) in terms of prediction from the reference frame. The motion estimator estimates block motion relative to one or more reference frames (369). When multiple reference frames are used, the multiple reference frames may be from different temporal directions or from the same temporal direction. Motion compensated prediction reference regions are regions of sample values in one or more reference frames that are used to generate motion compensated prediction values for sample values of blocks in the current frame. The reference region may be inverted relative to the prediction region for the block, as described herein. The motion estimator outputs motion information, such as motion vector ("MV") information, which is entropy encoded. The motion compensation unit applies MV to the reference frame (369) to determine a motion compensated prediction value for inter-frame prediction.

  The encoder can determine the difference (if any) between the predicted value (intra or inter) of the block and the corresponding original value. Such prediction residual values are further encoded using frequency transformation, quantization and entropy coding. For example, the encoder (340) sets the values of quantization parameters ("QP") for pictures, tiles, slices, and / or other parts of the video, and quantizes the transform coefficients accordingly. The entropy coding unit of the encoder (340) adds predetermined side information (eg, MV information, BV predictor index value, BV difference, QP value, mode determination, parameter selection, in addition to the quantized transform coefficient value) ) Also compress. Common entropy coding techniques include exponential Golomb coding, Golomb rice coding, arithmetic coding, differential coding, Huffman coding, run length coding, "V2V (variable-length-to-variable-length) coding" , “V2F (variable-length-to-fixed-length)” coding, “LZ (Lempel-Ziv) coding, dictionary coding,“ PIPE (probability interval partitioning entropy) ”coding, and combinations thereof including. The entropy coding unit may use different coding techniques for different types of information, and may combine and apply multiple techniques (e.g. after applying Golombi coding, arithmetic coding Can be selected from among a plurality of code tables in a particular coding technique.

  An adaptive deblocking filter is included in the motion compensation loop at the encoder (340) to smooth discontinuities across block boundary rows and / or block boundary columns in the decoded frame. Alternatively or additionally, other filtering (deringing filtering, adaptive loop filtering ("ALF"), or sample adaptive offset ("SAO") filtering, etc .; not shown) is applied as an in-loop filtering operation. May be

  The encoded data generated by the encoder (340) contains syntax elements for the various layers of bitstream syntax. H. For syntax according to the H.265 / HEVC standard, for example, a picture parameter set ("PPS") is a syntax structure that includes syntax elements that may be associated with a picture. PPS may be used for one picture, and PPS may be reused for multiple pictures in a sequence. The PPS is normally signaled separately from the coded data for the picture (eg, one NAL unit is for PPS and one or more other NAL units are for coded data for the picture) . In coded data for a picture, a syntax element indicates which PPS to use for the picture. Similarly, H. With respect to syntax according to the H.265 / HEVC standard, a sequence parameter set ("SPS") is a syntax structure that includes syntax elements that may be associated with a sequence of pictures. The bitstream may include one SPS or multiple SPSs. The SPS is usually signaled separately from other data for the sequence, and syntax elements in the other data indicate which SPS to use.

  The encoded frame (341) and the MMCO / RPS information (342) (or the information equivalent to the MMCO / RPS information (342), as the dependencies and order structure for the frames are already known in the encoder (340) Are processed by the decryption process emulator (350). The decoding process emulator (350) implements, for example, part of the functionality of the decoder such as the decoding task to reconstruct reference frames. The Decoding Process Emulator (350) is to be used as a reference frame in inter-frame prediction of a subsequent frame to be encoded, such that a given coding frame (341) is consistent with the MMCO / RPS information (342) Determine if it needs to be reconstructed and stored. If a coded frame (341) needs to be stored, the decoding process emulator (350) will be performed by a decoder that receives the coded frame (341) and generates a corresponding decoded frame (351) Simulate the decoding process. In doing so, when the encoder (340) uses one or more decoded frames (369) stored in the decoded frame storage area (360), the decoding process emulator (350) may be part of the decoding process , Storage area (360) and one or more decoded frames (369) are used.

  The decoded frame temporary memory storage area (360) includes a plurality of frame buffer storage areas (361, 362, ..., 36n). The Decoding Process Emulator (350) matches any MMCO / RPS information (342), any frame buffer (361, 362) with a frame that is no longer needed by the encoder (340) to use as a reference frame. Manage the contents of the storage area (360) to identify etc.). After simulating the decoding process, the decoding process emulator (350) stores the newly decoded frame (351) in the frame buffer (361, 362, etc.) thus identified.

  The coded frame (341) and the MMCO / RPS information (342) are buffered in a temporary coded data area (370). The encoded data collected in the encoded data area (370) includes encoded data of one or more pictures as part of the syntax of the elementary encoded video bitstream. The encoded data collected in the encoded data area (370) may also be (eg, as one or more parameters in one or more Supplemental Enhancement Information ("SEI") messages or video usability information ("VUI") messages). It may include media metadata associated with the encoded video data.

  The collected data (371) from the temporary coded data area (370) is processed by a channel encoder (380). The channel encoder (380) (eg, according to media program stream or transport stream format such as ITU-T H. 222.0 | ISO / IEC 13818-1 or Internet real-time transport protocol format such as IETF RFC 3550) The collected data may be packetized and / or multiplexed for transmission or storage as a media stream. In such cases, the channel encoder (380) may add syntax elements as part of the media transport stream syntax. Alternatively, the channel encoder (380) can organize the collected data for storage as a file (eg, according to a media container format such as ISO / IEC 14496-12). In such cases, the channel encoder (380) may add syntax elements as part of the media storage file syntax. Alternatively, more generally, the channel encoder (380) may implement one or more media system multiplexing or transport protocols. In such cases, the channel encoder (380) may add syntax elements as part of the syntax of one or more protocols. The channel encoder (380) provides an output to the channel (390). Channel (390) represents another channel for storage, communication connection, or output. Channel encoder (380) or channel (390) may also include, for example, forward error correction ("FEC") coding and other elements (not shown) for analog signal modulation.

IV. Exemplary Decoder System FIG. 4 is a block diagram of an exemplary decoder system (400) that may be implemented cooperatively with some described embodiments. The decoder system (400) can operate in any of a plurality of decoding modes, such as a low delay decoding mode for real time communication and a higher delay decoding mode for media playback from a file or stream It may be a general purpose decoding tool or a dedicated decoding tool adapted for such one decoding mode. The decoder system (400) may be adapted to decode particular types of content (eg, screen capture content). The decoder system (400) can be implemented as an operating system module, as part of an application library, or as a stand-alone application. In general, the decoder system (400) receives the encoded data from the channel (410) and generates a reconstructed frame as output to an output destination (490). The encoded data may include content encoded using block inversion and / or skip modes in intra BC prediction, as described herein.

  The decoder system (400) comprises a channel (410) which may represent storage, a communication connection, or another channel for coded data as input. The channel (410) generates channel coded coded data. A channel decoder (420) can process the coded data. For example, the channel decoder (420) may (for example, a media program stream or transport stream format such as ITU-T H. 222.0 | ISO / IEC 13818-1, or an Internet real-time transport protocol such as IETF RFC 3550. Depacketize and / or demultiplex data collected for transmission or storage as a media stream according to the format). In such cases, the channel decoder (420) can parse syntax elements added as part of the media transport stream syntax. Alternatively, the channel decoder (420) separates the encoded video data collected for storage as a file (eg, according to a media container format such as ISO / IEC 14496-12). In such cases, the channel decoder (420) can parse syntax elements added as part of the media storage file syntax. Alternatively, and more generally, the channel decoder (420) can implement one or more media system demultiplexing protocols or transport protocols. In such cases, the channel decoder (420) can parse syntax elements added as part of the syntax of one or more protocols. The channel (410) or channel decoder (420) may also include, for example, FEC decoding and other elements (not shown) for analog signal demodulation.

  The coded data (421) output from the channel decoder (420) is stored in the temporary coded data area (430) until a sufficient amount of data is received. The coded data (421) includes a coded frame (431) and MMCO / RPS information (432). The encoded data (421) in the encoded data area (430) includes encoded data of one or more pictures as part of the syntax of the elementary encoded video bitstream. The encoded data (421) in the encoded data area (430) may also include media metadata associated with the encoded video data (eg, as one or more parameters in one or more SEI messages or VUI messages) .

  Generally, the coded data area (430) temporarily stores such coded data (421) until the coded data (421) is used by the decoder (450). At that time, the encoded data of the encoded frame (431) and the MMCO / RPS information (432) are transmitted from the encoded data area (430) to the decoder (450). As decoding proceeds, new encoded data is added to the encoded data area (430), and the oldest encoded data remaining in the encoded data area (430) is transmitted to the decoder (450).

  The decoder (450) decodes the encoded frame (431) to generate a corresponding decoded frame (451). Optionally, when performing the decoding process, the decoder (450) can use one or more previously decoded frames (469) as a reference frame for inter-frame prediction. The decoder (450) reads such previously decoded frame (469) from the decoded frame temporary memory storage area (460). In general, the decoder (450) includes multiple decoding modules that perform decoding tasks such as entropy decoding, intra-frame prediction, motion compensated inter-frame prediction, inverse quantization, inverse frequency transform, and tile merging. The exact operation performed by the decoder (450) may vary depending on the compression format.

  For example, the decoder (450) receives encoded data of a compressed frame or series of frames and generates an output including a decoded frame (451). In the decoder (450), the buffer receives the encoded data of the compressed frame and makes the received encoded data available to the entropy decoding unit at the appropriate time. The entropy decoding unit entropy-decodes the entropy-coded side information in addition to the entropy-coded quantized data, usually by applying the inverse of the entropy coding performed in the encoder. The motion compensation unit applies motion information to one or more reference frames to form a motion compensated prediction value of the inter-coding block of the frame being reconstructed. The inter frame reference region may be inverted relative to the prediction region for the block, as described herein. The intra frame prediction module may spatially predict the sample values of the current block from the adjacent previously reconstructed sample values, or, for intra BC prediction, prior to the intra frame prediction region in the frame The reconstructed sample values can be used to predict the sample values of the current block. Intra-frame reference regions may be indicated by BV values. This reference area may be inverted relative to the prediction area for the block, as described herein. The decoder (450) also reconstructs prediction residual values. The inverse quantization unit inversely quantizes entropy-decoded data. For example, the decoder (450) sets QP values for pictures, tiles, slices, and / or other parts of the video based on syntax elements in the bitstream and dequantizes the transform coefficients accordingly Do. The inverse frequency transform unit transforms the quantized frequency domain data into space domain data. For inter-frame prediction blocks, the decoder (450) combines the reconstructed prediction residual value with the motion compensated prediction value. The decoder (450) may similarly combine prediction residual values with predictions from intra prediction. An adaptive deblocking filter is included in the motion compensation loop at the video decoder (450) to smooth discontinuities across block boundary rows and / or block boundary columns in the decoded frame (451). Alternatively or additionally, other filtering (such as deringing filtering, ALF or SAO filtering; not shown) may be applied as an in-loop filtering operation.

  The decoded frame temporary memory storage area (460) includes a plurality of frame buffer storage areas (461, 462, ..., 46n). The decoded frame storage area (460) is an example of a decoded picture buffer. The decoder (450) uses MMCO / RPS information (432) to identify frame buffers (461, 462, etc.) that can store the decoded frame (451). The decoder (450) stores the decoded frame (451) in its frame buffer.

  The output sequencer (480) identifies when the next frame generated in output order will be available in the decoded frame storage (460). When the next frame (481) generated in output order becomes available in the decoded frame storage area (460), that frame is read by the output sequencer (480), and the output destination (490) ( For example, it is output to a display). In general, the order in which frames are output from the decoded frame storage area (460) by the output sequencer (480) may be different from the order in which frames are decoded by the decoder (450).

V. Exemplary Video Encoders FIGS. 5a and 5b are block diagrams of generalized video encoders (500) that may be implemented cooperatively with some described embodiments. The encoder (500) receives a series of video pictures, including the current picture, as an input video signal (505) and produces as output an encoded data in an encoded video bitstream (595).

  The encoder (500) is block based and uses an implementation dependent block format. The blocks may be further subdivided at various stages, such as, for example, in the prediction stage, the frequency transformation stage, and / or the entropy coding stage. For example, pictures may be divided into 64 × 64 blocks, 32 × 32 blocks, or 16 × 16 blocks, which in turn are divided into smaller blocks of sample values for encoding and decoding. obtain. H. In the coding implementation for the H.265 / HEVC standard, the encoder divides the picture into CTU (CTB), CU (CB), PU (PB), and TU (TB).

  The encoder (500) compresses the picture using intra picture coding and / or inter picture coding. Many of the components of the encoder (500) are used for both intra-picture coding and inter-picture coding. The exact operations performed by these components may vary depending on the type of information being compressed.

  The tiling module (510) optionally divides the picture into multiple tiles of the same or different sizes. For example, the tiling module (510) divides the picture along tile rows and tile columns that use the picture boundaries to define the horizontal and vertical boundaries of the tiles in the picture. Here, each tile is a rectangular area. H. In the H.265 / HEVC implementation, the encoder (500) divides the picture into one or more slices. Here, each slice includes one or more slice segments.

  The general coding control (520) receives feedback (not shown) from various modules of the encoder (500) in addition to the pictures of the input video signal (505). In general, the general coding control unit (520) sets other modules (tiling module (510), transform unit / scaling unit / quantization unit (530) to set and change coding parameters during coding. , Supplies a control signal (not shown) to the scaling / inverse transform unit (535), the intra picture estimation unit (540), the motion estimation unit (550), the intra / inter switch, and the like. Specifically, whether the general coding control unit (520) uses an aspect (for example, skip mode, block inversion) of intra BC prediction during encoding and an aspect (for example, skip mode, block) for intra BC prediction It can be determined how to use inversion). The general coding control (520) may also perform, for example, rate distortion analysis to evaluate intermediate results during coding. The general coding control unit (520) generates general control data (522) indicating the decisions made during the coding, so that the corresponding decoder can make a matching decision. The general control data (522) is provided to the header formatter / entropy encoder (590).

  If the current picture is predicted using inter-picture prediction, the motion estimator (550) estimates the motion of blocks of sample values in the current picture of the input video signal (505) relative to one or more reference pictures. . The motion estimator (550) may evaluate options for inverting a given reference region for inter-picture coded blocks, as described below. The decoded picture buffer (570) buffers one or more previously reconstructed coded pictures for use as reference pictures. When multiple reference pictures are used, the multiple reference pictures may be from different temporal directions or from the same temporal direction. The motion estimation unit (550) generates, as side information, motion data (552) such as MV data, a merge mode index value, reference picture selection data, and whether or not block inversion is used. The motion data (552) is provided to a header formatter / entropy encoder (590) and a motion compensator (555).

  The motion compensation unit (555) applies the MV to one or more reconstructed reference pictures from the decoded picture buffer (570). The motion compensation unit (555) generates motion compensated prediction for the current picture. If block inversion is used, the motion compensation unit (555) may consider the inversion for the prediction area (for the current block) relative to the reference area, as described below.

  In the separate path within the encoder (500), how the intra picture estimator (540) performs intra picture prediction for the block of sample values of the current picture of the input video signal (505) Decide. The current picture may be encoded in whole or in part using intra-picture coding. For intra-spatial prediction, the intra-picture estimator (540) uses the values of the current picture's reconstruction (538) to determine the current block in the current picture from adjacent previously reconstructed sample values of the current picture. Determine how to predict spatially the sample values of. Alternatively, for intra BC prediction using BV values, the intra picture estimation unit (540) estimates displacement of sample values of the current block to different candidate reference regions in the current picture. The candidate reference region may include reconstructed sample values or, in some implementations for BV estimation, original sample values from the input video. The intra picture estimates (540) may evaluate different options for inversion of the intra BC prediction area (for the current block) for each candidate reference area, as described below.

  The intra picture estimation unit (540), as side information, information indicating whether intra prediction uses spatial prediction or intra BC prediction, prediction mode direction (for intra spatial prediction), BV value (for intra BC prediction And intra prediction data (542) such as whether or not block inversion is used (for intra BC prediction). The intra prediction data (542) is provided to the header formatting unit / entropy coding unit (590) and the intra picture prediction unit (545).

  The intra picture prediction unit (545) spatially predicts the sample values of the current block of the current picture from the adjacent previously reconstructed sample values of the current picture according to the intra prediction data (542). Alternatively, for intra BC prediction, the intra picture predictor (545) predicts sample values of the current block using previously reconstructed sample values of the intra frame prediction reference region indicated by the BV value of the current block. Do. For intra BC prediction, the intra picture predictor (545) may consider the inversion for the intra BC prediction region (for the current block) relative to the reference region, as described below. In some cases, the BV value may be a BV predictor (predicted BV value). In other cases, the BV value may be different than its predicted BV value. If the chroma data for the picture has the same resolution as the luma data (e.g. if the format is YUV 4: 4: 4 format or RGB 4: 4: 4 format) then the BV value applied for the chroma block is luma block May be the same as the BV value applied for On the other hand, if the chroma data for the picture has a lower resolution than the luma data (e.g. if the format is YUV 4: 2: 0 format), then the BV value applied for the chroma block is reduced, if possible , Can be rounded to adjust for differences in chroma resolution (eg, by dividing the vertical and horizontal components of the BV value by 2 and truncating or rounding them to integer values).

  The intra / inter switch selects whether the prediction (558) for a given block is motion compensated prediction or intra picture prediction. For non-skip mode blocks, the difference (if any) between the block of prediction (558) and the corresponding portion of the original current picture of the input video signal (505) provides the value of the residual (518) Do. For non-skip mode blocks, during reconstruction of the current picture, reconstructed residual values are combined with the prediction (558) to approximate or accurate reconstruction of the original content from the video signal (505) (538) is generated. (With lossy compression, some information is lost from the video signal (505).)

  In the transform / scaling / quantization unit (530), the frequency transform unit transforms the spatial domain video information into frequency domain (ie spectral transform) data. For block-based video coding, the frequency transform unit predicts discrete cosine transform ("DCT"), its integer approximation, or another type of forward block transform (eg, discrete sine transform or its integer approximation) to residual data Apply to the block (or sample value data if the prediction 558 is null) to generate a block of frequency transform coefficients. The encoder (500) may also indicate that such a conversion step is skipped. The scaling unit / quantization unit scales and quantizes the transform coefficient. For example, the quantization unit may be a dead-zone scalar quantum with a quantization step size that varies on a frame basis, a tile basis, a slice basis, a block basis, a frequency specific unit, or another unit. Applies to frequency domain data. The quantized transform coefficient data (532) is provided to the header formatter / entropy encoder (590).

  In the scaling unit / inverse transform unit (535), the scaling unit / inverse quantization unit performs inverse scaling and inverse quantization on the quantized transform coefficients. The inverse frequency transform performs inverse frequency transformation to generate a block of reconstructed prediction residual values or sample values. For non-skip mode blocks, the encoder (500) combines the reconstructed residual values with the values of the prediction (558) (eg, motion compensated prediction values, intra picture prediction values) to reconstruct Generate For the skip mode block, the encoder (500) uses the value of the prediction (558) as the reconstruction (538).

  For intra-picture prediction, the values of the reconstruction (538) may be fed back to the intra-picture estimator (540) and the intra-picture predictor (545). Also, the values of the reconstruction (538) may be used for motion compensated prediction of subsequent pictures. The values of reconstruction (538) may be further filtered. A filtering control (560) determines how to perform deblock filtering and SAO filtering on the value of reconstruction (538) for a given picture of the video signal (505). A filtering control unit (560) generates filtering control data (562). The filtering control data (562) is provided to the header formatter / entropy encoder (590) and the merge / one or more filters (565).

  In the merge section / one or more filters (565), the encoder (500) merges content from different tiles into a reconstructed version of the picture. The encoder (500) selectively performs deblock filtering and SAO filtering according to the filtering control data (562) to adaptively smooth discontinuities across boundaries in a frame. Alternatively or additionally, other filtering (such as deringing filtering or ALF; not shown) may be applied. The tile boundaries may or may not be selectively filtered depending on the settings of the encoder (500), and the encoder (500) may code a syntax indicating whether such filtering has been applied or not It can be provided in the stream. The decoded picture buffer (570) buffers the reconstructed current picture for use in subsequent motion compensated prediction.

  The header formatting unit / entropy coding unit (590) includes general control data (522), quantized transform coefficient data (532), intra prediction data (542), motion data (552), and filtering control data ( Format and / or entropy encode 562). For intra prediction data (542), the header formatter / entropy encoder (590) may select BV predictor index values and entropy encode (for intra BC prediction). The header formatting unit / entropy coding unit (590) can also entropy encode syntax elements indicating whether block inversion is used for intra BC prediction (or motion compensation). In some cases, the header formatter / entropy encoder (590) also determines the BV difference of the BV value (BV difference to the BV predictor for the BV value) and then, for example, context adaptive binary arithmetic The encoding is used to entropy encode the BV difference. Specifically, BV differences are signaled for the skip mode intra BC prediction block.

  The header formatter / entropy encoder (590) provides the encoded data in an encoded video bitstream (595). The format of the coded video bitstream (595) is H.264. H.265 / HEVC format variants or extensions, Windows® Media Video format, VC-1 format, MPEG-x format (eg, MPEG-1, MPEG-2, or MPEG-4), H.264, H.264, H.264, H.264, H.264, H.264, H.264, H.264, H.264, H.264, H.264, H.264, H.264, H.264, H.264, H.264, H.264, H.264, H.264, H.264, H. 26x format (e.g., H.261, H.262, H.263, H.264) or another format.

  Depending on the implementation and the type of compression desired, modules of the encoder may be added, omitted, split into multiple modules or combined with other modules And / or similar modules may be substituted. In alternative embodiments, encoders with different modules and / or other configurations of modules perform one or more of the described techniques. Certain embodiments of the encoder typically use a variation or complement version of the encoder (500). The illustrated relationships between modules in the encoder (500) are indicative of the general flow of information in the encoder. Other relationships are not shown for the sake of brevity.

VI. Exemplary Video Decoder FIG. 6 is a block diagram of a generalized decoder (600) that may be implemented cooperatively with some described embodiments. The decoder (600) receives the encoded data in the encoded video bitstream (605) and generates an output including pictures for the reconstructed video (695). The format of the encoded video bitstream (605) is H.264. H.265 / HEVC format variants or extensions, Windows® Media Video format, VC-1 format, MPEG-x format (eg, MPEG-1, MPEG-2, or MPEG-4), H.264, H.264, H.264, H.264, H.264, H.264, H.264, H.264, H.264, H.264, H.264, H.264, H.264, H.264, H.264, H.264, H.264, H.264, H.264, H.264, H. 26x format (e.g., H.261, H.262, H.263, H.264) or another format.

  The decoder (600) is block based and uses an implementation dependent block format. The blocks may be further subdivided at various stages. For example, a picture may be divided into 64x64 blocks, 32x32 blocks, or 16x16 blocks, which in turn may be divided into smaller blocks of sample values. H. In the decoding implementation for the H.265 / HEVC standard, pictures are divided into CTU (CTB), CU (CB), PU (PB), and TU (TB).

  The decoder (600) decompresses the picture using intra picture decoding and / or inter picture decoding. Many of the components of the decoder (600) are used for both intra-picture decoding and inter-picture decoding. The exact operations performed by these components may vary depending on the type of information being decompressed.

  The buffer receives the encoded data in the encoded video bitstream (605) and makes the received encoded data available to the analyzer / entropy decoder (610). The analysis unit / entropy decoding unit (610) performs entropy decoding on the entropy-coded data by applying the inverse (eg, context adaptive binary arithmetic decoding) of the entropy coding usually performed in the encoder (500) Do. As a result of the analysis and entropy decoding, the analysis unit / entropy decoding unit (610) comprises general control data (622), quantized transform coefficient data (632), intra prediction data (642), motion data (652), And generate filtering control data (662). For intra prediction data (642), the analysis unit / entropy decoding unit (610) entropy decodes the BV predictor index value (for intra BC prediction). The analysis unit / entropy decoding unit (610) also entropy decodes syntax elements indicating whether block inversion is used for intra BC prediction (or motion compensation). In some cases, the parser / entropy decoder (610) also entropy decodes the BV difference of the BV value (eg, using context adaptive binary arithmetic decoding), and then the BV difference to a corresponding BV prediction. Combine with the child to reconstruct the BV value. In particular, for skip mode intra BC prediction blocks, BV differences are analyzed from the bitstream and combined with a BV predictor (eg, indicated by BV predictor index values) to reconstruct BV values.

  A general decoding control unit (620) receives general control data (622), and other modules (scaling unit / inverse conversion unit (635), intra picture prediction unit, in order to set and change decoding parameters during decoding (645), supplies a control signal (not shown) to the motion compensation unit (655) and the intra / inter switch etc.

  If the current picture is predicted using inter-picture prediction, the motion compensation unit (655) is for motion data (652) such as MV data, reference picture selection data, merge mode index value, and (for motion compensation) blocks Receive a syntax element indicating whether inversion is used. The motion compensation unit (655) applies the MV to one or more reconstructed reference pictures from the decoded picture buffer (670). If block inversion is used, the motion compensation unit (655) may consider the inversion for the prediction area (for the current block) relative to the reference area, as described below. The motion compensation unit (655) generates motion compensated prediction for the inter coding block in the current picture. The decoded picture buffer (670) stores one or more previously reconstructed pictures for use as reference pictures.

  In the separation path in the decoder (600), the intra picture prediction / prediction unit (645) is information indicating whether intra prediction uses spatial prediction or intra BC prediction, prediction mode direction (for intra spatial prediction), Intra prediction data (642) such as BV values (for intra BC prediction) and syntax elements (for intra BC prediction) indicating whether block inversion is used are received. For intra-spatial prediction, the intra-picture predictor (645) uses the values of the current picture's reconstruction (638) to generate current values from adjacent previously reconstructed sample values of the current picture according to prediction mode data. Spatially predict sample values of the current block of the picture. Alternatively, for intra BC prediction using BV values, the intra picture predictor (645) uses the previously reconstructed sample values of the intra frame prediction reference region indicated by the BV value of the current block to generate the current block. Predict sample values for For intra BC prediction, the intra picture predictor (645) may consider the inversion for the intra BC prediction region (for the current block) relative to the reference region, as described below.

  The intra / inter switch selects whether the prediction (658) for a given block is motion compensated prediction or intra picture prediction. For example, H. According to the 265 / HEVC syntax, intra / inter switches may be controlled based on one or more syntax elements encoded for CUs in a picture that may include intra prediction CUs and inter prediction CUs. For non-skip mode blocks, the decoder (600) combines the prediction (658) with the reconstructed residual values to generate a reconstruction (638) of content from the video signal. For skip mode blocks, the decoder (600) uses the value of the prediction (658) as the reconstruction (638).

  In order to reconstruct the residuals for the non-skip mode block, the scaling / inverse transform unit (635) receives and processes the quantized transform coefficient data (632). In the scaling unit / inverse transform unit (635), the scaling unit / inverse quantization unit performs inverse scaling and inverse quantization on the quantized transform coefficients. The inverse frequency transform performs inverse frequency transformation to generate a block of reconstructed prediction residual values or sample values. For example, the inverse frequency transform unit applies inverse block transform to the frequency transform coefficients to generate sample value data or prediction residual data. The inverse frequency transform may be an inverse DCT, its integer approximation, or another type of inverse frequency transform (eg, an inverse discrete sine transform or its integer approximation).

  For intra-picture prediction, the values of the reconstruction (638) may be fed back to the intra-picture predictor (645). For inter-picture prediction, the values of the reconstruction (638) may be further filtered. In the merge section / one or more filters (665), the decoder (600) merges content from different tiles into a reconstructed version of the picture. The decoder (600) selectively performs deblocking filtering and SAO filtering according to the filtering control data (662) and rules for filter adaptation to adaptively smooth discontinuities across boundaries in a frame. Run. Alternatively or additionally, other filtering (such as deringing filtering or ALF; not shown) may be applied. The tile boundaries may be selectively filtered or not filtered depending on the settings of the decoder (600) or syntax indications in the coded bit stream data. The decoded picture buffer (670) buffers the reconstructed current picture for use in subsequent motion compensated prediction.

  The decoder (600) may also include a post-processing filter. Post-processing filters (608) may include deringing filtering, adaptive winner filtering, film-grain reproduction filtering, SAO filtering, or another type of filtering.

  Depending on the implementation and the type of decompression desired, the modules of the decoder may be added, omitted, divided into several modules or combined with other modules And / or similar modules may be substituted. In alternative embodiments, decoders with different modules and / or other configurations of modules perform one or more of the described techniques. Particular embodiments of the decoder typically use a modified or complemented version of the decoder (600). The illustrated relationships between modules in the decoder (600) are indicative of the general flow of information in the decoder. Other relationships are not shown for the sake of brevity.

VII. Innovations in Intra-block Copy Prediction This section presents the features of intra-block copy ("BC") prediction. For example, some of these features relate to block inversion in which intra BC prediction blocks are inverted relative to a reference region that may be indicated by a block vector ("BV") value. Another feature relates to the signaling in skip mode, where the current intra BC prediction block uses signaled BV differences but has no residual data. In many situations, these features improve the coding efficiency of intra BC prediction blocks.

  In particular, the described innovation can improve rate-distortion performance in encoding predetermined "artificially" produced video content, such as screen capture content. Screen capture content generally includes recurring structures (eg, graphics, text characters), which provide the opportunity for intra BC prediction to improve performance. Screen capture content is usually encoded in a high chroma sampling resolution format (eg, YUV 4: 4: 4 or RGB 4: 4: 4), but a lower chroma sampling resolution format (eg, YUV 4: 2: 0) ) May be encoded. Common scenarios for encoding / decoding screen capture content include remote desktop conferencing and encoding / decoding of graphical overlays on natural video or other "mixed content" video.

A. Intra BC Prediction Mode, BV Values, and BV Prediction-Introduction For intra BC prediction, sample values of the current block in a picture are predicted using sample values in the same picture. The BV value indicates the displacement from the current block to the area in the picture ("reference area") that contains the sample values used for prediction. The reference area provides the value predicted for the current block ("intra BC prediction area"). The sample values used for prediction are the previously reconstructed sample values and thus are available at the encoder during encoding and at the decoder during decoding. The BV values may be included in the bitstream and signaled, and the decoder may use the BV values to determine a reference region (reconstructed at the decoder) in the picture to use for prediction. it can. Intra-BC prediction is a form of intra-picture prediction, and intra-BC prediction for blocks in a picture does not use any sample values other than sample values in the same picture.

  FIG. 7a shows intra BC prediction for the current block (760) in the current frame (710). The current block is a coding block ("CB") of a coding unit ("CU"), a prediction block ("PB") of a prediction unit ("PU"), and a conversion block (" It may be TB ") or other blocks. The size of the current block may be 64x64, 32x32, 16x16, 8x8, or some other size. More generally, the size of the current block is m × n, where each of m and n is an integer, and m and n may be equal to one another or may have different values. . Alternatively, the current block may have some other shape (eg, a region of a non-rectangular shaped encoded video object).

BV (761) indicates the displacement (or offset) from the current block (760) to the reference area (780) in the picture, including the sample values used for prediction. The reference area (780) indicated by BV (761) is sometimes referred to as the "match block" for the current block (760). The matching block may be the same as the current block (760) or may be an approximation of the current block (760). Assume that the upper left position of the current block is at position (x ₀ , y ₀ ) in the current frame, and that the upper left position of the reference area is at position (x ₁ , y ₁ ) in the current frame. BV indicates displaced _{(x 1 -x 0, y 1} -y 0). For example, if the upper left position of the current block is at position (256, 128) and the upper left position of the reference area is at position (176, 104), then the BV value is (−80, −24). In this example, a negative horizontal displacement indicates the left position of the current block and a negative vertical displacement indicates the position above the current block.

  Intra BC prediction can improve coding efficiency by using redundancy (such as a repetition pattern in a frame) using BC operation. The sample values of the current block are represented using BV values instead of directly encoding the sample values of the current block. Even if the sample values of the current block do not exactly match the sample values of the reference region indicated by the BV value, this difference may be negligible (perceptually unnoticeable) . Alternatively, if the difference is large, the difference may be encoded as residual data that may be compressed more efficiently than the original sample values for the current block.

  Collectively, the BV values of blocks encoded using intra BC prediction may consume a large number of bits. The BV values may be entropy coded to reduce the bit rate. In order to further reduce the bit rate for BV values, the encoder can use prediction of BV values. BV values often indicate redundancy. That is, the BV value of a given block is often similar to the BV value of the previous block in the picture, or even the same as the BV value of the previous block in the picture. For BV prediction, the BV value of a given block is predicted using a BV predictor. Then, the difference between the BV value of a given block and the BV predictor (i.e., BV difference) is entropy encoded. Typically, BV differences are calculated for the horizontal and vertical components of BV values and BV predictors. If BV prediction works well, the BV difference has a probability distribution that supports efficient entropy coding. H. In one draft version of the H.265 / HEVC standard (JCTVC-P 1005), the BV predictor is the BV value of the last encoded CU in the current CTU (ie, the previous intra BC prediction block in the current CTU) BV value of Alternatively, the BV predictor is selected among a plurality of available BV values (e.g., in adjacent regions around the current block).

  Figure 7b shows the BV (761) of the current block (760) in the frame and the BV (751) of the previous block (750) in the frame (710). The BV (751) of the previous block (750) is used as the BV predictor for the BV (761) of the current block (760). For example, when the BV value is (−80, −24) and the BV predictor is (−80, −32), the BV difference of (0, 8) is entropy encoded.

  The decoder receives and entropy decodes the entropy encoded BV difference of the BV value. The decoder also determines the BV predictor for the BV value. The BV predictor determined by the decoder is the same as the BV predictor determined by the encoder. The decoder combines the BV predictor with the decoded BV difference to reconstruct the BV value.

B. Block Inversion in Intra BC Prediction In the previous approach of intra BC prediction, the reference region indicated by the BV value provides the intra BC prediction region for the current block. That is, the sample value of the reference area is the intra BC predicted value for the current block.

  According to one aspect of the innovation described herein, the intra BC prediction region for the current block is inverted relative to the reference region. The BV value of the current block can indicate the reference area. Instead of using the reference region directly as the intra BC prediction region for the current block, the reference region may be flipped horizontally and / or vertically. In particular, block inversion can improve the coding efficiency of text characters in screen capture content.

1. Example of Block Inversions FIGS. 8a-8d, 9a-9c, and 10a-10c show examples of block inversion in intra BC prediction.

  FIG. 8a shows the current block (860) in the current picture (810). The current block (860) contains the text letter p which is shown in detail in FIG. 8b. BV (861) of the current block (860) indicates the displacement to the reference area (880) in the current picture (810). The reference area (880) contains the text character d which is shown in detail in FIG. 8c. Without inversion, the reference area (880) is a bad predictor for the current block (860). (The sample-to-sample difference between the current block (860) and the reference area (880) is large.)

  The encoder and decoder can use block inversion in intra BC prediction to improve coding efficiency. For example, the reference area (880) may be flipped horizontally and vertically as shown in the inverted reference area (881) of FIG. 8d. In this example, when the reference area (880) is inverted horizontally and / or vertically, the inverted reference area (881) exactly matches the current block (860). (Ie, this intra BC prediction region is optimal for the current block (860) and the residual contains only zero valued samples).

  Similarly, the reference area (880) may be flipped horizontally. For example, if a given block contains the text letter b, horizontal flipping of the reference area (880) can result in a flipped reference area that exactly matches the given block. Alternatively, the reference area (880) may be flipped vertically. For example, if a given block contains text letter q, vertical flipping of the reference area (880) can result in a flipped reference area that exactly matches the given block.

  With block inversion, for many fonts, blocks containing text characters in a set of text characters (eg, a set of b, d, p and q or a set of u and n) are identical to the text characters. It can be accurately predicted from reference regions that contain other text characters in the set. For other fonts, blocks containing text characters in a set of text characters can be roughly predicted from a reference area containing another text character in the same set of text characters. More generally, block inversion can improve the coding efficiency of various alphabetic text characters or other patterns in screen content.

  Thus, with block inversion, blocks containing text characters (or other patterns) are intra BC predicted, even if the text characters (or other patterns) have not previously appeared in the picture. This can improve coding efficiency as compared to other methods of coding blocks. Alternatively, even if the text character (or other pattern) has not previously appeared in the picture, intra BC prediction is closer to this block than the reference region containing the same text character (or other pattern) An inverted reference region can be used. BV values for closer inverted regions may be encoded much more efficiently than BV values for more distant reference regions. For example, for the current block containing text letter q, the first candidate reference region containing the same text letter q may be indicated by the BV value (-280, -24) while the second containing the different text letter p It is assumed that the candidate reference region of can be indicated by the BV value (-32, 0). The second candidate reference area may be flipped horizontally to exactly match the current block. The BV values for the second candidate reference area are encoded (with fewer bits) more efficiently than the BV values for the first candidate reference area, even considering signaling of syntax elements indicating block inversion. obtain.

  As shown in FIGS. 9a-9c, the block and reference areas may include multiple text characters. FIG. 9a shows the current block (960) containing text characters dl. The BV value of the current block (960) indicates the displacement to the reference area (980) in the same picture. The reference area (980) contains text character groups lb, as shown in detail in FIG. 9b. Without inversion, the reference region (980) is a poor predictor for the current block (960). However, if the reference area (980) is flipped horizontally, the inverted reference area (981) will exactly match the current block (960).

  In the above example, the block and reference area contain the entire text character. As shown in FIGS. 10a-10c, the blocks and reference regions may instead include text characters, symbols, or one or more portions of symbols. FIG. 10a shows the current block (1060) which contains a portion of the text letter L. The BV value of the current block (1060) indicates the displacement to the reference area (1080) in the same picture. The reference area (1080) comprises a portion of the text letter F, as shown in detail in FIG. 10b. Without inversion, the reference area (1080) is a poor predictor for the current block (1060). However, if the reference area (1080) is vertically inverted, then the inverted reference area (1081) exactly matches the current block (1060).

2. Exemplary Inversion Operation When block inversion is used in intra BC prediction, the intra BC prediction region for the block is inverted relative to the reference region for the block. The block inversion operation may be implemented in various ways, depending on the implementation.

  According to one approach to performing a block inversion operation, when determining the intra BC prediction region for the current block, the encoder or decoder (a) determines the reference region, (b) inverts the reference region, and then (C) Assign sample values at each position of the inverted reference region to sample values at their respective positions in the intra BC prediction region. For example, for a 16x16 block, the encoder or decoder determines the 16x16 reference area indicated by the BV value of that block and then flips the 16x16 reference area horizontally and / or vertically. This involves making a copy of the inverted 16 × 16 reference area. The sample values at the position of the inverted reference region are then assigned to the sample values at the same position of the intra BC prediction region (eg, the sample values at the position (0, 0) of the inverted reference region are intra BC prediction) The sample values at the position (0, 1) of the reference position (0, 1) assigned to the sample values at the position (0, 0) of the area and the inverted reference areas are assigned to the sample values at the position (0, 1) of the intra BC prediction area Etc.).

  According to a second approach to performing a block inversion operation, when determining an intra BC prediction region for the current block, the encoder or decoder (a) determines a reference region, (b) at each position of the reference region. Sample values are assigned to sample values at their respective positions in the intra BC prediction region, and then (c) invert the intra BC prediction region. For example, for a 16x16 block, the encoder or decoder determines the 16x16 reference region indicated by the BV value of that block. Sample values at the position of the reference region are assigned to sample values at the same position of the intra BC prediction region (for example, sample values at the position (0, 0) of the reference region are position (0, 0) of the intra BC prediction region Assigned to the sample value in (1), the sample value at the position (0, 1) of the reference area is assigned to the sample value at the position (0, 1) of the intra BC prediction area, etc. The encoder or decoder then inverts the 16 × 16 intra BC prediction region horizontally and / or vertically. This involves making a copy of the inverted 16 × 16 intra BC prediction region.

  According to a third approach to performing a block inversion operation, the encoder and decoder avoid creating an intermediate copy of the reference region or intra BC prediction region. When determining the intra BC prediction region for the current block, the encoder or decoder determines (a) the reference region and then (b) sample values at each position of the reference region to the corresponding values of the intra BC prediction region. Assigned to sample values at locations, where the corresponding locations take into account block inversion. If horizontal inversion is used, the first column of the reference area provides the last column of the intra BC prediction area, and the second column of the reference area the second to last column of the intra BC prediction area Provide, etc. If vertical inversion is used, the first row of the reference region provides the last row of the intra BC prediction region, and the second row of the reference region the second to last row of the intra BC prediction region. Provide, etc. If both horizontal and vertical flips are used, the position of the reference area may be scanned in reverse order in the horizontal and vertical directions when assigning sample values to the position of the intra BC prediction area. For example, for a 16x16 block, the encoder or decoder determines the 16x16 reference region indicated by the BV value of that block. Sample values at the position of the reference region are assigned to sample values at corresponding positions of the intra BC prediction region in reverse order in the horizontal and / or vertical direction (eg, sample values at position (0, 0) of the reference region are , Assigned to sample values at the position (15, 15) of the intra BC prediction region, and sample values at the position (0, 1) of the reference region are assigned to the sample values at the position (15, 14) of the intra BC prediction region , Etc.).

3. Exemplary Signaling for Block Inversion When block inversion is enabled for intra BC prediction, the determination of whether or not to use block inversion may be signaled in various ways, depending on the implementation.

  Block inversion may be enabled for sequences of video, pictures, or other units. Sequence layer syntax elements (eg, in SPS), picture layer syntax elements (eg, in PPS), or slice header layer syntax elements (eg, in slice segment headers) have block inversion enabled. Can indicate whether it is disabled or not. Alternatively, block inversion may be enabled for several profiles or levels of encoding and decoding. The decision to enable block inversion may be made on a per-direction basis (eg, horizontal block inversion only, vertical block inversion only, or both horizontal block inversion and vertical block inversion). If block inversion is enabled, additional syntax elements signal when and how block inversion is used.

  If only vertical inversion is enabled, the flag value may indicate whether vertical inversion is used during intra BC prediction. If only horizontal inversion is enabled, the flag value may indicate whether horizontal inversion is used during intra BC prediction. If both vertical and horizontal flips are enabled, the two flag values indicate whether flips are used for vertical and horizontal flips during intra BC prediction Each flag indicates a decision on the direction of the inversion. Alternatively, one syntax element having multiple values may be used (eg, possible values are only vertical flip, only horizontal flip, both vertical and horizontal flip, Or indicates no inversion).

  A syntax element (eg, a flag value) indicating whether block inversion is used for the current block may be signaled by being included in the bitstream along with other syntax elements for the current block. For example, one or more syntax elements for block inversion for a PU are signaled for the PU. Alternatively, syntax elements indicating whether block inversion is used for the current block may be signaled by being included in the bitstream for larger blocks that include the current block. For example, one or more syntax elements for block inversion for one or more PUs may be signaled for a CU that includes one or more PUs. Alternatively, syntax elements indicating whether or not block inversion is used for the current block are included in the bitstream and signaled at some other level.

  Syntax elements indicating whether or not block inversion is used for the current block may be entropy coded. For example, flag values for the current block are encoded using context adaptive binary arithmetic coding and decoded using context adaptive binary arithmetic decoding. Alternatively, different forms of entropy coding may be used, or syntax elements may be signaled as fixed length values.

  One or more syntax elements indicating whether block inversion is used for the current block may be included in the bitstream and separately conditionally signaled. For example, a flag value indicating whether block inversion is used may be signaled if the current block is intra BC predicted, but not signaled if the current block is not intra BC predicted. Alternatively, one or more syntax elements indicating whether block inversion is used for the current block may be jointly encoded with other syntax elements in the bitstream. For example, a flag value indicating whether block inversion is used may be jointly encoded with a flag value indicating whether the current block is intra BC predicted.

4. Exemplary Coding Using Block Inversion in Intra BC Prediction FIG. 11 shows an exemplary technique (1100) for block inversion in intra BC prediction during encoding. An image encoder or video encoder such as the encoder described with reference to FIG. 3 or FIGS. 5a-5b may perform the technique (1100).

  The encoder determines 1110 an intra BC prediction region for the current block in the picture based on the reference region in the picture. The current block may be a PU, a CU, or another block. The BV value of the current block identified in the BV estimation can indicate the displacement to the reference region in the picture. The intra BC prediction region is inverted relative to the reference region. For example, the intra BC prediction region is horizontally and / or vertically inverted with respect to the reference region. An example of an approach for performing a block reversal operation is described above (see Section VII.B.2).

  The encoder encodes (1120) the current block using the intra BC prediction region and outputs (1130) the data encoded in the bitstream. The encoded data includes an indication of whether the intra BC prediction region is inverted relative to the reference region. For example, an indication is one or more syntax elements in a bitstream. Examples of approaches to signal whether block inversion is used are described above (see Section VII.B.3).

  The encoder may similarly encode other intra BC prediction blocks on a block-by-block basis, with or without block inversion, for slices, tiles, or pictures.

5. Exemplary Decoding Using Block Inversion in Intra BC Prediction FIG. 12 shows an exemplary technique (1200) for block inversion in intra BC prediction during decoding. An image decoder or video decoder such as the decoder described with reference to FIG. 4 or FIG. 6 may perform the technique (1200).

  The decoder receives (1210) the encoded data in the bitstream. The encoded data includes an indication of whether the intra BC prediction region for the current block in the picture is inverted relative to the reference region in the picture. The current block may be a PU, a CU, or another block. For example, an indication is one or more syntax elements in a bitstream. Examples of approaches to signal whether block inversion is used are described above (see Section VII.B.3).

  The decoder determines 1220 an intra BC prediction region for the current block in the picture based on the reference region in the picture. The BV value of the current block can indicate the displacement to the reference area. The intra BC prediction region is inverted relative to the reference region. For example, the intra BC prediction region is horizontally and / or vertically inverted with respect to the reference region. An example of an approach for performing a block reversal operation is described above (see Section VII.B.2). The decoder decodes 1230 the current block using the intra BC prediction region.

  The decoder may decode other intra BC prediction blocks as well, block by block, with or without block inversion, for slices, tiles, or pictures.

C. Skip Mode for Intra BC Prediction In some previous approaches of intra BC prediction, a flag for the current CU indicates whether the current CU is coded in intra BC prediction mode. If the current CU is encoded in intra BC prediction mode, a second flag for the current CU indicates whether the current CU has residual data. This scheme of signaling intra BC prediction blocks without residual data is not efficient in many screen content coding / decoding scenarios.

  According to another aspect of the innovation described herein, the encoder and decoder use a flag to signal intra BC prediction blocks that do not have residual data. In skip mode, the intra BC prediction block uses BV values signaled as BV differences are included in the bitstream, but has no residual data in the bitstream. In particular, for screen capture content, intra BC prediction blocks without residual data often occur. Using one flag (instead of multiple flags) to signal intra BC prediction blocks without residual data is efficient in such a scenario.

1. Exemplary Signaling for Intra BC Prediction Skip Mode In an exemplary implementation, one flag in the bitstream indicates whether the current block is an intra BC prediction block in skip mode. If the current block is not an intra BC prediction block in skip mode, a second flag in the bitstream indicates whether the current block is an intra BC prediction block (not in skip mode). If the current block is not an intra BC prediction block, one or more other syntax elements in the bitstream indicate the mode of the current block (eg, a flag as to whether or not it is a temporal skip mode, intra spatial prediction mode Flag about whether or not it is inter picture mode, flag about whether it is intra spatial prediction mode or inter picture mode). If the current block is an intra BC prediction block in skip mode, the second flag and other syntax elements are not present in the bitstream for the current block.

  A flag indicating whether the current block is an intra BC prediction block in skip mode is signaled at the block level. The current block may be a CU with size 2N × 2N. For example, in the case of 16 × 16 CU, N is 8. Alternatively, the current block may be a PU or other type of block. Other flags and syntax elements that indicate the mode of the current block may also be included in the bitstream and signaled at the block level.

  In some exemplary implementations, intra BC prediction blocks in skip mode include BV differences in the bitstream but do not have residual data. Alternatively, the intra BC prediction block in skip mode may use the predicted BV value (and thus not include the BV difference in the bitstream).

2. Exemplary Coding with Intra BC Prediction Skip Mode FIG. 13 shows an exemplary technique (1300) for coding an intra BC prediction block in skip mode. An image encoder or video encoder such as the encoder described with reference to FIG. 3 or FIGS. 5a-5b may perform the technique (1300).

  The encoder determines 1310 the BV value of the current block (eg, CU, PU) in the picture, eg, using BV estimation. The BV value of the current block indicates the displacement to the reference area in the picture. The encoder then uses the BV value of the current block and the BV predictor of the current block to determine 1320 the BV difference of the current block. The BV predictor may be selected by a rule, or the encoder may select a BV predictor from among a plurality of BV predictor candidates.

  The encoder encodes the current block using intra BC prediction using BV values (1330). In an exemplary implementation, intra BC prediction may include determining an intra BC prediction region for the current block using a reference region, where the intra BC prediction region is inverted relative to the reference region ing. The intra BC prediction region may be horizontally and / or vertically inverted with respect to the reference region. Options for performing block inversion operations and signaling that block inversion is used are described above. Alternatively, the encoder does not use block inversion in intra BC prediction.

  The encoder outputs data encoded in the bitstream (1340). The encoded data includes a flag indicating that the current block is encoded in skip mode using intra BC prediction. Since the current block is an intra BC prediction block in skip mode, the bitstream includes the BV difference of the current block but does not include residual data of the current block. If the encoder selects a BV predictor from among the plurality of BV predictor candidates, the bitstream includes an index value indicating the selected BV predictor candidate for use as a BV predictor for the current block.

  FIG. 14 shows an example technique (1400) for encoding blocks in a picture in skip mode and / or other modes using intra BC prediction. An image encoder or video encoder such as the encoder described with reference to FIG. 3 or FIGS. 5a-5b may perform the technique (1400).

  Initially, the encoder gets the next block (1410) and determines whether to encode this block in skip mode using intra BC prediction (1420). For example, the encoder evaluates whether intra BC prediction provides sufficient coding efficiency for the block and evaluates whether residual data contains large values. Alternatively, the encoder considers other criteria.

  The encoder may include in the bitstream a flag indicating whether the block is encoded in skip mode using intra BC prediction and may signal. For intra BC prediction blocks in skip mode, the encoder blocks in skip mode with intra BC prediction using the operations shown in steps (1310) to (1330) of FIG. 13 or using another approach. Are encoded (1430). As it is an intra BC prediction block in skip mode, this block has BV differences in the bitstream but no residual data in the bitstream.

  If not (the block is not an intra BC prediction block in skip mode), the encoder encodes the block in another mode (1440). Another mode may be intra BC prediction non-skip mode, intra spatial prediction mode, inter picture mode, or other modes. In this case, one or more other syntax elements may indicate the mode of the block. For example, after a first flag that indicates whether the block is encoded in skip mode using intra BC prediction (in response to the decision in step (1420)), the block uses intra BC prediction If not coded in skip mode, the second flag indicates whether the block is coded in non-skip mode using intra BC prediction. If the block is not coded in non-skip mode using intra BC prediction, one or more other syntax elements indicate the coding mode of the block. For example, a flag indicates whether the prediction mode of the block is intra spatial prediction or inter picture prediction.

  In some exemplary implementations, there is a further advantage of using a flag that indicates whether the block is intra BC predicted in skip mode. In some cases, signaling of syntax elements that indicate intra BC prediction block segmentation modes (eg, 2Nx2N, 2NxN, Nx2N, or NxN) may be avoided. For example, if the block is encoded in non-skip mode using intra BC prediction, the encoder signals by including a syntax element indicating the division mode of the block in the bitstream. On the other hand, if the block is encoded in skip mode using intra BC prediction, the encoder skips signaling that a syntax element indicating the division mode of the block is included in the bitstream, and The split mode is instead considered to have a defined value (e.g. 2Nx2N). Therefore, in these cases, it is also signaled that the flag to which the block is marked as being intra BC predicted in skip mode signals that the division mode of the block has a defined value.

  Also, in many cases, signaling of a flag indicating the presence or absence of residual data of the block may be avoided. Of course, if the block is coded in skip mode using intra-BC prediction, the encoder skips including in the bitstream a flag indicating the presence or absence of residual data of the block and signaling. (A flag on which to mark the block as intra BC predicted in skip mode signals such information.) The residual data of the block is considered not to be in the bitstream.

  The signaling of a flag indicating the presence or absence of residual data may be avoided in other cases as well. In particular, if the block is encoded in non-skip mode using intra BC prediction and the division mode of the block has a defined value (eg 2N × 2N), then the encoder Skipping signaling by including a flag indicating presence or absence of difference data in the bitstream. In this case, the residual data of the block is considered to be in the bitstream. (If the division mode of the block is a defined value and the block has no residual data, the block will be an intra BC prediction block in skip mode, as indicated by the previous flag. Finally, if the block is coded in non-skip mode using intra BC prediction, and the division mode of the block does not have a defined value, the encoder checks for the presence or absence of residual data of the block. The flag to indicate is included in the bit stream and signaled.

  The encoder determines (1450) whether to proceed to the next block in the picture. When proceeding to the next block, the encoder gets 1410 the next block and continues encoding.

  The encoder may repeat the technique (1400) for each picture, for each tile, for each slice, or in some other unit.

3. Exemplary Decoding Using Intra BC Prediction Skip Mode FIG. 15 shows an exemplary technique (1500) for decoding an intra BC prediction block in skip mode. An image decoder or video decoder such as the decoder described with reference to FIG. 4 or FIG. 6 may perform the technique (1500).

  The decoder receives (1510) the encoded data from the bitstream. The encoded data includes a flag indicating that the current block (eg, CU, PU) in the picture is encoded in skip mode using intra BC prediction. Since the current block is an intra BC prediction block in skip mode, the bitstream includes the BV difference of the current block but does not include residual data of the current block.

  The decoder uses the BV difference of the current block and the BV predictor of the current block to determine 1520 the BV value of the current block. The BV value of the current block indicates the displacement to the reference area in the picture. BV predictors may be selected by rules. Alternatively, the decoder uses the index value in the bitstream to select which BV predictor candidate to use as the BV predictor for the current block, and then uses the BV predictor among the plurality of BV predictor candidates. Can be selected.

  The decoder decodes (1530) the current block using intra BC prediction using BV values. In an exemplary implementation, intra BC prediction may include determining an intra BC prediction region for the current block using a reference region, where the intra BC prediction region is inverted relative to the reference region ing. The intra BC prediction region may be horizontally and / or vertically inverted with respect to the reference region. Options for performing block inversion operations and signaling that block inversion is used are described above. Alternatively, the decoder does not use block inversion in intra BC prediction.

  FIG. 16 shows an example technique (1600) for decoding blocks in a picture in skip mode and / or other modes using intra BC prediction. An image decoder or video decoder such as the decoder described with reference to FIG. 4 or FIG. 6 may perform the technique (1600).

  First, the decoder obtains encoded data for the next block (1610) and determines whether to decode this block in skip mode using intra BC prediction (1620). For example, the decoder receives and analyzes a flag in the bitstream indicating whether the block is encoded in skip mode using intra BC prediction.

  For intra BC prediction blocks in skip mode, the decoder may block in skip mode with intra BC prediction using the operations shown in steps (1520) and (1530) of FIG. 15 or using another approach. (1630). As it is an intra BC prediction block in skip mode, this block has BV differences in the bitstream but no residual data in the bitstream.

  Otherwise (if the block is not an intra BC prediction block in skip mode), the decoder decodes the block in another mode (1640). Another mode may be intra BC prediction non-skip mode, intra spatial prediction mode, inter picture mode, or other modes. In this case, one or more other syntax elements may indicate the mode of the block. For example, after a first flag indicating whether the block (according to the decision in step (1620)) is coded in skip mode using intra BC prediction, the block uses intra BC prediction If not coded in skip mode, the second flag indicates whether the block is coded in non-skip mode using intra BC prediction. If the block is not coded in non-skip mode using intra BC prediction, one or more other syntax elements indicate the coding mode of the block. For example, a flag indicates whether the prediction mode of the block is intra spatial prediction or inter picture prediction.

  As noted in the previous section, in some exemplary implementations, there is an additional advantage of using a flag to indicate whether a block is intra BC predicted in skip mode. For example, if the block is encoded in non-skip mode using intra BC prediction, the bitstream includes syntax elements that indicate the division mode of the block. On the other hand, if the block is encoded in skip mode using intra BC prediction, the bitstream does not include a syntax element indicating the block division mode. The decoder estimates that the division mode of the block is a defined value (eg, 2N × 2N).

  Also, in many cases, signaling of a flag indicating the presence or absence of residual data of the block may be avoided. Of course, if the block is coded in skip mode using intra BC prediction, the bitstream does not include a flag indicating the presence or absence of residual data of the block. Instead, the decoder infers that there is no residual data of the block in the bitstream from the flag that marks the block as intra BC predicted in skip mode.

  The signaling of a flag indicating the presence or absence of residual data may be avoided in other cases as well. In particular, if the block is coded in non-skip mode using intra BC prediction and the division mode of the block has a defined value (eg 2N × 2N), then the bitstream is of the block Does not include a flag indicating the presence or absence of residual data. In this case, the decoder estimates that the residual data of the block is in the bitstream. Finally, if the block is encoded in non-skip mode using intra BC prediction, and the division mode of the block does not have a defined value, then the bitstream will be checked for the presence or absence of residual data for the block. Contains a flag to indicate.

  The decoder determines whether to proceed to the next block in the picture (1650). When proceeding to the next block, the decoder obtains encoded data for the next block (1610) and continues decoding.

  The decoder may repeat the technique (1600) for each picture, for each tile, for each slice, or in some other unit.

4. Exemplary Syntax for a Coding Unit FIGS. 17a and 17b show a syntax structure (1700) for a coding unit ("CU") according to the conventional approach. The syntax elements shown in the syntax structure (1700) are defined in JCT VC-P 1005. The selected syntax elements are summarized here.

  The syntax element intra_bc_flag may be signaled for a CU when intra BC prediction is enabled (which is indicated by intra_block_copy_enabled_flag). The syntax element intra_bc_flag indicates whether the CU is encoded in the intra BC prediction mode. If the value of intra_bc_flag is 1, the CU is encoded in intra BC prediction mode. If the value of intra_bc_flag is 0, the CU is not coded in intra BC prediction mode. In this case (if intra_bc_flag is 0), there is a syntax element pred_mode_flag, which indicates whether the CU is coded in inter prediction mode or in intra spatial prediction mode.

  If the block is intra BC predicted (and few others), then the bitstream contains part_mode syntax elements for the CU. The part_mode syntax element indicates the division mode of the CU (ie, 2N × 2N, 2N × N, N × 2N, N × N).

  Then, the CU syntax structure (1700) may be the BV value for the partition (if CU is intra BC predicted), intra prediction direction information (if CU is intra space predicted), or prediction unit information (CU (If it is inter-predicted). The CU syntax structure (1700) then, in some cases, includes the rqt_root_cbf syntax element. In particular, if the CU is intra BC predicted, there is an rqt_root_cbf syntax element. The rqt_root_cbf syntax element indicates whether a transform_tree () syntax structure exists for a CU. The transform_tree () syntax structure is for residual data for a CU. If rqt_root_cbf is 1, then a transform_tree () syntax structure exists for the CU. If rqt_root_cbf is 0, no transform_tree () syntax structure exists for the CU. If rqt_root_cbf does not exist, the value of rqt_root_cbf is assumed to be 1. Therefore, as described above, in this approach, two flags (intra_bc_flag and rqt_root_cbf) are used to indicate the intra BC prediction block in the skip mode.

  FIG. 18 shows a new syntax structure (1800) for a coding unit that may be coded as an intra BC prediction block in skip mode. The changes to the syntax structure (1700) shown in FIGS. 17a and 17b are highlighted in FIG.

  intra_bc_skip_flag may be signaled for a CU when intra BC prediction is enabled (which is indicated by intra_block_copy_enabled_flag). The syntax element intra_bc_skip_flag indicates whether the CU is an intra BC prediction block in the skip mode. When the CU is intra BC predicted in skip mode (intra_bc_skip_flag is 1), syntax element intra_bc_flag and syntax element pred_mode_flag are omitted as in syntax element rqt_root_cbf. Also, in this case, the division mode of the CU has a defined value 2N × 2N.

  On the other hand, when the syntax element intra_bc_skip_flag indicates that the CU is not intra BC predicted in the skip mode (intra_bc_skip_flag is 0), the syntax element intra_bc_flag is present. The syntax element intra_bc_flag indicates whether the CU is encoded in intra BC prediction mode, as described with reference to FIGS. 17a and 17b. Furthermore, if intra_bc_flag is 0, there is a syntax element pred_mode_flag, which is either CU encoded in inter prediction mode or intra as described with reference to FIGS. 17a and 17b. It indicates whether it is encoded in spatial prediction mode.

  If the block is intra BC predicted in non-skip mode (ie, if intra_bc_skip_flag is 0 and intra_bc_flag is 1), then the bitstream contains part_mode syntax elements for the CU. The part_mode syntax element indicates a division mode (for example, 2N × 2N, 2N × N, N × 2N, N × N) of a CU. On the other hand, if the block is intra BC predicted in skip mode, the part_mode syntax element is included in the bitstream for the CU and not signaled. Instead, it is assumed that the split mode has a defined value of 2N × 2N. Thus, for such a block, intra_bc_skip_flag signals that the split mode of the CU has a defined value of 2N × 2N, and separate signaling of part_mode syntax elements is avoided.

  In the section omitted from FIG. 18, the syntax structure (1800) is then the BV value for the partition (if CU is intra BC predicted), intra prediction direction information (CU is intra spatial predicted) Or predicted unit information (if CU is inter-predicted).

  Finally, the syntax structure (1800) includes, in some cases, rqt_root_cbf syntax elements. In particular, if the CU is intra BC predicted in non-skip mode, there is a rqt_root_cbf syntax element unless the split mode of the CU is a defined value of 2N × 2N. If the rqt_root_cbf syntax element is present, the value of the rqt_root_cbf syntax element indicates whether a transform_tree () syntax structure exists for a CU, as described with reference to FIGS. 17a and 17b. On the other hand, when (1) CU is intra BC predicted in skip mode, or (2) CU is intra BC predicted, and the division mode of CU is a predetermined value of 2N × 2N, There is no rqt_root_cbf syntax element. If the CU is intra BC predicted in skip mode, the value of rqt_root_cbf is assumed to be 0 (no residual data for the CU). Alternatively, if the CU is intra BC predicted and the division mode of the CU is a defined value of 2N × 2N, the value of rqt_root_cbf is estimated to be 1 (there is residual data for the CU )

  In view of the many possible embodiments to which the disclosed principles of the invention can be applied, it should be appreciated that the illustrated embodiments are only preferred examples of the invention, and which therefore limit the scope of the invention. It should not be interpreted as Instead, the scope of the invention is defined by the claims. Accordingly, we claim as our invention all that comes within the scope and spirit of the claims.

Claims (10)

A method in a computing device having a video decoder or an image decoder, comprising:
Receiving from the bitstream encoded data including a flag indicating that the current block in the picture is encoded in skip mode using intra block copy (BC) prediction, said bits A stream includes block vector (BV) differences of the current block but does not include residual data of the current block;
Determining the BV value of the current block using the BV difference of the current block and the BV predictor of the current block, wherein the BV value of the current block is to a reference region in the picture Showing the displacement of the
Decoding the current block using intra BC prediction using the BV values;
Method including.   The method of claim 1, wherein the bitstream further comprises an index value indicating selection of BV predictor candidates from a set of BV predictor candidates for use as the BV predictor for the current block. Regarding subsequent blocks in the picture:
The first flag indicates whether the subsequent block is encoded in skip mode using intra BC prediction,
If the subsequent block is not coded in skip mode using intra BC prediction, the second flag indicates whether the subsequent block is coded in non skip mode using intra BC prediction The method of claim 1 which is shown. Regarding subsequent blocks in the picture:
If the subsequent block is encoded in non-skip mode using intra BC prediction, the bitstream includes a syntax element indicating a division mode of the subsequent block,
When the subsequent block is encoded in skip mode using intra BC prediction, the bitstream does not include the syntax element indicating the division mode of the subsequent block, and the division mode of the subsequent block The method according to claim 1, wherein has a defined value. Regarding subsequent blocks in the picture:
When the subsequent block is encoded in a skip mode using intra BC prediction, the bitstream does not include a flag indicating presence or absence of residual data of the subsequent block, and the residual data of the subsequent block The method according to claim 1, wherein is considered not to be in the bitstream. Regarding the subsequent block in the picture:
If the subsequent block is encoded in non-skip mode using intra BC prediction, and the division mode of the subsequent block has a defined value, then the bitstream is of residual data of the subsequent block. Not including the flag indicating presence or absence, the residual data of the subsequent block is considered to be in the bitstream;
If the subsequent block is encoded in non-skip mode using intra BC prediction, and the division mode of the subsequent block does not have the determined value, then the bitstream is of the subsequent block. The method according to claim 5, further comprising the flag indicating presence or absence of residual data. The intra BC prediction is
Using the reference region to determine an intra BC prediction region for the current block, the intra BC prediction region comprising determining inverted with respect to the reference region. Item 2. The method according to Item 1.   The method according to claim 7, wherein the intra BC prediction region is horizontally and / or vertically inverted with respect to the reference region.   A computing device adapted to perform the method according to any one of the preceding claims.   A program that causes a computing device to perform the method according to any one of the preceding claims.

Images