JP2018088715A - Block flipping and skip mode in intra block copy prediction - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide innovations in the area of encoding or decoding of blocks using intra block copy (BC) prediction.SOLUTION: A method according to an embodiment includes the steps of: receiving, from a bitstream, encoded data which includes a flag indicating that a current block in a picture is encoded in a skip mode using intra block copy (BC) prediction, the bitstream including a block vector (BV) differential of the current block but lacking residual data of the current block; determining a BV value of the current block using the BV differential of the current block and a BV predictor of the current block, the BV value of the current block indicating a displacement to a reference region in the picture; and decoding the current block using intra BC prediction with the BV value.SELECTED DRAWING: Figure 15

Description

  Engineers use compression (also called source coding) to reduce the bit rate of digital video. Compression reduces the cost of storing and transmitting video information by converting the video information to a lower bit rate format. Decompression (also called decoding) reconstructs the version of the original information from the compressed format. A “codec” is an encoder / decoder system.

  For the last 20 years, ITU-TH H.261 standard, H.264. 262 (MPEG-2 or ISO / IEC 13818-2) standard; H.263 standard, and H.264 standard. H.264 (MPEG-4 AVC or ISO / IEC 14496-10) standard, MPEG-1 (ISO / IEC 11172-2) standard, MPEG-4 Visual (ISO / IEC 14496-2) standard, and SMPTE 421M (VC -1) Various video codec standards including standards have been adopted. More recently, H.C. The H.265 / HEVC (ITU-T H.265 or ISO / IEC 23008-2) standard is approved. H. Extensions to the H.265 / HEVC standard (eg, relating to scalable video encoding / decoding, relating to video encoding / decoding with higher fidelity in terms of sample color depth or chroma sampling rate, relating to screen capture content, (Or multi-view encoding / decoding) is currently being developed. Video codec standards typically define options for the syntax of an encoded video bitstream that detail parameters in the encoded video bitstream when specific features are used in encoding and decoding . In many cases, video codec standards also provide details regarding decoding operations that the decoder should perform to achieve consistent results in decoding. Apart from codec standards, various proprietary codec formats define other options for the syntax of the encoded video bitstream and corresponding decoding operations.

  Intra-block copy (“BC”) This is a prediction mode that is being considered for H.265 / HEVC extension. For intra BC prediction mode, the sample value of the current block in a picture is predicted using previously reconstructed sample values in the same picture. The block vector (“BV”) indicates the displacement from the current block to a region in the picture that contains previously reconstructed sample values used for prediction. The BV is signaled by being included in the bitstream. Intra BC prediction is a form of intra picture prediction (intra-picture prediction), and intra BC prediction for blocks in a picture does not use any sample values other than the sample values in the same picture.

  Intra BC prediction mode is currently H.264. H.265 / HEVC standard, Although implemented in some reference software for the H.265 / HEVC standard, the intra BC prediction mode has several problems. In particular, coding of blocks with predictable BC displacement is not efficiently processed, and intra BC prediction for content with inverted patterns is not efficiently processed.

  In summary, the detailed description presents innovations in the area of block coding or decoding using intra block copy (“BC”) prediction. For example, some of the innovations relate to block inversion, where an intra BC prediction block is inverted with respect to a reference region that may be indicated by a block vector (“BV”) value. Another innovation relates to skip mode signaling where the current intra BC prediction block uses the signaled BV difference but has no residual data. In many situations, this innovation improves the coding efficiency of intra BC prediction blocks.

  According to a first aspect of the innovation described herein, an image encoder or a video encoder is configured to convert an intra BC prediction region for a current block (eg, coding unit, prediction unit) in a picture to a reference region in the picture. Determine based on. The intra BC prediction area is inverted with respect to the reference area. For example, the intra BC prediction region is flipped horizontally with respect to the reference region, flipped vertically with respect to the reference region, or flipped both horizontally and vertically with respect to the reference region. Has been.

  The encoder encodes the current block using the intra BC prediction region, and outputs the encoded data in the bitstream. The encoded data includes an indication as to whether the intra BC prediction region is inverted with respect to the reference region. For example, an indication is one or more syntax elements in a bitstream that can be signaled for the current block or a larger block that includes the current block. The one or more syntax elements can be one or more flags, each flag indicating a decision about the direction of inversion. One or more syntax elements may be jointly coded with another syntax element or included in the bitstream and signaled separately.

  A corresponding decoder receives the encoded data in the bitstream. The encoded data includes an indication as to whether the intra BC prediction region for the current block (eg, encoding unit, prediction unit) in the picture is inverted with respect to the reference region in the picture. For example, an indication is one or more syntax elements in a bitstream that can be signaled for the current block or a larger block that includes the current block. The one or more syntax elements can be one or more flags, each flag indicating a decision about the direction of inversion. One or more syntax elements may be jointly encoded with another syntax element or included in the bitstream and signaled separately.

  The decoder determines an intra BC prediction region for the current block based on a reference region in the picture. The intra BC prediction region is inverted with respect to the reference region (for example, in the horizontal direction and / or the vertical direction). The decoder decodes the current block using the intra BC prediction region.

  When the encoder or decoder determines an intra BC prediction region that is inverted with respect to the reference region, the encoder or decoder determines (a) the reference region, (b) inverts the reference region, and then (c ) The sample values at the positions of the inverted reference regions can be assigned to the sample values at those locations of the intra BC prediction region. Alternatively, the encoder or decoder (a) determines reference areas, (b) assigns sample values at the positions of the reference areas to sample values at those positions of the intra BC prediction area, and then (c) intra BC prediction. The region can be reversed. Alternatively, the encoder or decoder can (a) determine the reference region and then (b) assign the sample value at the position of the reference region to the sample value at the corresponding position of the intra BC prediction region, where The corresponding position takes into account inversion.

  In some exemplary implementations, the encoded data includes the BV value of the current block. The BV value indicates the displacement to the reference area in the picture. During encoding, the BV value may be a predicted BV value, or the BV value may be identified in the BV estimation and signaled with a BV difference relative to the predicted BV value. During decoding, the BV value may be a predicted BV value, or the BV value may be reconstructed by adding the BV difference to the predicted BV value.

  According to another aspect of the innovation described herein, an image encoder or video encoder determines a BV value of a current block (eg, encoding unit, prediction unit) in a picture. The BV value indicates the displacement to the reference area in the picture. The encoder uses the BV value of the current block and the BV predictor (predicted BV value) to determine the BV difference of the current block. The bitstream may include an index value indicating selection of a BV predictor candidate from a set of multiple BV predictor candidates for use as a BV predictor. Alternatively, the BV predictor may be selected in some other way. The encoder encodes the current block using intra BC prediction using the BV value. The encoder outputs encoded data including a flag indicating that the current block is encoded in the skip mode using intra BC prediction in the bitstream. Since the current block is encoded in skip mode using intra BC prediction, the bitstream contains the BV difference of the current block but not the residual data of the current block.

  In some exemplary implementations, if a given block (eg, current block, subsequent block) is not encoded in skip mode using intra BC prediction, another flag may be It may indicate whether it is encoded in non-skip mode using intra BC prediction. If a given block is not encoded in non-skip mode using intra BC prediction, the given block may be an intra spatial prediction mode or interpicture, as indicated by one or more other syntax elements. It may be encoded in another mode, such as a mode.

  In some exemplary implementations, a given block (eg, current block, subsequent block) that is intra BC predicted in skip mode has a value defined for the partitioning mode (partitioning mode). . This affects the signaling of syntax elements for split mode. If a given block is encoded in non-skip mode using intra BC prediction, the bitstream includes a syntax element that indicates the split mode of the given block. However, if a given block is encoded in skip mode using intra BC prediction, the bitstream does not include a syntax element indicating the partition mode of the given block, and the partition of the given block The mode has a defined value.

  In some exemplary implementations, a given block (eg, current block, subsequent block) that is intra BC predicted in skip mode does not have a flag that indicates the presence or absence of residual data for the given block. . The residual data for a given block is considered not in the bitstream. Also, if a given block is encoded in non-skip mode using intra BC prediction and the partition mode of the given block has a defined value, the bitstream is Does not include a flag indicating the presence or absence of residual data. In this case, the residual data for a given block is considered to be in the bitstream. Also, if a given block is encoded in non-skip mode using intra BC prediction and the partition mode of the given block does not have a defined value, the bitstream is Contains a flag indicating the presence or absence of residual data for the block.

  The corresponding decoder is encoded including a flag from the bitstream indicating that the current block in the picture (eg, encoding unit, prediction unit) is encoded in skip mode using intra BC prediction. Receive data. Since the current block is encoded in skip mode using intra BC prediction, the bitstream contains the BV difference of the current block but not the residual data of the current block. The decoder uses the BV difference of the current block and the BV predictor (predicted BV value) to determine the BV value of the current block. The bitstream may include an index value indicating selection of a BV predictor candidate from a set of multiple BV predictor candidates for use as a BV predictor. Alternatively, the BV predictor may be selected in some other way. The BV value indicates the displacement to the reference area in the picture. The decoder decodes the current block using intra BC prediction using the BV value.

  If the current block is encoded in skip mode using intra BC prediction, the intra BC prediction region for the current block may be inverted with respect to its reference region. Examples of inversion operations, direction of inversion, and signaling whether inversion is used are summarized above.

  Innovations for intra BC prediction include having stored a computer-executable instruction as part of a method, as part of a computing device adapted to perform the method, or causing a computing device to perform the method. It may be implemented as part of a body computer readable medium. Various innovations can be used in combination or separately. In particular, block inversion in intra BC prediction may be used in combination with the skip mode for intra BC prediction blocks.

  The foregoing and other objects, features and advantages of the present invention will become more apparent from the following detailed description, which proceeds with reference to the accompanying figures.

1 is an illustration of an example computing system in which some described embodiments can be implemented. FIG. 4 is an illustration of an example network environment in which some described embodiments can be implemented. FIG. 4 is an illustration of an example network environment in which some described embodiments can be implemented. 1 is an illustration of an example encoder system in which some described embodiments can be implemented in concert. FIG. 4 is an illustration of an example decoder system in which some described embodiments can be implemented in concert. FIG. 3 illustrates an example video encoder that can cooperatively implement some described embodiments. FIG. 3 illustrates an example video encoder that can cooperatively implement some described embodiments. FIG. 3 illustrates an example video decoder that can cooperatively implement some described embodiments. The figure which shows intra BC prediction about the block in a picture. FIG. 6 is a diagram illustrating BV prediction for blocks. The figure which shows BV prediction about the block in a picture. The figure which shows inversion of the reference area about a block. The figure which shows inversion of the reference area about a block. The figure which shows inversion of the reference area about a block. The figure which shows inversion of the reference area about a block. The figure which shows inversion of the reference area about a block. The figure which shows inversion of the reference area about a block. The figure which shows inversion of the reference area about a block. The figure which shows inversion of the reference area about a block. The figure which shows inversion of the reference area about a block. The figure which shows inversion of the reference area about a block. The flowchart which shows the technique for the encoding by which an intra BC prediction area | region is inverted with respect to a reference area. 6 is a flowchart illustrating a technique for decoding in which an intra BC prediction region is inverted with respect to a reference region. The flowchart which shows the technique for the encoding containing the skip mode about an intra BC prediction block. The flowchart which shows the technique for the encoding containing the skip mode about an intra BC prediction block. 7 is a flowchart illustrating a technique for decoding including a skip mode for an intra BC prediction block. 7 is a flowchart illustrating a technique for decoding including a skip mode for an intra BC prediction block. A table showing the syntax structure for a coding unit, according to a conventional approach. A table showing the syntax structure for a coding unit, according to a conventional approach. A table showing a new syntax structure for a coding unit that may be coded as an intra BC prediction block in skip mode.

  The detailed description presents innovations in the area of block coding or decoding using intra block copy (“BC”) prediction. For example, some of the innovations relate to block inversion, where an intra BC prediction block is inverted with respect to a reference region that may be indicated by a block vector (“BV”) value. Another innovation relates to skip mode signaling where the current intra BC prediction block uses the signaled BV difference but has no residual data. In many situations, this innovation improves the coding efficiency of intra BC prediction blocks.

  Although the operations described herein are described in some places as being performed by a video encoder or video decoder, in many cases such operations are performed by other types of media processing tools (eg, image encoders or Image decoder).

  Some of the innovations described herein are It is shown with reference to syntax elements and operations specific to the H.265 / HEVC standard. For example, H.M. Reference is made to the draft version JCTVC-P1005 ("High Efficiency Video Coding (HEVC) Range Extensions Text Specification: Draft 6", JCTVC-P1005_v1, February 2014) of the H.265 / HEVC standard. The innovations described herein can also be implemented for other standards or formats.

  Some of the innovations described herein (eg, block inversion) are described with reference to intra BC prediction. The innovation can also be applied in other contexts (eg, block inversion for reference regions in motion compensation).

  More generally, various alternatives to the examples described herein are possible. For example, some of the methods described herein may include, for example, changing the order of method operations described, dividing predetermined method operations, repeating predetermined method operations, or predetermined It can be changed by omitting the method operation. Various aspects of the disclosed technology may be used in combination or separately. Various embodiments use one or more of the described innovations. Some of the innovations described herein address one or more of the problems noted in the background art. In general, a given technology / tool does not solve all such problems.

I. Exemplary Computing System FIG. 1 shows a generalized example of a suitable computing system (100) that can implement some of the innovations described. The computing system (100) is not intended to suggest limitations as to the scope of use or functionality. This is because the innovation can be implemented in a variety of general purpose or dedicated computing systems.

  Referring to FIG. 1, the computing system (100) includes one or more processing units (110, 115) and memory (120, 125). The processing devices (110, 115) execute computer-executable instructions. The processing unit may be a general purpose central processing unit (“CPU”), a processor in an application specific integrated circuit (“ASIC”), or any other type of processor. In a multi-processing system, multiple processing devices execute computer-executable instructions to increase processing power. For example, FIG. 1 shows a graphics processing unit or co-processing unit (115) in addition to the central processing unit (110). The tangible memory (120, 125) may be volatile memory (eg, register, cache, RAM) accessible by one or more processing devices, or non-volatile memory (eg, ROM, EEPROM, A flash memory or the like, or some combination of these two. The memory (120, 125) is software that implements one or more innovations for block inversion and / or skip mode in intra BC prediction in the form of computer-executable instructions suitable for execution by one or more processing units. 180) is stored.

  The computing system can have additional features. For example, the computing system (100) includes a storage (140), one or more input devices (150), one or more output devices (160), and one or more communication connections (170). An interconnection mechanism (not shown) such as a bus, controller, or network interconnects the components of the computing system (100). Typically, operating system software (not shown) provides an operating environment for other software executing on the computing system (100) and coordinates the operation of the components of the computing system (100).

  The tangible storage (140) may be removable or non-removable and is used for storing magnetic disks, magnetic tapes, magnetic cassettes, CD-ROMs, DVDs or information. And any other medium that can be accessed within the computing system (100). The storage (140) stores software (180) instructions that implement one or more innovations for block inversion and / or skip mode in intra BC prediction.

  The one or more input devices (150) may be touch input devices such as a keyboard, mouse, pen, or trackball, voice input devices, scanning devices, or another device that provides input to the computing system (100). be able to. With respect to video, one or more input devices (150) may be a camera, video card, TV tuner card, screen capture module, or similar device that accepts video input in analog or digital form, or video input to a computing system (100 CD-ROM or CD-RW. The one or more output devices (160) may be a display, printer, speaker, CD writer, or another device that provides output from the computing system (100).

  One or more communication connections (170) allow communication with another computing entity via a communication medium. The communication medium conveys information such as computer-executable instructions, audio input, video input, audio output, video output, or other data in a modulated data signal. A modulated data signal is a signal that has one or more of the characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media can use electrical, optical, RF, or other carriers.

  The innovation can be described in the general context of computer-readable media. Computer readable media can be any available tangible media that can be accessed within a computing environment. By way of example, and not limitation, in computing system (100), computer-readable media include memory (120, 125), storage (140), and any combination thereof.

  The innovation can be described in the general context of computer-executable instructions. Computer-executable instructions are, for example, included in program modules and executed on a target real processor or virtual processor in a computing system. Generally, program modules include routines, programs, libraries, objects, classes, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The functions of the program modules may be combined or divided among the program modules as necessary in various embodiments. Computer-executable instructions for program modules may be executed within a local computing system or a distributed computing system.

  The terms “system” and “device” are used interchangeably herein. Unless the context clearly indicates otherwise, these terms do not imply any limitation with respect to the type of computing system or computing device. Generally, a computing system or computing device may be localized or distributed and includes dedicated and / or general purpose hardware and software that implements the functionality described herein. Any combination may be included.

  The disclosed methods can also be implemented using dedicated computing hardware configured to perform any of the disclosed methods. For example, the disclosed method may be an integrated circuit (eg, an ASIC (such as an ASIC digital signal processor (“DSP”)), graphics processing device (“GPU”) that is specifically designed or configured to perform any of the disclosed methods. Or a programmable logic device ("PLD") such as a field programmable gate array ("FPGA")).

  For purposes of presentation, the detailed description uses terms such as “determine” and “use” to describe computer operations in the computing system. These terms are high-level abstract representations of operations performed by a computer and should not be confused with operations performed by a human. The actual computer operations corresponding to these terms will vary depending on the implementation.

II. Exemplary Network Environment FIGS. 2a and 2b illustrate an exemplary network environment (201, 202) that includes a video encoder (220) and a video decoder (270). The encoder (220) and decoder (270) are connected via the network (250) using an appropriate communication protocol. The network (250) may include the Internet or another computer network.

  In the network environment (201) shown in FIG. 2a, each real-time communication (“RTC”) tool (210) includes both an encoder (220) and a decoder (270) for bidirectional communication. A given encoder (220) Producing output conforming to the H.265 / HEVC standard variant or extension, SMPTE 421M standard, ISO / IEC 14496-10 standard (also known as H.264 or AVC), another standard, or proprietary format The corresponding decoder (270) can accept the encoded data from the encoder (220). Two-way communication may be part of a video conference, video call, or other two party or multi party communication scenario. Although the network environment (201) of FIG. 2a includes two real-time communication tools (210), the network environment (201) may include three or more real-time communication tools (210) that participate in multi-party communication.

  The real-time communication tool (210) manages encoding by the encoder (220). FIG. 3 shows an exemplary encoder system (300) that may be included in the real-time communication tool (210). Alternatively, the real-time communication tool (210) may use another encoder system. The real-time communication tool (210) also manages the decoding by the decoder (270). FIG. 4 shows an exemplary decoder system (400) that may be included in the real-time communication tool (210). Alternatively, the real-time communication tool (210) may use another decoder system.

  In the network environment (202) shown in FIG. 2b, the encoding tool (212) includes an encoder (220) that encodes video for delivery to a plurality of playback tools (214) including a decoder (270). Unidirectional communication is for video surveillance systems, webcam monitoring systems, screen capture modules, remote desktop conference presentations, or other scenarios where video is encoded and transmitted from one location to one or more other locations. Can be provided. Although the network environment (202) of FIG. 2b includes two playback tools (214), the network environment (202) may include more or fewer playback tools (214). In general, playback tool (214) communicates with encoding tool (212) to determine a stream of video received by playback tool (214). The playback tool (214) receives the stream, buffers the received encoded data for an appropriate period, and starts decoding and playback.

  FIG. 3 shows an exemplary encoder system (300) that may be included in the encoding tool (212). Alternatively, the encoding tool (212) may use another encoder system. The encoding tool (212) may also include server-side controller logic for managing connections with one or more playback tools (214). FIG. 4 shows an exemplary decoder system (400) that may be included in the playback tool (214). Alternatively, the playback tool (214) may use another decoder system. The playback tool (214) may also include client-side controller logic for managing connections with the encoding tool (212).

III. Exemplary Encoder System FIG. 3 is a block diagram of an exemplary encoder system (300) in which some described embodiments can be implemented in concert. The encoder system (300) includes a plurality of codes such as a low delay encoding mode for real-time communication, a transcoding mode, and a higher delay encoding mode for generating media for playback from a file or stream. It may be a general purpose coding tool that can operate in any of the coding modes, or it may be a dedicated coding tool adapted for one such coding mode. The encoder system (300) may be adapted to encode a particular type of content (eg, screen capture content). The encoder system (300) can be implemented as an operating system module, as part of an application library, or as a stand-alone application. In general, the encoder system (300) receives a series of source video frames (311) from a video source (310) and generates encoded data as output to a channel (390). The encoded data output to the channel may include content encoded using block inversion and / or skip mode in intra BC prediction, as described herein.

  The video source (310) can be a camera, tuner card, storage medium, screen capture module, or other digital video source. The video source (310) generates a series of video frames at a frame rate of, for example, 30 frames per second. As used herein, the term “frame” generally refers to source encoded or reconstructed image data. With respect to progressive scan video, the frame is a progressive scan video frame. With respect to interlaced video, in an exemplary embodiment, interlaced video frames may be de-interlaced prior to encoding. Alternatively, two complementary interlaced video fields may be encoded together as one video frame or may be encoded as two separately encoded fields. Whether referring to progressive scan video frames or interlaced scan video frames, the term “frame” or “picture” refers to a single unpaired video field, a complementary pair of video fields, video at a given time. A video object plane representing the object, or a region of interest in a larger image may be shown. A video object plane or region may be part of a larger image that includes multiple objects or regions of the scene.

  The arriving source frame (311) is stored in a source frame temporary memory storage area (320) including a plurality of frame buffer storage areas (321, 322, ..., 32n). The frame buffer (321, 322, etc.) holds one source frame in the source frame storage area (320). After one or more of the source frames (311) are stored in the frame buffer (321, 322, etc.), the frame selector (330) selects individual source frames from the source frame storage area (320). The order in which frames are selected by the frame selector (330) for input to the encoder (340) may be different from the order in which frames are generated by the video source (310). For example, the encoding of several frames may be delayed in order to allow several subsequent frames to be encoded first, thereby facilitating temporal backward prediction. Prior to the encoder (340), the encoder system (300) may include a preprocessor (not shown) that performs preprocessing (eg, filtering) of the selected frame (331) prior to encoding. Preprocessing includes color space conversion to primary (eg, luma) and secondary (eg, chroma difference for red and blue) components for encoding and (eg, to reduce the spatial resolution of the chroma component). A resampling process may be included. Usually, before encoding, the video is converted to a color space such as YUV. In YUV, the sample value of the luma (Y) component represents a luminance value or brightness value, and the sample value of the chroma (U, V) component represents a color difference value. The exact definition of the color difference values (and the conversion operation from the YUV color space to another color space such as RGB and the conversion operation from another color space such as RGB to the YUV color space) depends on the implementation. In general, as used herein, the term YUV refers to any color space having one luma (or luminance) component and one or more chroma (or chrominance) components, and Y′UV, YIQ, In addition to Y′IQ and YDbDr, modifications such as YCbCr and YCoCg are included. Chroma sample values can be subsampled to a lower chroma sampling rate (eg, to match YUV4: 2: 0 format), or chroma sample values can be subsampled (eg, to match YUV4: 4: 4 format). ) Can have the same resolution as the luma sample value. Alternatively, the video can be encoded in another format (eg, RGB 4: 4: 4 format).

  The encoder (340) encodes the selected frame (331) to generate an encoded frame (341), and a memory management control operation (“MMCO”) signal (342) or a reference picture set (“RPS”). ) Generate information. The RPS is a set of frames that can be used for reference in motion compensation for the current frame or any subsequent frame. If the current frame is not the first frame encoded, when performing the encoding process, the encoder (340) may encode one or more previously encoded data stored in the decoded frame temporary memory storage area (360). / Decoded frame (369) can be used. Such stored decoded frame (369) is used as a reference frame for interframe prediction (interframe prediction) of the content of the current source frame (331). The MMCO / RPS information (342) indicates to the decoder which reconstructed frame can be used as a reference frame and therefore should be stored in the frame storage area.

  In general, the encoder (340) includes multiple encoding modules that perform encoding tasks such as division into tiles, intra prediction estimation and prediction, motion estimation and motion compensation, frequency conversion, quantization, and entropy encoding. . The exact operation performed by the encoder (340) may vary depending on the compression format. The format of the encoded data to be output is H.264. H.265 / HEVC format modification or expansion, Windows (registered trademark) Media Video format, VC-1 format, MPEG-x format (eg, MPEG-1, MPEG-2, or MPEG-4), H.264. 26x format (eg, H.261, H.262, H.263, H.264), or another format.

  The encoder (340) may divide the frame into multiple tiles of the same size or different sizes. For example, the encoder (340) divides the frame along tile rows and tile columns that define the horizontal and vertical boundaries of the tiles in the frame using the frame boundaries. Here, each tile is a rectangular area. Tiles are often used to provide options for parallel processing. A frame may also be organized as one or more slices. Here, the slice may be the entire frame or a region of the frame. Slices can be decoded independently of other slices in the frame, which improves error resilience. The content of the slice or tile is further divided into other sets of blocks or samples for encoding and decoding.

  H. For syntax according to the H.265 / HEVC standard, the encoder divides the content of a frame (or slice or tile) into coding tree units. A coding tree unit (“CTU”) includes luma sample values organized as one luma coding tree block (“CTB”) and corresponding chroma sample values organized as two chroma CTBs. The size of the CTU (and its CTB) is selected by the encoder and can be, for example, 64 × 64 sample values, 32 × 32 sample values, or 16 × 16 sample values. A CTU includes one or more encoding units. The coding unit (“CU”) has one luma coding block (“CB”) and two corresponding chroma CBs. For example, a CTU (YUV 4: 4: 4 format) with one 64 × 64 luma CTB and two 64 × 64 chroma CTBs may be divided into four CUs. Here, each CU includes one 32 × 32 luma CB and two 32 × 32 chroma CBs, and each CU is further divided into smaller CUs if possible. Alternatively, as another example, a CTU (YUV 4: 2: 0 format) with one 64 × 64 luma CTB and two 32 × 32 chroma CTBs may be divided into four CUs. Here, each CU includes one 32 × 32 luma CB and two 16 × 16 chroma CBs, and each CU is further divided into smaller CUs if possible. The minimum allowable size of a CU (eg, 8 × 8, 16 × 16) may be included in the bitstream and signaled.

  In general, a CU has a prediction mode such as inter or intra. A CU includes one or more prediction units for signaling and / or prediction processing of prediction information (eg, prediction mode details, displacement values, etc.). The prediction unit (“PU”) has one luma prediction block (“PB”) and two chroma PBs. For intra-predicted CUs, the PU has the same size as the CU unless the CU has a minimum size (eg, 8 × 8). In this case, the CU may be divided into 4 smaller PUs (eg, 4 × 4 each if the minimum CU size is 8 × 8), or the PU is indicated by a syntax element for the CU. Can have a minimum CU size. The CU also has one or more transform units for residual encoding / decoding. Here, the transform unit (“TU”) has one luma transform block (“TB”) and two chroma TBs. A PU in an intra-predicted CU may include one TU (equal in size to the PU) or multiple TUs. The encoder determines how to split the video into CTUs, CUs, PUs, TUs, etc.

  H. In a H.265 / HEVC implementation, a slice may include one slice segment (independent slice segment) or may be divided into multiple slice segments (one independent slice segment and one or more dependent slice segments). A slice segment is an integer number of CTUs that are sequentially ordered in a tile scan, contained in one network abstraction layer (“NAL”) unit. For independent slice segments, the slice segment header includes the value of the syntax element that is applied for the independent slice segment. For a dependent slice segment, the truncated slice segment header contains a few values of the syntax elements applied for that dependent slice segment, and the values of the other syntax elements for that dependent slice segment are in decoding order. Inferred from the value for the preceding independent slice segment.

  As used herein, the term “block” refers to a macroblock, a prediction unit, a residual data unit, ie, CB, PB, or TB, or some other set of sample values, depending on the context. Can be shown.

  Returning to FIG. 3, the encoder represents the intra-coded block of the source frame (331) in terms of prediction from other previously reconstructed sample values in the source frame (331). For intra BC prediction, the intra picture estimator estimates the displacement of the block relative to other previously reconstructed sample values (or in some implementations relative to the original sample values in frame (331)). . An intra-frame prediction reference region is a region of samples in a frame that is used to generate a BC prediction value for a block. The reference region may be indicated by a BV value (determined in the block vector (“BV”) estimation). The reference region may be one that is inverted with respect to the prediction region for the block, as described herein. For intra spatial prediction for a block, an intra picture estimator estimates the extrapolation of adjacent reconstructed sample values to the block. The intra picture estimation unit can output prediction information (a BV value for intra BC prediction, a prediction mode (direction) for intra spatial prediction, etc.), and the prediction information is entropy encoded. The intra frame prediction prediction unit applies prediction information to determine an intra prediction value.

  The encoder (340) represents the inter-frame encoded prediction block of the source frame (331) from the viewpoint of prediction from the reference frame. The motion estimation unit estimates the motion of the block with respect to one or more reference frames (369). When multiple reference frames are used, the multiple reference frames may be from different temporal directions or from the same temporal direction. A motion compensated prediction reference region is a region of sample values in one or more reference frames that is used to generate a motion compensated prediction value for the sample values of a block in the current frame. The reference region may be one that is inverted with respect to the prediction region for the block, as described herein. The motion estimator outputs motion information such as motion vector (“MV”) information, and the motion information is entropy encoded. The motion compensation unit applies MV to the reference frame (369) in order to determine a motion compensation prediction value for inter-frame prediction.

  The encoder can determine the difference (if any) between the predicted value of the block (intra or inter) and the corresponding original value. Such prediction residual values are further encoded using frequency transform, quantization, and entropy encoding. For example, the encoder (340) sets the value of a quantization parameter (“QP”) for a picture, tile, slice, and / or other part of the video and quantizes the transform coefficients accordingly. The entropy encoder of the encoder (340) adds predetermined side information (for example, MV information, index value of BV predictor, BV difference, QP value, mode determination, parameter selection) in addition to the quantized transform coefficient value ) Is also compressed. General entropy coding techniques include exponential Golomb coding, Golomu-Rice coding, arithmetic coding, differential coding, Huffman coding, run-length coding, and “V2V (variable-length-to-variable-length)” codes. , “V2F (variable-length-to-fixed-length)” coding, “LZ (Lempel-Ziv)” coding, dictionary coding, “PIPE (probability interval partitioning entropy)” coding, and combinations thereof including. The entropy encoding unit can use various encoding techniques for various types of information, and can apply a plurality of techniques in combination (for example, arithmetic encoding after applying Goromrice encoding) Can be selected from among a plurality of code tables in a particular coding technique.

  An adaptive deblocking filter is included in the motion compensation loop in encoder (340) to smooth discontinuities across block boundary rows and / or block boundary columns in the decoded frame. Alternatively or additionally, other filtering (such as deringing filtering, adaptive loop filtering (“ALF”), or sample adaptive offset (“SAO”) filtering; not shown) is applied as an in-loop filtering operation. May be.

  The encoded data generated by the encoder (340) includes syntax elements for various layers of the bitstream syntax. H. With respect to syntax according to the H.265 / HEVC standard, for example, a picture parameter set (“PPS”) is a syntax structure that includes syntax elements that can be associated with a picture. PPS may be used for one picture and PPS may be reused for multiple pictures in a sequence. The PPS is usually signaled separately from the encoded data for the picture (eg, one NAL unit is for PPS and one or more other NAL units are for encoded data for the picture). . In the encoded data for a picture, the syntax element indicates which PPS to use for the picture. Similarly, H.M. With respect to syntax according to the H.265 / HEVC standard, a sequence parameter set (“SPS”) is a syntax structure that includes syntax elements that can be associated with a sequence of pictures. The bitstream may include one SPS or multiple SPSs. The SPS is typically signaled separately from other data about the sequence, and the syntax element in the other data indicates which SPS to use.

  Encoded frame (341) and MMCO / RPS information (342) (or information equivalent to MMCO / RPS information (342) since dependency and order structure for frames are already known in encoder (340)) Are processed by the decryption process emulator (350). The decoding process emulator (350) implements a part of decoder functions such as a decoding task for reconstructing a reference frame, for example. The decoding process emulator (350) uses a given encoded frame (341) as a reference frame in interframe prediction of subsequent frames to be encoded, consistent with MMCO / RPS information (342). It is determined whether it needs to be reconstructed and stored. If the encoded frame (341) needs to be stored, the decoding process emulator (350) will be performed by a decoder that receives the encoded frame (341) and generates a corresponding decoded frame (351). Simulate the decryption process. In doing so, when the encoder (340) uses one or more decoded frames (369) stored in the decoded frame storage area (360), the decoding process emulator (350) may be part of the decoding process. , One or more decoded frames (369) are used from the storage area (360).

  The decoded frame temporary memory storage area (360) includes a plurality of frame buffer storage areas (361, 362, ..., 36n). The decoding process emulator (350) can match any frame buffer (361, 362) with a frame that is no longer needed by the encoder (340) for use as a reference frame to be consistent with the MMCO / RPS information (342). Etc.), the contents of the storage area (360) are managed. After simulating the decoding process, the decoding process emulator (350) stores the newly decoded frame (351) in the frame buffer (361, 362, etc.) thus identified.

  The encoded frame (341) and the MMCO / RPS information (342) are buffered in the temporary encoded data area (370). The encoded data collected in the encoded data area (370) includes encoded data of one or more pictures as part of the syntax of the elementary encoded video bitstream. The encoded data collected in the encoded data field (370) is also (eg, as one or more parameters in one or more supplemental enhancement information (“SEI”) messages or video usability information (“VUI”) messages). Media metadata associated with the encoded video data may be included.

  The collected data (371) from the temporary encoded data area (370) is processed by the channel encoder (380). The channel encoder (380) is in accordance with a media program stream or transport stream format such as ITU-T H.222.0 | ISO / IEC 13818-1, or an Internet real-time transport protocol format such as IETF RFC 3550. The collected data can be packetized and / or multiplexed for transmission or storage as a media stream. In such a case, the channel encoder (380) may add syntax elements as part of the syntax of the media transport stream. Alternatively, the channel encoder (380) can organize the collected data for storage as a file (eg, according to a media container format such as ISO / IEC 14496-12). In such a case, the channel encoder (380) may add syntax elements as part of the media storage file syntax. Alternatively, more generally, the channel encoder (380) may implement one or more media system multiplexing protocols or transport protocols. In such a case, the channel encoder (380) may add syntax elements as part of the syntax of one or more protocols. Channel encoder (380) provides an output to channel (390). Channel (390) represents another channel for storage, communication connection, or output. Channel encoder (380) or channel (390) may also include other elements (not shown), eg, for forward error correction (“FEC”) coding and analog signal modulation.

IV. Exemplary Decoder System FIG. 4 is a block diagram of an exemplary decoder system (400) in which some described embodiments may be implemented in concert. The decoder system (400) can operate in any of a plurality of decoding modes, such as a low delay decoding mode for real-time communication and a higher delay decoding mode for media playback from a file or stream. It may be a general purpose decoding tool or a dedicated decoding tool adapted for one such decoding mode. The decoder system (400) may be adapted to decode certain types of content (eg, screen capture content). The decoder system (400) can be implemented as an operating system module, as part of an application library, or as a stand-alone application. In general, the decoder system (400) receives encoded data from the channel (410) and generates a reconstructed frame as output to an output destination (490). The encoded data may include content encoded using block inversion and / or skip mode in intra BC prediction, as described herein.

  The decoder system (400) includes a channel (410) that may represent another channel for storage, communication connection, or encoded data as input. The channel (410) generates encoded data that is channel-encoded. The channel decoder (420) can process the encoded data. For example, the channel decoder (420) may be a media program stream or transport stream format such as ITU-T H.222.0 | ISO / IEC 13818-1, or an Internet real-time transport protocol such as IETF RFC 3550. Depacketize and / or demultiplex data collected for transmission or storage as a media stream (according to the format). In such a case, the channel decoder (420) can analyze the syntax elements added as part of the syntax of the media transport stream. Alternatively, the channel decoder (420) separates the encoded video data collected for storage as a file (eg, according to a media container format such as ISO / IEC 14496-12). In such a case, the channel decoder (420) can parse the syntax elements added as part of the media storage file syntax. Alternatively, more generally, the channel decoder (420) may implement one or more media system demultiplexing protocols or transport protocols. In such a case, the channel decoder (420) can parse the syntax elements added as part of the syntax of one or more protocols. The channel (410) or channel decoder (420) may also include other elements (not shown), eg, for FEC decoding and analog signal demodulation.

  The encoded data (421) output from the channel decoder (420) is stored in the temporary encoded data area (430) until a sufficient amount of data is received. The encoded data (421) includes an encoded frame (431) and MMCO / RPS information (432). The encoded data (421) in the encoded data area (430) includes encoded data of one or more pictures as part of the syntax of the elementary encoded video bitstream. The encoded data (421) in the encoded data area (430) may also include media metadata associated with the encoded video data (eg, as one or more parameters in one or more SEI messages or VUI messages). .

  In general, the encoded data area (430) temporarily stores such encoded data (421) until the encoded data (421) is used by the decoder (450). At that time, the encoded data of the encoded frame (431) and the MMCO / RPS information (432) is transmitted from the encoded data area (430) to the decoder (450). As decoding progresses, new encoded data is added to the encoded data area (430), and the oldest encoded data remaining in the encoded data area (430) is transmitted to the decoder (450).

  The decoder (450) decodes the encoded frame (431) to generate a corresponding decoded frame (451). If necessary, when performing the decoding process, the decoder (450) may use one or more previously decoded frames (469) as reference frames for inter-frame prediction. The decoder (450) reads such previously decoded frame (469) from the decoded frame temporary memory storage area (460). In general, the decoder (450) includes a plurality of decoding modules that perform decoding tasks such as entropy decoding, intra-frame prediction, motion compensated inter-frame prediction, inverse quantization, inverse frequency transform, and tile merging. The exact operation performed by the decoder (450) may vary depending on the compression format.

  For example, the decoder (450) receives the encoded data of the compressed frame or series of frames and generates an output that includes the decoded frame (451). In the decoder (450), the buffer accepts the encoded data of the compressed frame and makes the received encoded data available to the entropy decoding unit at an appropriate time. The entropy decoding unit applies entropy decoding to the entropy-encoded side information in addition to the entropy-encoded quantized data by applying the inverse of the entropy encoding normally performed in the encoder. The motion compensation unit applies motion information to one or more reference frames in order to form a motion compensated prediction value of the inter-coded block of the reconstructed frame. The inter-frame reference region may be one that is inverted with respect to the prediction region for the block, as described herein. The intra frame prediction module can spatially predict the sample value of the current block from neighboring previously reconstructed sample values, or for intra BC prediction, before the intra frame prediction region in the frame. The reconstructed sample values can be used to predict sample values for the current block. The intra frame reference area may be indicated by a BV value. This reference region may be inverted with respect to the prediction region for the block, as described herein. The decoder (450) also reconstructs the prediction residual value. The inverse quantization unit performs inverse quantization on the entropy-decoded data. For example, the decoder (450) sets QP values for pictures, tiles, slices, and / or other parts of video based on syntax elements in the bitstream and dequantizes the transform coefficients accordingly. To do. The inverse frequency transform unit transforms the quantized frequency domain data into spatial domain data. For inter-frame prediction blocks, the decoder (450) combines the reconstructed prediction residual value with the motion compensated prediction value. The decoder (450) can similarly combine the prediction residual value with the prediction value from the intra prediction. An adaptive deblocking filter is included in the motion compensation loop in the video decoder (450) to smooth discontinuities across the block boundary rows and / or block boundary columns in the decoded frame (451). Alternatively or additionally, other filtering (such as deringing filtering, ALF or SAO filtering; not shown) may be applied as an in-loop filtering operation.

  The decoded frame temporary memory storage area (460) includes a plurality of frame buffer storage areas (461, 462, ..., 46n). The decoded frame storage area (460) is an example of a decoded picture buffer. The decoder (450) uses the MMCO / RPS information (432) to identify a frame buffer (461, 462, etc.) that can store the decoded frame (451). The decoder (450) stores the decoded frame (451) in the frame buffer.

  The output sequencer (480) identifies when the next frame generated in the output order becomes available in the decoded frame storage area (460). When the next frame (481) generated in the output order becomes available in the decoded frame storage area (460), the frame is read by the output sequencer (480), and the output destination (490) ( For example, it is output to a display. In general, the order in which frames are output from the decoded frame storage area (460) by the output sequencer (480) may be different from the order in which frames are decoded by the decoder (450).

V. Exemplary Video Encoder FIGS. 5a and 5b are block diagrams of a generalized video encoder (500) in which some described embodiments may be implemented in concert. The encoder (500) receives a series of video pictures including a current picture as an input video signal (505), and generates encoded data in an encoded video bitstream (595) as an output.

  The encoder (500) is block based and uses an implementation dependent block format. The blocks may be further subdivided in various stages, such as in a prediction stage, a frequency conversion stage, and / or an entropy encoding stage. For example, a picture may be divided into 64 × 64 blocks, 32 × 32 blocks, or 16 × 16 blocks, which are then divided into smaller sampled blocks for encoding and decoding. obtain. H. In the coding implementation for the H.265 / HEVC standard, the encoder divides the picture into CTU (CTB), CU (CB), PU (PB), and TU (TB).

  The encoder (500) compresses a picture using intra picture coding and / or inter picture coding. Many of the components of the encoder (500) are used for both intra-picture coding and inter-picture coding. The exact operations performed by these components can vary depending on the type of information being compressed.

  The tiling module (510) optionally divides the picture into multiple tiles of the same size or different sizes. For example, the tiling module (510) uses the picture boundaries to divide the picture along tile rows and tile columns that define the horizontal and vertical boundaries of the tiles in the picture. Here, each tile is a rectangular area. H. In the H.265 / HEVC implementation, the encoder (500) divides a picture into one or more slices. Here, each slice includes one or more slice segments.

  The general coding controller (520) receives feedback (not shown) from various modules of the encoder (500) in addition to the pictures of the input video signal (505). In general, the general encoding control unit (520) may use other modules (tiling module (510), transform unit / scaling unit / quantization unit (530)) to set and change encoding parameters during encoding. A control signal (not shown) is supplied to a scaling unit / inverse transform unit (535), an intra picture estimation unit (540), a motion estimation unit (550), an intra / inter switch, and the like. Specifically, the general coding control unit (520) determines whether to use an intra BC prediction mode (for example, skip mode, block inversion) and an intra BC prediction mode (for example, skip mode, block) during encoding. It can be determined how to use (invert). The general coding controller (520) can also perform rate distortion analysis, for example, to evaluate intermediate results during coding. The general encoding control unit (520) generates general control data (522) indicating the decisions made during encoding, so that the corresponding decoder can make matching decisions. The general control data (522) is provided to the header formatting unit / entropy encoding unit (590).

  When the current picture is predicted using inter picture prediction, the motion estimation unit (550) estimates the motion of a block of sample values in the current picture of the input video signal (505) with respect to one or more reference pictures. . The motion estimator (550) can evaluate an option for inverting a given reference region for an inter-picture coded block, as described below. The decoded picture buffer (570) buffers one or more previously reconstructed encoded pictures for use as reference pictures. When multiple reference pictures are used, the multiple reference pictures may be from different temporal directions or from the same temporal direction. The motion estimation unit (550) generates whether or not motion data (552) such as MV data, merge mode index value, reference picture selection data, and block inversion are used as side information. The motion data (552) is provided to the header formatter / entropy encoder (590) and the motion compensator (555).

  The motion compensation unit (555) applies MV to one or more reconstructed reference pictures from the decoded picture buffer (570). The motion compensation unit (555) generates motion compensated prediction for the current picture. If block inversion is used, the motion compensation unit (555) may consider inversion for the prediction region (for the current block) relative to the reference region, as described below.

  How the intra picture estimation unit (540) performs intra picture prediction for a block of sample values of the current picture of the input video signal (505) in a separate path in the encoder (500) To decide. The current picture may be encoded in whole or in part using intra picture encoding. For intra spatial prediction, the intra picture estimator (540) uses the value of the current picture reconstruction (538) to determine the current block in the current picture from neighboring previously reconstructed sample values. Determine how to predict the sample value of. Alternatively, for intra BC prediction using a BV value, the intra picture estimation unit (540) estimates the displacement of the sample value of the current block to a different candidate reference region in the current picture. The candidate reference region may include reconstructed sample values or, in some implementations for BV estimation, original sample values from the input video. The intra picture estimate (540) can evaluate different options for inversion of the intra BC prediction region (for the current block) for each candidate reference region, as described below.

  The intra picture estimation unit (540) includes, as side information, information indicating whether intra prediction uses spatial prediction or intra BC prediction, a prediction mode direction (for intra spatial prediction), and a BV value (for intra BC prediction). And intra prediction data (542) such as whether block inversion is used (for intra BC prediction). The intra prediction data (542) is provided to the header formatting unit / entropy encoding unit (590) and the intra picture prediction unit (545).

  The intra picture prediction unit (545) spatially predicts the sample value of the current block of the current picture from the adjacent reconstructed sample values of the current picture according to the intra prediction data (542). Alternatively, for intra BC prediction, the intra picture prediction unit (545) predicts the sample value of the current block using the previously reconstructed sample value of the intra frame prediction reference area indicated by the BV value of the current block. To do. For intra BC prediction, the intra picture prediction unit (545) may consider inversion for the intra BC prediction region (for the current block) relative to the reference region, as described below. In some cases, the BV value may be a BV predictor (predicted BV value). In other cases, the BV value may be different from its predicted BV value. If the chroma data for a picture has the same resolution as the luma data (eg, if the format is YUV 4: 4: 4 format or RGB 4: 4: 4 format), the BV value applied for the chroma block is the luma block May be the same as the BV value applied for. On the other hand, if the chroma data for the picture has a lower resolution than the luma data (eg, if the format is YUV 4: 2: 0 format), the BV value applied for the chroma block is reduced, if possible. Can be rounded to adjust for differences in chroma resolution (eg, by dividing the vertical and horizontal components of the BV value by 2 and truncating or rounding them to integer values).

  The intra / inter switch selects whether the prediction (558) for a given block is motion compensated prediction or intra picture prediction. For non-skip mode blocks, the difference (if any) between the prediction (558) block and the corresponding portion of the original current picture of the input video signal (505) provides the value of the residual (518). To do. For non-skip mode blocks, during reconstruction of the current picture, the reconstructed residual value is combined with the prediction (558) to approximate or accurate reconstruction of the original content from the video signal (505). (538) is generated. (In lossy compression, some information is lost from the video signal (505).)

  In the transform unit / scaling unit / quantization unit (530), the frequency transform unit transforms the spatial domain video information into frequency domain (ie, spectral transform) data. For block-based video coding, the frequency transform unit performs a discrete cosine transform (“DCT”), an integer approximation thereof, or another type of forward block transform (eg, a discrete sine transform or an integer approximation thereof) to predict residual data. (Or sample value data if prediction (558) is null) to generate a block of frequency transform coefficients. The encoder (500) can also indicate that such a conversion step is skipped. The scaling unit / quantization unit scales and transforms the transform coefficient. For example, the quantization unit may perform dead-zone scalar quanta on a per-frame, per-tile, per-slice, per-block, quantization-step size that varies in frequency-specific units or other units. Applying to the frequency domain data. The quantized transform coefficient data (532) is provided to the header formatter / entropy encoder (590).

  In the scaling unit / inverse transform unit (535), the scaling unit / inverse quantization unit performs inverse scaling and inverse quantization on the quantized transform coefficient. The inverse frequency transform unit performs inverse frequency transform to generate a block of reconstructed prediction residual values or sample values. For non-skip mode blocks, the encoder (500) combines the reconstructed residual value with the value of the prediction (558) (eg, motion compensated prediction value, intra picture prediction value) to reconstruct (538). Is generated. For the skip mode block, encoder (500) uses the value of prediction (558) as reconstruction (538).

  Regarding intra picture prediction, the value of reconstruction (538) may be fed back to the intra picture estimation unit (540) and the intra picture prediction unit (545). The value of reconstruction (538) can also be used for motion compensated prediction of subsequent pictures. The value of reconstruction (538) may be further filtered. The filtering controller (560) determines how to perform deblocking filtering and SAO filtering on the value of reconstruction (538) for a given picture of the video signal (505). The filtering control unit (560) generates filtering control data (562). The filtering control data (562) is provided to the header formatting unit / entropy encoding unit (590) and the merging unit / 1 or more filters (565).

  In the merge unit / 1 or more filters (565), the encoder (500) merges content from different tiles into a reconstructed version of the picture. The encoder (500) selectively performs deblocking filtering and SAO filtering according to the filtering control data (562) to adaptively smooth discontinuities across the boundaries in the frame. Alternatively or additionally, other filtering (such as deringing filtering or ALF; not shown) may be applied. The tile boundaries can be selectively filtered or not filtered, depending on the settings of the encoder (500), and the encoder (500) uses a syntax to indicate whether such filtering has been applied or not. Can be provided in the stream. The decoded picture buffer (570) buffers the reconstructed current picture for use in subsequent motion compensated prediction.

  The header formatting unit / entropy encoding unit (590) includes general control data (522), quantized transform coefficient data (532), intra prediction data (542), motion data (552), and filtering control data ( 562) is formatted and / or entropy encoded. Regarding the intra prediction data (542), the header formatting unit / entropy encoding unit (590) can select and entropy encode a BV predictor index value (for intra BC prediction). The header formatter / entropy encoder (590) may also entropy encode a syntax element that indicates whether block inversion is used for intra BC prediction (or motion compensation). In some cases, the header formatter / entropy encoder (590) also determines the BV difference of the BV value (the BV difference for the BV predictor for the BV value) and then, for example, context adaptive binary arithmetic Encode the BV difference using encoding. Specifically, the BV difference is signaled for the skip mode intra BC prediction block.

  The header formatter / entropy encoder (590) provides the encoded data in the encoded video bitstream (595). The format of the encoded video bitstream (595) is H.264. H.265 / HEVC format modification or expansion, Windows (registered trademark) Media Video format, VC-1 format, MPEG-x format (eg, MPEG-1, MPEG-2, or MPEG-4), H.264. 26x format (eg, H.261, H.262, H.263, H.264), or another format.

  Depending on the implementation and the type of compression desired, additional encoder modules may be added, omitted, split into multiple modules, or combined with other modules. And / or may be replaced with similar modules. In alternative embodiments, encoders with different modules and / or other configurations of modules perform one or more of the described techniques. Certain embodiments of the encoder typically use a modified or complementary version of the encoder (500). The illustrated relationship between modules within the encoder (500) is indicative of the general flow of information within the encoder. Other relationships are not shown for brevity.

VI. Exemplary Video Decoder FIG. 6 is a block diagram of a generalized decoder (600) that can implement several described embodiments in concert. The decoder (600) receives the encoded data in the encoded video bitstream (605) and generates an output that includes pictures for the reconstructed video (695). The format of the encoded video bitstream (605) is H.264. H.265 / HEVC format modification or expansion, Windows (registered trademark) Media Video format, VC-1 format, MPEG-x format (eg, MPEG-1, MPEG-2, or MPEG-4), H.264. 26x format (eg, H.261, H.262, H.263, H.264), or another format.

  The decoder (600) is block based and uses an implementation dependent block format. Blocks can be further subdivided at various stages. For example, a picture may be divided into 64 × 64 blocks, 32 × 32 blocks, or 16 × 16 blocks, which in turn may be divided into blocks with smaller sample values. H. In the decoding implementation for the H.265 / HEVC standard, a picture is divided into CTU (CTB), CU (CB), PU (PB), and TU (TB).

  The decoder (600) decompresses the picture using intra picture decoding and / or inter picture decoding. Many of the components of the decoder (600) are used for both intra picture decoding and inter picture decoding. The exact operations performed by these components can vary depending on the type of information being decompressed.

  The buffer accepts the encoded data in the encoded video bitstream (605) and makes the received encoded data available to the analyzer / entropy decoder (610). The analysis / entropy decoding unit (610) entropy decodes the entropy encoded data by applying the inverse of the entropy encoding (eg, context adaptive binary arithmetic decoding) normally performed in the encoder (500). To do. As a result of analysis and entropy decoding, the analysis unit / entropy decoding unit (610) includes general control data (622), quantized transform coefficient data (632), intra prediction data (642), motion data (652), And filtering control data 662 is generated. Regarding the intra prediction data (642), the analysis unit / entropy decoding unit (610) performs entropy decoding on the BV predictor index value (for intra BC prediction). The analysis unit / entropy decoding unit (610) also entropy decodes a syntax element indicating whether block inversion is used for intra BC prediction (or motion compensation). In some cases, the analyzer / entropy decoder (610) also entropy decodes the BV difference of the BV value (eg, using context adaptive binary arithmetic decoding) and then converts the BV difference to the corresponding BV prediction. Combine with the child to reconstruct the BV value. Specifically, for a skip mode intra BC prediction block, the BV difference is analyzed from the bitstream and combined with a BV predictor (eg, indicated by a BV predictor index value) to reconstruct the BV value.

  The general decoding control unit (620) receives the general control data (622), and sets other modules (scaling unit / inverse conversion unit (635), intra picture prediction unit) to set and change decoding parameters during decoding. (645), a motion compensation unit (655), an intra / inter switch, etc.) are supplied with a control signal (not shown).

  When the current picture is predicted using inter-picture prediction, the motion compensation unit (655) performs motion data (652) such as MV data, reference picture selection data, a merge mode index value, and a block (for motion compensation). A syntax element is received indicating whether inversion is used. The motion compensation unit (655) applies MV to one or more reconstructed reference pictures from the decoded picture buffer (670). If block inversion is used, the motion compensation unit (655) may consider inversion for the prediction region (for the current block) relative to the reference region, as described below. The motion compensation unit (655) generates motion compensated prediction for inter-coded blocks in the current picture. The decoded picture buffer (670) stores one or more previously reconstructed pictures for use as reference pictures.

  In the separation path in the decoder (600), the intra picture prediction prediction unit (645) includes information indicating whether the intra prediction uses spatial prediction or intra BC prediction, a prediction mode direction (for intra spatial prediction), Intra prediction data (642) such as a BV value (for intra BC prediction) and a syntax element (for intra BC prediction) indicating whether block inversion is used or not are received. For intra spatial prediction, the intra picture prediction unit (645) uses the value of the current picture reconstruction (638) from the previous previously reconstructed sample values of the current picture according to the prediction mode data. Predict spatially the sample value of the current block of the picture. Alternatively, for intra BC prediction using the BV value, the intra picture prediction unit (645) uses the previously reconstructed sample value of the intra frame prediction reference area indicated by the BV value of the current block, and Predict the sample value of. For intra BC prediction, the intra picture prediction unit (645) may consider inversion for the intra BC prediction region (for the current block) relative to the reference region, as described below.

  The intra / inter switch selects whether the prediction (658) for a given block is motion compensated prediction or intra picture prediction. For example, H.M. In accordance with the H.265 / HEVC syntax, an intra / inter switch may be controlled based on one or more syntax elements encoded for a CU in a picture that may include an intra-predicted CU and an inter-predicted CU. For non-skip mode blocks, the decoder (600) combines the prediction (658) with the reconstructed residual value to produce a reconstruction (638) of content from the video signal. For the skip mode block, the decoder (600) uses the value of prediction (658) as reconstruction (638).

  In order to reconstruct the residual for the non-skip mode block, the scaling / inverse transform (635) receives and processes the quantized transform coefficient data (632). In the scaling unit / inverse transform unit (635), the scaling unit / inverse quantization unit performs inverse scaling and inverse quantization on the quantized transform coefficient. The inverse frequency transform unit performs inverse frequency transform to generate a block of reconstructed prediction residual values or sample values. For example, the inverse frequency transform unit applies the inverse block transform to the frequency transform coefficient to generate sample value data or prediction residual data. The inverse frequency transform may be an inverse DCT, an integer approximation thereof, or another type of inverse frequency transform (eg, an inverse discrete sine transform or an integer approximation thereof).

  For intra picture prediction, the value of reconstruction (638) may be fed back to the intra picture prediction unit (645). For inter-picture prediction, the reconstruction (638) value may be further filtered. In the merge unit / one or more filters (665), the decoder (600) merges content from different tiles into a reconstructed version of the picture. The decoder (600) selectively performs deblocking filtering and SAO filtering according to the filtering control data (662) and rules for filter adaptation to adaptively smooth discontinuities across the boundaries in the frame. Run. Alternatively or additionally, other filtering (such as deringing filtering or ALF; not shown) may be applied. Tile boundaries can be selectively filtered or not filtered depending on the settings of the decoder (600) or syntax indications in the encoded bitstream data. The decoded picture buffer (670) buffers the reconstructed current picture for use in subsequent motion compensated prediction.

  The decoder (600) may also include a post-processing filter. The post-processing filter (608) may include deringing filtering, adaptive winner filtering, film-grain reproduction filtering, SAO filtering, or another type of filtering.

  Depending on the implementation and the type of decompression desired, decoder modules may be added, omitted, split into multiple modules, or combined with other modules. And / or may be replaced with similar modules. In alternative embodiments, decoders with different modules and / or other configurations of modules perform one or more of the described techniques. Certain embodiments of the decoder typically use a modified or complementary version of the decoder (600). The relationship shown between the modules in the decoder (600) shows the general flow of information in the decoder. Other relationships are not shown for brevity.

VII. Innovations in intra block copy prediction This section presents the features of intra block copy (“BC”) prediction. For example, some of these features relate to block inversion, where an intra BC prediction block is inverted with respect to a reference region that may be indicated by a block vector (“BV”) value. Another feature relates to skip mode signaling where the current intra BC prediction block uses the signaled BV difference but has no residual data. In many situations, these features improve the coding efficiency of intra BC prediction blocks.

  Specifically, the described innovation can improve rate distortion performance when encoding predetermined “artificially” created video content, such as screen capture content. Screen capture content generally includes repetitive structures (eg, graphics, text characters), which provides an opportunity for intra BC prediction to improve performance. Screen capture content is typically encoded in a high chroma sampling resolution format (eg, YUV 4: 4: 4 or RGB 4: 4: 4), but a lower chroma sampling resolution format (eg, YUV 4: 2: 0). ). Common scenarios for encoding / decoding screen capture content include remote desktop conferencing and encoding / decoding of graphical overlays on natural or other “mixed content” video.

A. Intra BC Prediction Mode, BV Value, and BV Prediction—Introduction For intra BC prediction, the sample value of the current block in a picture is predicted using the sample value in the same picture. The BV value indicates the displacement from the current block to a region in the picture (“reference region”) that contains the sample values used for prediction. The reference area provides a value predicted for the current block (“intra BC prediction area”). The sample values used for prediction are the previously reconstructed sample values and are therefore available at the encoder during encoding and at the decoder during decoding. The BV value may be signaled included in the bitstream, and the decoder may use the BV value to determine a reference region (reconstructed at the decoder) in the picture to use for prediction. it can. Intra BC prediction is a form of intra-picture prediction, and intra BC prediction for blocks in a picture does not use any sample values other than the sample values in the same picture.

  FIG. 7a shows intra BC prediction for the current block (760) in the current frame (710). The current block includes a coding block (“CB”) of the coding unit (“CU”), a prediction block (“PB”) of the prediction unit (“PU”), and a transform block (“TU”) of the transform unit (“TU”). TB "), or other blocks. The size of the current block may be 64x64, 32x32, 16x16, 8x8, or some other size. More generally, the size of the current block is m × n, where each of m and n is an integer, and m and n may be equal to each other or may have different values. . Alternatively, the current block may have some other shape (eg, a region of a non-rectangular encoded video object).

BV (761) indicates the displacement (or offset) from the current block (760) to the reference region (780) in the picture containing the sample values used for prediction. The reference region (780) indicated by BV (761) is sometimes referred to as the “collation block” for the current block (760). The verification block may be the same as the current block (760) or may be an approximation of the current block (760). Assume that the upper left position of the current block is at position (x ₀ , y ₀ ) in the current frame, and the upper left position of the reference area is at position (x ₁ , y ₁ ) in the current frame. BV represents a displacement (x ₁ −x ₀ , y ₁ −y ₀ ). For example, if the upper left position of the current block is at position (256, 128) and the upper left position of the reference area is at position (176, 104), the BV value is (−80, −24). In this example, a negative horizontal displacement indicates a position to the left of the current block, and a negative vertical displacement indicates a position above the current block.

  Intra BC prediction can improve coding efficiency by using redundancy (such as a repetitive pattern inside a frame) using a BC operation. Instead of encoding the current block sample values directly, the current block sample values are represented using BV values. Even if the current block sample value does not exactly match the reference region sample value indicated by the BV value, this difference can be negligible (not perceptually noticeable). . Alternatively, if the difference is large, the difference can be encoded as residual data that can be compressed more efficiently than the original sample value for the current block.

  Collectively, the BV value of a block encoded using intra BC prediction may consume a large number of bits. The BV value can be entropy encoded to reduce the bit rate. To further reduce the bit rate for BV values, the encoder can use prediction of BV values. BV values often indicate redundancy. That is, the BV value of a given block is often similar to or even the same as the BV value of the previous block in the picture. For BV prediction, the BV value for a given block is predicted using a BV predictor. The difference between the BV value of the given block and the BV predictor (ie BV difference) is then entropy encoded. Normally, the BV difference is calculated for the horizontal component and the vertical component of the BV value and the BV predictor. If BV prediction works well, the BV difference has a probability distribution that supports efficient entropy coding. H. In one draft version of the H.265 / HEVC standard (JCTVC-P1005), the BV predictor is the BV value of the last encoded CU in the current CTU (ie, the previous intra BC prediction block in the current CTU). BV value of Alternatively, the BV predictor is selected from among a plurality of available BV values (eg, in adjacent regions around the current block).

  FIG. 7b shows the BV (761) of the current block (760) in the frame and the BV (751) of the previous block (750) in the frame (710). The BV (751) of the previous block (750) is used as the BV predictor for the BV (761) of the current block (760). For example, when the BV value is (−80, −24) and the BV predictor is (−80, −32), the BV difference of (0, 8) is entropy encoded.

  The decoder receives the entropy-encoded BV difference of the BV value and performs entropy decoding. The decoder also determines a BV predictor for the BV value. The BV predictor determined by the decoder is the same as the BV predictor determined by the encoder. The decoder combines the BV predictor and the decoded BV difference to reconstruct the BV value.

B. Block Inversion in Intra BC Prediction In previous approaches to intra BC prediction, the reference region indicated by the BV value provides an intra BC prediction region for the current block. That is, the sample value of the reference area is an intra BC prediction value for the current block.

  According to one aspect of the innovation described herein, the intra BC prediction region for the current block is inverted with respect to the reference region. The BV value of the current block can indicate a reference area. Instead of using the reference area directly as the intra BC prediction area for the current block, the reference area can be flipped horizontally and / or vertically. In particular, block inversion can improve the encoding efficiency of text characters in screen capture content.

1. Example of Block Inversion FIGS. 8a to 8d, 9a to 9c, and 10a to 10c show examples of block inversion in intra BC prediction.

  FIG. 8a shows the current block (860) in the current picture (810). The current block (860) includes the text character p shown in detail in FIG. 8b. The BV (861) of the current block (860) indicates the displacement to the reference area (880) in the current picture (810). The reference area (880) contains the text character d shown in detail in FIG. 8c. Without inversion, the reference region (880) is a poor predictor for the current block (860). (The difference between the current block (860) and the reference area (880) for each sample is large.)

  Encoders and decoders can use block inversion in intra BC prediction to improve coding efficiency. For example, the reference region (880) can be flipped horizontally and vertically, as shown in the inverted reference region (881) of FIG. 8d. In this example, when the reference area (880) is flipped horizontally and / or vertically, the flipped reference area (881) exactly matches the current block (860). (That is, this intra BC prediction region is optimal for the current block (860), and the residual contains only zero value samples.)

  Similarly, the reference area (880) can be flipped horizontally. For example, if a given block contains the text character b, a horizontal inversion of the reference area (880) can result in an inverted reference area that exactly matches the given block. Alternatively, the reference region (880) can be flipped vertically. For example, if a given block contains the text character q, a vertical flip of the reference region (880) can result in an inverted reference region that exactly matches the given block.

  With block reversal, for many fonts, blocks containing text characters in a set of text characters (eg, a set of b, d, p, and q, or a set of u and n) It can be accurately predicted from a reference area containing another text character in the set. For other fonts, a block containing a text character in a set of text characters can be roughly predicted from a reference area containing another text character in the same set of text characters. More generally, block inversion can improve the encoding efficiency of various alphabetic text characters or other patterns in the screen content.

  Thus, with block inversion, a block containing a text character (or other pattern) is intra BC predicted even if that text character (or other pattern) has not previously appeared in the picture. This can improve encoding efficiency compared to other methods of encoding blocks. Alternatively, even if a text character (or other pattern) has not previously appeared in the picture, the intra BC prediction is closer to this block than a reference region containing the same text character (or other pattern) An inverted reference area can be used. BV values for closer inverted regions can be encoded much more efficiently than BV values for more distant reference regions. For example, for a current block that includes text character q, a first candidate reference region that includes the same text character q may be indicated by a BV value (−280, −24), whereas a second that includes a different text character p. Is assumed to be indicated by the BV value (−32, 0). The second candidate reference area can be flipped horizontally to exactly match the current block. The BV value for the second candidate reference region is encoded more efficiently (with fewer bits) than the BV value for the first candidate reference region, even considering the signaling of syntax elements indicating block inversion. obtain.

  As shown in FIGS. 9a-9c, the block and reference area may include multiple text characters. FIG. 9a shows the current block (960) containing the text character group dl. The BV value of the current block (960) indicates the displacement to the reference area (980) in the same picture. The reference area (980) includes a text character group lb, as shown in detail in FIG. 9b. Without inversion, the reference region (980) is a poor predictor for the current block (960). However, if the reference area (980) is flipped horizontally, the flipped reference area (981) exactly matches the current block (960).

  In the above example, the block and reference area include the entire text character. As shown in FIGS. 10a-10c, the blocks and reference regions may instead include one or more portions of text characters, symbols, or symbols. FIG. 10a shows the current block (1060) containing the text character L part. The BV value of the current block (1060) indicates the displacement to the reference area (1080) in the same picture. The reference area (1080) includes a portion of the text character F, as shown in detail in FIG. 10b. Without inversion, the reference region (1080) is a poor predictor for the current block (1060). However, if the reference area (1080) is flipped vertically, the flipped reference area (1081) exactly matches the current block (1060).

2. Exemplary Inversion Operation When block inversion is used in intra BC prediction, the intra BC prediction region for the block is inverted with respect to the reference region for the block. The block inversion operation can be implemented in various ways, depending on the implementation.

  According to one approach to performing block inversion operations, when determining the intra BC prediction region for the current block, the encoder or decoder (a) determines the reference region, (b) inverts the reference region, and then (C) The sample values at the respective positions of the inverted reference region are assigned to the sample values at the respective positions of the intra BC prediction region. For example, for a 16 × 16 block, the encoder or decoder determines the 16 × 16 reference region indicated by the BV value for that block, and then inverts the 16 × 16 reference region in the horizontal and / or vertical direction. This involves making a copy of the 16 × 16 reference area that is inverted. The sample value at the inverted reference region position is then assigned to the sample value at the same location in the intra BC prediction region (eg, the sample value at the inverted reference region position (0, 0) is the intra BC prediction). The sample value at the position (0,1) of the inverted reference region assigned to the sample value at the location (0,0) of the region is assigned to the sample value at the location (0,1) of the intra BC prediction region. Etc.).

  According to a second approach of performing block inversion operations, when determining the intra BC prediction region for the current block, the encoder or decoder determines (a) a reference region and (b) at each position of the reference region. Sample values are assigned to sample values at their respective positions in the intra BC prediction region, and then (c) the intra BC prediction region is inverted. For example, for a 16 × 16 block, the encoder or decoder determines a 16 × 16 reference region indicated by the BV value for that block. The sample value at the position of the reference area is assigned to the sample value at the same position of the intra BC prediction area (for example, the sample value at the position (0, 0) of the reference area is the position (0, 0) of the intra BC prediction area). And the sample value at the reference region position (0, 1) is assigned to the sample value at the intra BC prediction region position (0, 1), etc.). The encoder or decoder then inverts the 16 × 16 intra BC prediction region in the horizontal direction and / or the vertical direction. This involves making a copy of the inverted 16 × 16 intra BC prediction region.

  According to a third approach of performing block inversion operations, the encoder and decoder avoid making an intermediate copy of the reference region or intra BC prediction region. When determining the intra BC prediction region for the current block, the encoder or decoder (a) determines a reference region, and then (b) samples values at respective positions of the reference region corresponding to the intra BC prediction region. Assigned to the sample value at a position, where the corresponding position takes into account block inversion. When horizontal inversion is used, the first column of the reference region provides the last column of the intra BC prediction region, and the second column of the reference region represents the second column from the end of the intra BC prediction region. Provide, etc. When vertical inversion is used, the first row of the reference region provides the last row of the intra BC prediction region, and the second row of the reference region contains the second row from the end of the intra BC prediction region. Provide, etc. When both horizontal inversion and vertical inversion are used, the position of the reference area can be scanned in reverse order in the horizontal and vertical directions when assigning sample values to the positions of the intra BC prediction areas. For example, for a 16 × 16 block, the encoder or decoder determines a 16 × 16 reference region indicated by the BV value for that block. The sample value at the position of the reference region is assigned to the sample value at the corresponding position of the intra BC prediction region in reverse order in the horizontal direction and / or the vertical direction (for example, the sample value at the position (0, 0) of the reference region is The sample value at the position (15, 15) of the intra BC prediction region is assigned to the sample value at the position (0, 1) of the reference region, and is assigned to the sample value at the position (15, 14) of the intra BC prediction region. , Etc.).

3. Exemplary Signaling for Block Inversion If block inversion is enabled for intra BC prediction, the decision whether to use block inversion can be signaled in various ways, depending on the implementation.

  Block inversion may be enabled for video sequences, pictures, or other units. Sequence layer syntax elements (eg, in SPS), picture layer syntax elements (eg, in PPS), or slice header layer syntax elements (eg, in slice segment headers) are enabled for block inversion. Can indicate whether it is disabled or disabled. Alternatively, block inversion may be enabled for several profiles or levels of encoding and decoding. The decision to enable block inversion can be made on a per-direction basis (eg, horizontal block inversion only, vertical block inversion only, or both horizontal and vertical block inversion). If block inversion is enabled, an additional syntax element signals when and how block inversion is used.

  If only vertical inversion is enabled, the flag value can indicate whether vertical inversion is used during intra BC prediction. If only horizontal inversion is enabled, the flag value can indicate whether horizontal inversion is used during intra BC prediction. If both vertical and horizontal inversion are enabled, the two flag values indicate whether inversion is used for vertical and horizontal inversion during intra BC prediction. Each flag indicates a decision about the direction of inversion. Alternatively, a single syntax element with multiple values may be used (e.g., possible values are vertical inversion only, horizontal inversion only, both vertical inversion and horizontal inversion, Or no inversion).

  A syntax element (eg, flag value) indicating whether block inversion is used for the current block may be signaled included in the bitstream along with other syntax elements for the current block. For example, one or more syntax elements related to block inversion for the PU are signaled for the PU. Alternatively, a syntax element indicating whether block inversion is being used for the current block may be signaled included in the bitstream for larger blocks including the current block. For example, one or more syntax elements for block inversion for one or more PUs are signaled for a CU that includes one or more PUs. Alternatively, a syntax element indicating whether block inversion is being used for the current block is included in the bitstream and signaled at some other level.

  A syntax element indicating whether block inversion is used for the current block may be entropy encoded. For example, the flag value for the current block is encoded using context adaptive binary arithmetic coding and decoded using context adaptive binary arithmetic decoding. Alternatively, different forms of entropy coding may be used, or syntax elements may be signaled as fixed length values.

  One or more syntax elements indicating whether block inversion is used for the current block may be included in the bitstream and conditionally signaled separately. For example, a flag value indicating whether block inversion is used may be signaled if the current block is intra BC predicted, but not signaled if the current block is not intra BC predicted. Alternatively, one or more syntax elements that indicate whether block inversion is being used for the current block may be jointly encoded with another syntax element in the bitstream. For example, a flag value indicating whether or not block inversion is used may be jointly encoded together with a flag value indicating whether or not the current block is intra BC predicted.

4). Exemplary Coding Using Block Inversion in Intra BC Prediction FIG. 11 shows an exemplary technique (1100) for block inversion in intra BC prediction during encoding. An image or video encoder, such as the encoder described with reference to FIG. 3 or FIGS. 5a-5b, may perform the technique (1100).

  The encoder determines an intra BC prediction region for the current block in the picture based on the reference region in the picture (1110). The current block may be a PU, CU, or other block. The BV value of the current block identified in the BV estimation can indicate the displacement to the reference area in the picture. The intra BC prediction area is inverted with respect to the reference area. For example, the intra BC prediction region is inverted in the horizontal direction and / or the vertical direction with respect to the reference region. An example approach to performing a block inversion operation is described above (see section VII.B.2).

  The encoder encodes the current block using the intra BC prediction region (1120), and outputs the encoded data in the bitstream (1130). The encoded data includes an indication as to whether the intra BC prediction region is inverted with respect to the reference region. For example, an indication is one or more syntax elements in a bitstream. An example of an approach for signaling whether block inversion is used is described above (see section VII.B.3).

  The encoder can similarly encode other intra BC prediction blocks for each block, with or without block inversion, for slices, tiles, or pictures.

5. Exemplary Decoding Using Block Inversion in Intra BC Prediction FIG. 12 shows an exemplary technique (1200) for block inversion in intra BC prediction during decoding. An image or video decoder such as the decoder described with reference to FIG. 4 or FIG. 6 may perform the technique (1200).

  The decoder receives encoded data in the bitstream (1210). The encoded data includes an indication of whether the intra BC prediction area for the current block in the picture is inverted with respect to the reference area in the picture. The current block may be a PU, CU, or other block. For example, an indication is one or more syntax elements in a bitstream. An example of an approach for signaling whether block inversion is used is described above (see section VII.B.3).

  The decoder determines an intra BC prediction region for the current block in the picture based on the reference region in the picture (1220). The BV value of the current block can indicate the displacement to the reference area. The intra BC prediction area is inverted with respect to the reference area. For example, the intra BC prediction region is inverted in the horizontal direction and / or the vertical direction with respect to the reference region. An example approach to performing a block inversion operation is described above (see section VII.B.2). The decoder decodes the current block using the intra BC prediction region (1230).

  The decoder can similarly decode other intra BC prediction blocks for each block, with or without block inversion, for slices, tiles, or pictures.

C. Skip Mode for Intra BC Prediction In some previous approaches to intra BC prediction, a flag for the current CU indicates whether the current CU is encoded in intra BC prediction mode. If the current CU is encoded in intra BC prediction mode, the second flag for the current CU indicates whether the current CU has residual data. This scheme of signaling intra BC prediction blocks without residual data is not efficient in many screen content encoding / decoding scenarios.

  According to another aspect of the innovation described herein, the encoder and decoder use a flag to signal an intra BC prediction block that does not have residual data. In skip mode, the intra BC prediction block uses the BV value signaled with the BV difference included in the bitstream, but does not have residual data in the bitstream. In particular, for screen capture content, intra BC prediction blocks without residual data often occur. It is efficient in such a scenario to use one flag (rather than multiple flags) for signaling intra BC prediction blocks that do not have residual data.

1. Exemplary Signaling for Intra BC Prediction Skip Mode In an exemplary implementation, one flag in the bitstream indicates whether the current block is an intra BC prediction block in skip mode. If the current block is not an intra BC prediction block in skip mode, the second flag in the bitstream indicates whether the current block is an intra BC prediction block (not in skip mode). If the current block is not an intra BC prediction block, one or more other syntax elements in the bitstream indicate the mode of the current block (eg, flag for whether or not temporal skip mode, intra spatial prediction mode, etc. Flag for whether or not, flag for whether or not the inter picture mode, flag for whether or not the intra spatial prediction mode or the inter picture mode). If the current block is an intra BC prediction block in skip mode, the second flag and other syntax elements are not present in the bitstream for the current block.

  A flag indicating whether or not the current block is an intra BC prediction block in skip mode is signaled at the block level. The current block may be a CU having a size 2N × 2N. For example, in the case of 16 × 16 CU, N is 8. Alternatively, the current block may be a PU or other type of block. Other flags and syntax elements indicating the mode of the current block may also be included in the bitstream and signaled at the block level.

  In some exemplary implementations, an intra BC prediction block in skip mode includes a BV difference in the bitstream but no residual data. Alternatively, an intra BC prediction block in skip mode may use the predicted BV value (thus not including BV differences in the bitstream).

2. Exemplary Coding Using Intra BC Prediction Skip Mode FIG. 13 shows an exemplary technique (1300) for encoding an intra BC prediction block in skip mode. An image encoder or video encoder, such as the encoder described with reference to FIG. 3 or FIGS. 5a-5b, may perform the technique (1300).

  The encoder determines the BV value of the current block (eg, CU, PU) in the picture using, for example, BV estimation (1310). The BV value of the current block indicates the displacement to the reference area in the picture. The encoder then determines (1320) the BV difference of the current block using the BV value of the current block and the BV predictor of the current block. The BV predictor may be selected by a rule, or the encoder may select a BV predictor from a plurality of BV predictor candidates.

  The encoder encodes (1330) the current block using intra BC prediction using the BV value. In an exemplary implementation, intra BC prediction may include determining an intra BC prediction region for the current block using a reference region, where the intra BC prediction region is inverted with respect to the reference region. ing. The intra BC prediction region may be inverted in the horizontal direction and / or the vertical direction with respect to the reference region. Options for performing block inversion operations and signaling that block inversion is being used are described above. Alternatively, the encoder does not use block inversion in intra BC prediction.

  The encoder outputs data encoded in the bitstream (1340). The encoded data includes a flag indicating that the current block is encoded in skip mode using intra BC prediction. Since the current block is an intra BC prediction block in the skip mode, the bitstream includes the BV difference of the current block but does not include the residual data of the current block. When the encoder selects a BV predictor from among a plurality of BV predictor candidates, the bitstream includes an index value indicating the selected BV predictor candidate for use as the BV predictor of the current block.

  FIG. 14 shows an example technique (1400) for encoding blocks in a picture in skip mode and / or other modes using intra BC prediction. An image encoder or video encoder, such as the encoder described with reference to FIG. 3 or FIGS. 5a-5b, may perform the technique (1400).

  First, the encoder obtains the next block (1410) and determines whether to encode this block in skip mode using intra BC prediction (1420). For example, the encoder evaluates whether intra BC prediction provides sufficient coding efficiency for the block and evaluates whether the residual data contains large values. Alternatively, the encoder considers other criteria.

  The encoder can signal by including in the bitstream a flag indicating whether the block is encoded in skip mode using intra BC prediction. For intra BC prediction blocks in skip mode, the encoder blocks in skip mode using intra BC prediction using the operations shown in steps (1310) to (1330) of FIG. 13 or using another approach. Is encoded (1430). Since this is an intra BC prediction block in skip mode, this block has a BV difference in the bitstream, but no residual data in the bitstream.

  Otherwise (if the block is not an intra BC prediction block in skip mode), the encoder encodes the block in another mode (1440). Another mode may be intra BC prediction non-skip mode, intra spatial prediction mode, inter picture mode, or other modes. In this case, one or more other syntax elements may indicate the mode of the block. For example, after a first flag indicating whether the block is encoded in skip mode using intra BC prediction (according to the determination in step (1420)), the block uses intra BC prediction. If not encoded in skip mode, the second flag indicates whether the block is encoded in non-skip mode using intra BC prediction. If the block is not encoded in non-skip mode using intra BC prediction, one or more other syntax elements indicate the encoding mode of the block. For example, a certain flag indicates whether the prediction mode of a block is intra spatial prediction or inter picture prediction.

  In some exemplary implementations, there is a further advantage of using a flag that indicates whether the block is intra BC predicted in skip mode. In some cases, syntax element signaling indicating a split mode (eg, 2N × 2N, 2N × N, N × 2N, or N × N) of an intra BC prediction block may be avoided. For example, when a block is encoded in the non-skip mode using intra BC prediction, the encoder signals the syntax element indicating the division mode of the block in the bitstream. On the other hand, when a block is encoded in the skip mode using intra BC prediction, the encoder skips signaling by including a syntax element indicating the division mode of the block in the bitstream, and The split mode is instead considered to have a defined value (eg 2N × 2N). Therefore, in these cases, it is also signaled that the flag to which the block is marked as intra BC predicted in skip mode has a value that defines the partition mode of the block.

  In many cases, signaling of a flag indicating the presence / absence of residual data of a block can be avoided. Of course, if the block is encoded in skip mode using intra BC prediction, the encoder skips signaling by including a flag indicating the presence or absence of residual data in the block in the bitstream. (The flag to which the block is marked as intra BC predicted in skip mode signals such information.) The residual data for the block is considered not in the bitstream.

  Signaling of a flag indicating the presence or absence of residual data can be avoided in other cases. Specifically, if the block is encoded in non-skip mode using intra BC prediction and the block partition mode has a defined value (eg, 2N × 2N), the encoder Skipping signaling by including a flag indicating the presence or absence of difference data in the bitstream. In this case, the residual data of the block is considered to be in the bitstream. (If the block split mode is a defined value and the block does not have residual data, the block will be an intra BC prediction block in skip mode as indicated by the previous flag. ) Finally, if the block has been encoded in non-skip mode using intra BC prediction and the block split mode does not have a defined value, the encoder will determine if there is residual data in the block. A flag to indicate is included in the bitstream and signaled.

  The encoder determines (1450) whether to proceed to the next block in the picture. If proceeding to the next block, the encoder obtains the next block (1410) and continues encoding.

  The encoder may repeat the technique (1400) for each picture, for each tile, for each slice, or in some other unit.

3. Exemplary Decoding Using Intra BC Prediction Skip Mode FIG. 15 shows an exemplary technique (1500) for decoding an intra BC prediction block in skip mode. An image or video decoder such as the decoder described with reference to FIG. 4 or FIG. 6 may perform the technique (1500).

  The decoder receives encoded data from the bitstream (1510). The encoded data includes a flag indicating that the current block (eg, CU, PU) in the picture is encoded in the skip mode using intra BC prediction. Since the current block is an intra BC prediction block in the skip mode, the bitstream includes the BV difference of the current block but does not include the residual data of the current block.

  The decoder uses the BV difference of the current block and the BV predictor of the current block to determine a current block BV value (1520). The BV value of the current block indicates the displacement to the reference area in the picture. The BV predictor can be selected by rules. Alternatively, the decoder uses an index value in the bitstream to select which BV predictor candidate to use for use as the BV predictor of the current block, and uses the BV predictor candidates from among the plurality of BV predictor candidates. Can be selected.

  The decoder decodes the current block using intra BC prediction using the BV value (1530). In an exemplary implementation, intra BC prediction may include determining an intra BC prediction region for the current block using a reference region, where the intra BC prediction region is inverted with respect to the reference region. ing. The intra BC prediction region may be inverted in the horizontal direction and / or the vertical direction with respect to the reference region. Options for performing block inversion operations and signaling that block inversion is being used are described above. Alternatively, the decoder does not use block inversion in intra BC prediction.

  FIG. 16 shows an example technique (1600) for decoding blocks in a picture in skip mode and / or other modes using intra BC prediction. An image or video decoder, such as the decoder described with reference to FIG. 4 or FIG. 6, may perform the technique (1600).

  First, the decoder obtains encoded data for the next block (1610) and determines whether to decode this block in skip mode using intra BC prediction (1620). For example, the decoder receives and analyzes a flag indicating whether or not a block in the bitstream is encoded in the skip mode using intra BC prediction.

  For intra BC prediction blocks in skip mode, the decoder blocks in skip mode using intra BC prediction using the operations shown in steps (1520) and (1530) of FIG. 15 or using another approach. Is decoded (1630). Since this is an intra BC prediction block in skip mode, this block has a BV difference in the bitstream, but no residual data in the bitstream.

  Otherwise (if the block is not an intra BC prediction block in skip mode), the decoder decodes the block in another mode (1640). Another mode may be intra BC prediction non-skip mode, intra spatial prediction mode, inter picture mode, or other modes. In this case, one or more other syntax elements may indicate the mode of the block. For example, after a first flag indicating whether the block is encoded in skip mode using intra BC prediction (according to the determination in step (1620)), the block uses intra BC prediction. If not encoded in skip mode, the second flag indicates whether the block is encoded in non-skip mode using intra BC prediction. If the block is not encoded in non-skip mode using intra BC prediction, one or more other syntax elements indicate the encoding mode of the block. For example, a certain flag indicates whether the prediction mode of a block is intra spatial prediction or inter picture prediction.

  As noted in the previous section, in some exemplary implementations, there is a further advantage of using a flag that indicates whether a block is intra BC predicted in skip mode. For example, if a block is encoded in non-skip mode using intra BC prediction, the bitstream includes a syntax element that indicates the partition mode of the block. On the other hand, when the block is encoded in the skip mode using intra BC prediction, the bit stream does not include a syntax element indicating the division mode of the block. The decoder infers that the block division mode is a predetermined value (for example, 2N × 2N).

  In many cases, signaling of a flag indicating the presence / absence of residual data of a block can be avoided. Of course, if the block is encoded in skip mode using intra BC prediction, the bitstream does not include a flag indicating the presence or absence of residual data in the block. Instead, the decoder infers that there is no residual data for the block in the bitstream from the destination flag that marks the block as being intra BC predicted in skip mode.

  Signaling of a flag indicating the presence or absence of residual data can be avoided in other cases. Specifically, if a block is encoded in non-skip mode using intra BC prediction, and the partition mode of the block has a defined value (eg, 2N × 2N), the bitstream is Does not include a flag indicating the presence or absence of residual data. In this case, the decoder assumes that the residual data for the block is in the bitstream. Finally, if the block is encoded in non-skip mode using intra BC prediction, and the block partition mode does not have a defined value, the bitstream will determine whether there is residual data for the block. Contains a flag to indicate.

  The decoder determines (1650) whether to proceed to the next block in the picture. If proceeding to the next block, the decoder obtains the encoded data for the next block (1610) and continues decoding.

  The decoder may repeat the technique (1600) for each picture, for each tile, for each slice, or in some other unit.

4). Exemplary Syntax for an Encoding Unit FIGS. 17a and 17b show a syntax structure (1700) for an encoding unit (“CU”), according to a conventional approach. The syntax elements shown in the syntax structure (1700) are defined in JCTVC-P1005. The selected syntax elements are summarized here.

  The syntax element intra_bc_flag may be signaled for a CU if intra BC prediction is enabled (this is indicated by intra_block_copy_enabled_flag). The syntax element intra_bc_flag indicates whether or not the CU is encoded in the intra BC prediction mode. When the value of intra_bc_flag is 1, the CU is encoded in the intra BC prediction mode. When the value of intra_bc_flag is 0, the CU is not encoded in the intra BC prediction mode. In this case (when intra_bc_flag is 0), there is a syntax element pred_mode_flag, and this syntax element indicates whether the CU is encoded in the inter prediction mode or the intra spatial prediction mode.

  If the block is intra BC predicted (and a few other cases), the bitstream includes a part_mode syntax element for the CU. The part_mode syntax element indicates a partition mode of the CU (that is, 2N × 2N, 2N × N, N × 2N, N × N).

  Then, the CU syntax structure (1700) may include a BV value for the partition (when the CU is intra BC predicted), intra prediction direction information (when the CU is intra spatial predicted), or prediction unit information (CU In the case of inter prediction). The CU syntax structure (1700) then includes an rqt_root_cbf syntax element in some cases. Specifically, when the CU is predicted to be intra BC, the rqt_root_cbf syntax element is present. The rqt_root_cbf syntax element indicates whether or not a transform_tree () syntax structure exists for the CU. The transform_tree () syntax structure is for residual data about a CU. If rqt_root_cbf is 1, a transform_tree () syntax structure exists for the CU. If rqt_root_cbf is 0, no transform_tree () syntax structure exists for the CU. When rqt_root_cbf does not exist, the value of rqt_root_cbf is estimated to be 1. Therefore, as described above, in this approach, two flags (intra_bc_flag and rqt_root_cbf) are used to indicate the intra BC prediction block in the skip mode.

  FIG. 18 shows a new syntax structure (1800) for a coding unit that may be coded as an intra BC prediction block in skip mode. Changes to the syntax structure (1700) shown in FIGS. 17a and 17b are highlighted in FIG.

  intra_bc_skip_flag may be signaled for the CU if intra BC prediction is enabled (this is indicated by intra_block_copy_enabled_flag). The syntax element intra_bc_skip_flag indicates whether or not the CU is an intra BC prediction block in the skip mode. When intra BC is predicted in the skip mode (intra_bc_skip_flag is 1), the syntax element intra_bc_flag and the syntax element pred_mode_flag are omitted as in the syntax element rqt_root_cbf. In this case, the division mode of the CU has a predetermined value 2N × 2N.

  On the other hand, when the syntax element intra_bc_skip_flag indicates that the intra BC is not predicted in the skip mode (when intra_bc_skip_flag is 0), the syntax element intra_bc_flag exists. The syntax element intra_bc_flag indicates whether or not the CU is encoded in the intra BC prediction mode, as described with reference to FIGS. 17a and 17b. Further, when intra_bc_flag is 0, there is a syntax element pred_mode_flag, which is the same as that described with reference to FIGS. 17a and 17b, or whether the CU is encoded in the inter prediction mode. Indicates whether encoding is in spatial prediction mode.

  If the block is intra BC predicted in non-skip mode (ie, intra_bc_skip_flag is 0 and intra_bc_flag is 1), the bitstream includes a part_mode syntax element for the CU. The part_mode syntax element indicates a CU partition mode (for example, 2N × 2N, 2N × N, N × 2N, N × N). On the other hand, if the block is intra BC predicted in skip mode, the part_mode syntax element is included in the bitstream and not signaled for the CU. Instead, the split mode is assumed to have a defined value of 2N × 2N. Thus, for such a block, intra_bc_skip_flag signals that the CU partition mode has a defined value of 2N × 2N, and separate signaling of the part_mode syntax element is avoided.

  In the section omitted from FIG. 18, the syntax structure (1800) then shows the BV value for the partition (if CU is intra BC predicted), intra prediction direction information (CU is intra space predicted). Or prediction unit information (when CU is inter-predicted).

  Finally, the syntax structure (1800) includes an rqt_root_cbf syntax element in some cases. Specifically, when the CU is predicted to be intra BC in the non-skip mode, the rqt_root_cbf syntax element exists unless the division mode of the CU is a predetermined value of 2N × 2N. When the rqt_root_cbf syntax element is present, the value of the rqt_root_cbf syntax element indicates whether the transform_tree () syntax structure exists for the CU as described with reference to FIGS. 17a and 17b. On the other hand, when (1) the CU is predicted to be intra BC in skip mode, or (2) the CU is predicted to be intra BC, and the division mode of the CU is a predetermined value of 2N × 2N, There is no rqt_root_cbf syntax element. When the CU is predicted for intra BC in skip mode, the value of rqt_root_cbf is estimated to be 0 (there is no residual data for the CU). Alternatively, if the CU is predicted to be intra BC and the division mode of the CU is a predetermined value of 2N × 2N, the value of rqt_root_cbf is estimated to be 1 (there is residual data for the CU) )

  In view of the many possible embodiments to which the disclosed principles of the invention can be applied, it should be recognized that the illustrated embodiments are merely preferred examples of the invention and limit the scope of the invention. Should not be interpreted as. Instead, the scope of the invention is defined by the claims. We therefore claim as our invention all that comes within the scope and spirit of these claims.

Claims (10)

A method in a computing device having a video decoder or an image decoder comprising:
Receiving encoded data including a flag from a bitstream that includes a flag indicating that a current block in a picture is encoded in skip mode using intra block copy (BC) prediction; A stream includes block vector (BV) differences of the current block, but does not include residual data of the current block;
Determining a BV value of the current block using the BV difference of the current block and a BV predictor of the current block, wherein the BV value of the current block is transferred to a reference area in the picture; A step indicating the displacement of
Decoding the current block using intra BC prediction using the BV value;
Including methods.   The method of claim 1, wherein the bitstream further includes an index value indicating selection of a BV predictor candidate from a set of multiple BV predictor candidates for use as the BV predictor of the current block. For subsequent blocks in the picture:
The first flag indicates whether the subsequent block is encoded in skip mode using intra BC prediction;
If the subsequent block is not encoded in skip mode using intra BC prediction, a second flag indicates whether the subsequent block is encoded in non-skip mode using intra BC prediction. The method of claim 1, wherein: For subsequent blocks in the picture:
If the subsequent block is encoded in non-skip mode using intra BC prediction, the bitstream includes a syntax element indicating a split mode of the subsequent block;
When the subsequent block is encoded in skip mode using intra BC prediction, the bitstream does not include the syntax element indicating the division mode of the subsequent block, and the division mode of the subsequent block The method of claim 1, wherein has a defined value. For subsequent blocks in the picture:
When the subsequent block is encoded in skip mode using intra BC prediction, the bitstream does not include a flag indicating the presence or absence of residual data in the subsequent block, and the residual data in the subsequent block The method of claim 1, wherein is considered not in the bitstream. For the subsequent block in the picture:
If the subsequent block is encoded in a non-skip mode using intra BC prediction, and the partition mode of the subsequent block has a predetermined value, the bit stream includes the residual data of the subsequent block. Does not include the flag indicating presence or absence, the residual data of the subsequent block is considered to be in the bitstream;
If the subsequent block is encoded in non-skip mode using intra BC prediction and the partition mode of the subsequent block does not have the defined value, the bitstream 6. The method of claim 5, comprising the flag indicating the presence or absence of residual data. The intra BC prediction is
Using the reference region to determine an intra BC prediction region for the current block, wherein the intra BC prediction region is inverted with respect to the reference region. Item 2. The method according to Item 1.   The method according to claim 7, wherein the intra BC prediction region is inverted in a horizontal direction and / or a vertical direction with respect to the reference region.   A computing device adapted to perform the method of any one of claims 1-8.   A program for causing a computing device to execute the method according to claim 1.

Images