KR102210946B1 - Dictionary encoding and decoding of screen content - Google Patents

Abstract

A method and device are provided for encoding and/or decoding video and/or image content using a dictionary mode. For example, the method and device predicts current pixel values from previous pixel values stored in a 1D dictionary. The method and device predicts current pixel values from previous pixel values using a pseudo 2D dictionary mode. In addition, the method and device predict current pixel values using a pseudo 2D dictionary mode. In addition, the method and device predicts current pixel values from previous pixel values of a reference picture using an inter pseudo 2D dictionary mode. Pixel values can be predicted from previous pixel values (eg, stored in a dictionary) identified by offset and length. In addition, the method and device encode pixel values using hash matching of pixel values.

Description

Dictionary encoding and decoding of screen content {DICTIONARY ENCODING AND DECODING OF SCREEN CONTENT}

Engineers use compression (also called source coding or source encoding) to reduce the bit rate of digital video. Compression reduces the cost of storing and transmitting video information by converting the information into a lower bit rate form. Decompression (also called decoding) reconstructs a version of the original information from its compressed form. "Codec" is an encoder/decoder system.

Over the past two decades, ITU-T H.261, H.262 (MPEG-2 or ISO/EC 13818-2), H.263 and H.264 (MPEG-4 AVC or ISO/IEC 14496-10) standards , MPEG-1 (ISO/IEC 11172-2) and MPEG-4 visual (ISO/IEC 14496-2) standards, and SMPTE 421M standards, various video codec standards have been adopted. More recently, the HEVC standard (ITU-T H.265 or ISO/IEC 23008-2) has been approved. Extensions to the HEVC standard (e.g., for scalable video coding/decoding, for coding/decoding of video with higher fidelity in terms of sample bit depth or chroma sampling rate, or for multi-view coding/decoding) An is currently under development. Video codec standards typically define options for the syntax of an encoded video bitstream, detailing the parameters in the bitstream when certain features are used in encoding and decoding. In many cases, the video codec standard also provides details regarding the decoding operation the decoder must perform to achieve a consistent result in decoding. In addition to the codec standard, various proprietary codec formats define other options for the syntax of the encoded video bitstream and the corresponding decoding operation.

Encoding and decoding certain types of content, such as screen content, can present different challenges than coding general video content. For example, the screen content may include an area of similar content (eg, a large graphic area having the same color or a smooth gradient) and an area of repeated content. Encoding and decoding such content using conventional coding techniques can result in inefficient and reducing quality (eg, by creating compression artifacts).

This summary is provided to introduce in a simplified form a selection of concepts that are further described in the detailed description that follows. This summary is not intended to identify major or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.

Techniques for improving the efficiency of encoding and/or decoding video and/or image data are described. In some innovations, pixel values are encoded and/or decoded using previous pixel values (e.g., previously reconstructed or previously decoded pixel values) stored in a one-dimensional (1D) dictionary. To do this, 1D dictionary mode is used. In the 1D dictionary mode, the current pixel value can be predicted using a length representing the number of pixel values being predicted and an offset identifying a location within the 1D dictionary (e.g., any residual is required. Can be accurately predicted without doing).

In some innovations, to encode and/or decode pixel values using previous pixel values (e.g., previously reconstructed or previously decoded pixel values), a pseudo 2D dictionary mode is used. . In 2D dictionary mode, the current pixel value can be predicted using X and Y offsets and lengths (eg, can be accurately predicted without requiring any residuals). In a reference picture (e.g., identified within the reference picture by a length and X and Y offsets from the corresponding pixel position in the reference picture corresponding to the current pixel position in the current picture being encoded or being decoded) An inter pseudo 2D dictionary mode may also be used to encode and/or decode pixel values using the pixel values of.

In another innovation, the encoder computes hash values for previously encoded pixel values (eg, for every 1, 2, 4, and 8 pixel values). The current pixel value being encoded is then matched against the previously encoded pixel value by generating a hash of the current pixel value and matching it to the hash value.

The techniques described herein can be applied to coding of screen content. Screen content refers to computer-generated video and/or image content (eg, text, graphics, and/or other computer-generated artificial content). One example of screen content is an image of a computer desktop graphical user interface, including text, icons, menus, windows, and/or other computer text and graphics. The techniques described herein can also be applied to content other than screen content.

The above and other objects, features, and advantages of the present invention will become more apparent from the following detailed description proceeding with reference to the accompanying drawings.

1 is a diagram of an exemplary computing system in which some of the described embodiments may be implemented.
2A and 2B are diagrams of an exemplary network environment in which some of the described embodiments may be implemented.
3 is a diagram of an exemplary encoder system in which several described embodiments may be implemented in conjunction.
4 is a diagram of an exemplary decoder system in which several described embodiments may be implemented in conjunction.
5A and 5B are diagrams illustrating an example video encoder in which some described embodiments may be implemented in conjunction.
6 is a diagram illustrating an example video decoder in which several described embodiments may be implemented in conjunction.
7 is a diagram illustrating an example of encoding a block of pixel values using a 1D dictionary mode.
8 is a diagram illustrating an example of decoding a block of pixel values by using a 1D dictionary mode.
9 is a flowchart of an exemplary method for decoding pixel values using dictionary mode.
10 is a flowchart of an exemplary method for decoding pixel values using a 1D dictionary mode.
11 is a flowchart of an exemplary method for encoding pixel values using dictionary mode.

The detailed description presents an innovation in the use of dictionary mode during encoding and/or decoding. In particular, the detailed description includes encoding and/or encoding digital video and/or image content (e.g., video content such as screen content) using a 1D dictionary mode, a pseudo 2D dictionary mode, and/or an inter pseudo 2D dictionary mode. Or suggest innovations for decoding. For example, previously encoded or decoded (e.g., in a video picture) stored in a dictionary (e.g., a 1D dictionary) or stored in another location (e.g., in a reconstructed picture) For example, to encode and/or decode pixel values in video content (e.g., in a video picture) based on reconstructed) pixel values, various 1D dictionary modes, pseudo 2D dictionary modes, and inter pseudo 2D Dictionary mode can be applied.

Techniques for improving the efficiency of encoding and/or decoding video and/or image data are described. In some innovations, the dictionary mode is used to encode and/or decode pixel values using previous pixel values stored in a dictionary or elsewhere (e.g., previously reconstructed or previously decoded pixel values). . In dictionary mode, the current pixel value can be predicted using a length representing the number of pixel values being predicted and an offset identifying a position within the previous pixel value (e.g., in the dictionary) (e.g. For example, it can be accurately predicted without requiring any residuals). Lossless prediction can be performed by accurately predicting pixel values from previous pixel values.

Some of these innovations improve the efficiency of encoding and/or decoding digital picture content (eg, image content and/or video content). For example, in order to reduce bits required to code digital picture content, a dictionary coding mode may be applied. In situations where screen content is being encoded and/or decoded, in order to reduce the number of bits and/or coding complexity required to code the content, various 1D dictionary coding modes, pseudo 2D dictionary coding modes, and inter pseudo 2D dictionary coding mode can be applied. In another innovation, the encoding of digital picture content can be done by computing the hash value of various groups of pixels (e.g. 1 pixel, 2 pixels, 4 pixels, 8 pixels, etc.), and the current pixel being encoded. It can be improved by matching hash values that identify matching pixel values to use to predict values (eg, to encode using the various dictionary modes described herein).

The techniques described herein can be applied to coding of screen content. Screen content refers to computer-generated video and/or image content (eg, text, graphics, and/or other computer-generated artificial content). One example of screen content is an image of a computer desktop graphical user interface, including text, icons, menus, windows, and/or other computer text and graphics. The techniques described herein may also be applied to content other than screen content (eg, other types of digital video and/or image content).

Although the operations described herein are described here and there as being performed by a video encoder or video decoder, in many cases, the operation is performed by other types of media processing tools (e.g., digital image or digital picture encoders, digital image or Digital picture decoder).

Some of the innovations described herein are illustrated with reference to syntax elements and operations specific to the HEVC standard. For example, reference is made to the draft version of the HEVC standard JCTVC-N1005-JCTVC-P1005 "High Efficiency Video Coding (HEVC) Range Extensions Text Specification: Draft 4" of July 2013. The innovations described herein may also be implemented for other standards or formats.

More generally, various alternatives to the examples described herein are possible. For example, some of the methods described herein may be modified by changing the order of the described method acts, by dividing, repeating, or omitting certain method acts, etc. . Various aspects of the disclosed technology may be used in combination or individually. Different embodiments use one or more of the described innovations. Some of the innovations described herein focus on one or more of the problems mentioned in the background. Typically, a given technique/tool does not solve all of these problems.

I. Example Computing System

1 illustrates a generalized example of a suitable computing system 100 in which some of the described innovations may be implemented. Computing system 100 is not intended to present any limitations as to the scope of use or functionality, as innovations may be implemented in a variety of general purpose or special purpose computing systems.

Referring to FIG. 1, computing system 100 includes one or more processing units 110 and 115 and memories 120 and 125. The processing units 110 and 115 execute computer-executable instructions. The processing unit may be a general purpose central processing unit ("CPU"), a processor in an application-specific integrated circuit ("ASIC"), or any other type of processor. In multiple processing systems, multiple processing units execute computer-executable instructions to increase processing power. For example, FIG. 1 shows a central processing unit 110 as well as a graphics processing unit or co-processing unit 115. The tangible memory 120, 125 is volatile memory (e.g., registers, cache, RAM), nonvolatile memory (e.g., ROM, EEPROM, flash memory), which can be accessed by the processing unit(s). Etc.), or some combination of the two. The memories 120 and 125 are executed by the processing unit(s) for software 180 implementing one or more innovations for 1D dictionary mode coding, pseudo 2D dictionary mode coding, and/or inter pseudo 2D dictionary mode coding. It stores in the form of computer-executable instructions suitable for.

The computing system may have additional features. For example, computing system 100 includes storage 140, one or more input devices 150, one or more output devices 160, and one or more communication connections 170. An interconnection mechanism (not shown), such as a bus, controller, or network, interconnects components of computing system 100. Typically, operating system software (not shown) provides an operating environment for other software running on computing system 100 and coordinates the activities of components of computing system 100.

The tangible storage 140 may be removable or non-removable, a magnetic disk, a magnetic tape or cassette, a CD-ROM, a DVD, or any other that can be used to store information and accessed within the computing system 100. Includes other media of. Storage 140 stores instructions for software 180 implementing one or more innovations for 1D dictionary mode coding, pseudo 2D dictionary mode coding, and/or inter pseudo 2D dictionary mode coding.

The input device(s) 150 may be a touch input device such as a keyboard, mouse, pen, or trackball, a voice input device, a scanning device, or other device that provides input to the computing system 100. In the case of video, the input device(s) 150 may be a camera, video card, TV tuner card, or similar device that accepts video input in analog or digital form, or a CD that reads video samples into computing system 100. It may be -ROM or CD-RW. Output device(s) 160 may be a display, printer, speaker, CD-writer, or other device that provides output from computing system 100.

The communication connection(s) 170 enable communication to other communication entities via a communication medium. Communication media carry information such as computer executable instructions, audio or video input or output, or other data in a modulated data signal. The modulated data signal refers to a signal in which one or more of its characteristics are set or changed in the same manner as encoding information in the signal. As a non-limiting example, the communication medium may use an electrical carrier, an optical carrier, an RF carrier, or other carrier.

Any of the disclosed innovations are stored on one or more computer readable storage media and executed on a computing device (eg, any available computing device, including a smartphone or other mobile device including computing hardware). It may be implemented as a computer executable instruction or a computer program product. Computer-readable storage media may be any available tangible media that can be accessed within a computing environment (e.g., one or more optical media disks such as DVDs or CDs, volatile memory components (eg DRAM or SRAM), or non- Is a volatile memory component (eg flash memory or hard drive). As an example and referring to FIG. 1, computer-readable storage media includes memories 1020 and 1025, and storage 1040. The term computer readable storage medium does not include signals and carrier waves. Further, the computer-readable storage medium does not include a communication connection (eg, 170).

The innovation can be described in the general context of computer-executable instructions, such as contained in program modules, that are executing on a target real or virtual processor in a computing system. Generally, program modules include routines, programs, libraries, objects, classes, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The functionality of the program modules may be divided or combined between program modules as desired in the various embodiments. Computer-executable instructions for program modules may be executed within a local computing system or a distributed computing system.

The terms “system” and “device” are used interchangeably herein. Unless the context clearly indicates otherwise, no term implies any limitation on the type of computing device or computing system. In general, a computing system or computing device may be local or distributed and may include any combination of special purpose hardware and/or general purpose hardware with software implementing the functionality described herein.

The disclosed methods may also be implemented using special computing hardware configured to perform any of the disclosed methods. For example, the disclosed method includes an integrated circuit (e.g., an ASIC (e.g. ASIC digital signal processor (“DSP”)), a graphics processing unit) specifically designed or configured to implement any of the disclosed methods. (graphics processing unit; “GPU”), or a programmable logic device (“PLD”), such as a field programmable gate array (“FPGA”)).

For reasons of presentation, the detailed description uses terms such as "determining" and "using" to describe computer operation in a computing system. These terms are high-level abstractions of actions performed by computers and should not be confused with actions performed by humans. Actual computer operation corresponding to these terms will depend on the implementation.

II. Example network environment

2A and 2B illustrate exemplary network environments 201 and 202 that include a video encoder 220 and a video decoder 270. The encoder 220 and decoder 270 are connected via network 250 using an appropriate communication protocol. Network 250 may include the Internet or other computer network.

In the network environment 201 shown in FIG. 2A, each real-time communication (“RTC”) tool 210 includes both an encoder 220 and a decoder 270 for bidirectional communication. A given encoder 220 can produce outputs that conform to the HEVC standard, the SMPTE 421M standard, the ISO-IEC 14496-10 standard (also known as H.264 or AVC), another standard, or a variant or extension of the proprietary format. However, the corresponding decoder 270 receives the encoded data from the encoder 220. Two-way communication may be part of a video conference, video telephony, or other bilateral communication scenario. Although the network environment 201 of FIG. 2A includes two real-time communication tools 210, the network environment 201 instead includes three or more real-time communication tools 210 that participate in multiparty communication. can do.

The real-time communication tool 210 manages the encoding by the encoder 220. 3 shows an exemplary encoder system 300 that may be included in a real-time communication tool 210. Alternatively, the real-time communication tool 210 uses another encoder system. The real-time communication tool 210 also manages decoding by the decoder 270. 4 shows an exemplary decoder system 400 that may be included in the real-time communication tool 210. Alternatively, the real-time communication tool 210 uses a different decoder system.

In the network environment 202 shown in FIG. 2B, the encoding tool 212 includes an encoder 220 that encodes the video for delivery to a number of playback tools 214 including a decoder 270. One-way communication may be provided for video surveillance systems, web camera monitoring systems, remote desktop conference presentations, or other scenarios in which video is encoded and transmitted from one location to one or more other locations. Although the network environment 202 of FIG. 2B includes two playback tools 214, the network environment 202 may include more or fewer playback tools 214. In general, playback tool 214 communicates with encoding tool 212 to determine the stream of video that playback tool 214 will receive. The playback tool 214 receives the stream, buffers the received encoded data for an appropriate period, and begins decoding and playback.

3 shows an exemplary encoder system 300 that may be included in the encoding tool 212. Alternatively, the encoding tool 212 uses a different encoder system. The encoding tool 212 may also include server-side controller logic to manage connections with one or more playback tools 214. 4 shows an exemplary decoder system 400 that may be included in the playback tool 214. Alternatively, the playback tool 214 uses a different decoder system. The playback tool 214 may also include client side controller logic to manage the connection with the encoding tool 212.

III. Exemplary Encoder System

3 is a block diagram of an exemplary encoder system 300 in which some described embodiments may be implemented in conjunction. The encoder system 300 can operate in any of a number of encoding modes, such as a low-latency encoding mode for real-time communication, a transcoding mode, and a general encoding mode for media playback from a file or stream. There may be a general purpose encoding tool, or the encoder system 300 may be a special purpose encoding tool adapted for one such encoding mode. The encoder system 300 may be implemented as an operating system module, as part of an application library, or as a standalone application. Taken together, encoder system 300 receives a sequence of source video frames 311 from video source 310 and produces encoded data as output to channel 390. The encoded data output through the channel may include content encoded using a 1D dictionary mode, a pseudo 2D dictionary mode, and/or an inter pseudo 2D dictionary mode.

The video source 310 may be a camera, tuner card, storage medium, or other digital video source. Video source 310 generates a sequence of video frames at a frame rate of 30 frames per second, for example. As used herein, the term “frame” generally refers to source, coded or reconstructed image data. In the case of progressive video, the frame is a progressive video frame. In the case of interlaced video, in an exemplary embodiment, the interlaced video frames are de-interlaced prior to encoding. Alternatively, two complementary interlaced video fields are encoded as interlaced video frames or separate fields. In addition to denoting a progressive video frame, the term "frame" or "picture" refers to a single non-paired video field, a complementary pair of video fields, at a given time. It may represent a video object plane representing a video object of, or an area of interest in a larger image. The video object plane or region may be part of a larger image comprising multiple objects or regions of a scene.

The arriving source frame 311 is stored in the source frame temporary memory storage area 320 including a plurality of frame buffer storage areas 321, 322, ..., 32n. The frame buffers 321, 322, etc. maintain one source frame in the source frame storage area 320. After one or more of the source frames 311 are stored in the frame buffers 321, 322, etc., the frame selector 330 selects individual source frames from the source frame storage area 320. The order in which frames are selected by the frame selector 330 for input to the encoder 340 may be different from the order in which frames are generated by the video source 310, for example, temporally reverse prediction. To facilitate this, some frames may precede in order. Before the encoder 340, the encoder system 300 is a pre-processor (not shown) that performs pre-processing (e.g., filtering) of the selected frame 331 before encoding. May include). The preprocessing may also include color space conversion to a primary component and a secondary component for encoding. Typically, before encoding, the video has been converted into a color space such as YUV, in which case the sample value of the luma (Y) component represents the luminance or intensity value, and the sample value of the chroma (U, V) component Represents the color difference value. The chroma sample values may be subsampled at a lower chroma sampling rate (e.g., for YUV 4:2:0 format), or the chroma sample values may be (e.g., for YUV 4:4:4 format). ) It may have the same resolution as the luma sample value. Alternatively, the video may be encoded in another format (eg, RGB 4:4:4 format).

The encoder 340 encodes the selected frame 331 and also generates a coded frame 341, a memory management control operation ("MMCO") signal 342 or a reference picture set. picture set; "RPS") information is created. If the current frame is not the first encoded frame and is performing its own encoding process, the encoder 340 may perform one or more previously encoded/decoded frames stored in the decoded frame temporary memory storage area 360. A frame 369 can also be used. The decoded frame 369 thus stored is used as a reference frame for inter-frame prediction of the content of the current source frame 331. In general, the encoder 340 includes a number of encoding modules that perform encoding tasks such as partitioning into tiles, intra prediction estimation and prediction, motion estimation and compensation, frequency transformation, quantization, and entropy coding. The exact operation performed by encoder 340 may vary depending on the compression format. The format of the output encoded data is HEVC format, Windows Media Video format, VC-1 format, MPEG-x format (eg, MPEG-1, MPEG-2, or MPEG-4), It may be an H.26x format (eg, H.261, H.262, H.263, H.264), or a variant or extension of another format.

The encoder 340 may divide the frame into multiple tiles of the same size or different sizes. For example, the encoder 340 divides the frame along tile rows and tile columns that define horizontal and vertical boundaries of tiles within the frame, along with frame boundaries, in which case each The tile is a rectangular area. Tiles are often used to enhance options for parallel processing. A frame may also be organized as one or more slices, in which case the slice may be an entire frame or a region of a frame. A slice can be decoded irrespective of other slices in a frame, and the fact that a slice is decoded irrespective of other slices in a frame improves error resilience. The content of a slice or tile is further partitioned into blocks or different sets of sample values for purposes of encoding and decoding.

In the case of syntax according to the HEVC standard, the encoder divides the content of a frame (or slice or tile) into a coding tree unit. A coding tree unit ("CTU") contains luma sample values organized into a luma coding tree block ("CTB") and corresponding chroma sample values organized into two chroma CTBs. The size of the CTU (and the CTB of the CTU) is selected by the encoder, and may be, for example, 64x64, 32x32 or 16x16 sample values. CTU contains one or more coding units. A coding unit (“CU”) has a luma coding block (“CB”) and two corresponding chroma CBs. For example, a CTU with a 64x64 luma CTB and two 64x64 chroma CTBs (YUV 4:4:4 format) can be divided into four CUs, each of which has a 32x32 luma CB and two It contains two 32x32 chroma CBs, and each CU is probably further divided into smaller CUs. Alternatively, as another example, a CTU (YUV 4:2:0 format) having a 64×64 luma CTB and two 32×32 chroma CTBs may be divided into four CUs, each CU being a 32×32 luma CB And two 16×16 chroma CBs, and each CU is possibly further divided into smaller CUs. The smallest allowable size of the CU (eg, 8×8, 16×16) can be signaled in the bitstream.

In general, the CU has a prediction mode such as inter or intra. The CU contains one or more prediction units for the purpose of signaling prediction information (eg prediction mode details, etc.) and/or prediction processing. The prediction unit (“PU”) includes a luma prediction block (“PB”) and two chroma PBs. In the case of an intra-predicted CU, the PU has the same size as the CU, unless the CU has a minimum size (eg, 8×8). In that case, the CU can be divided into four smaller PUs (e.g., 4x4 each if the minimum CU size is 8x8), or the PU is the minimum, as indicated by the syntax element for the CU. It can have a CU size. The CU also has one or more transform units for the purpose of residual coding/decoding, in which case a transform unit ("TU") has a transform block ("TB") and two chroma TBs. A PU in an intra-predicted CU may include a single TU (same as a PU in size) or multiple TUs. As used herein, the term “block” can refer to CU, CB, PB, TB or other sets of sample values, depending on the context. The encoder determines how to partition the video into CTU, CU, PU, TU, and so on.

Referring to FIG. 3, the encoder represents the intra-coded block of the source frame 331 in terms of prediction from other previously reconstructed sample values in the frame 331. In the case of intra spatial prediction for a block, the intra picture estimator estimates an extrapolation of neighboring reconstructed sample values into a block. The intra picture estimator outputs prediction information (eg, a prediction mode (direction) for intra spatial prediction), and the prediction information is entropy-coded. An intra-prediction predictor applies prediction information to determine an intra prediction value.

For the various dictionary coding modes described herein, the encoder retrieves a hash value of a previously reconstructed sample value (e.g., a grouping of 1 pixel, 2 pixels, 4 pixels, 8 pixels, and so on). You can compute and compare these hash values against the hash value of the current pixel value being encoded. Matching of the length can be identified in one or more previously reconstructed sample values, based on a hash comparison, and the current pixel value (or values) is the various 1D dictionary modes and pseudo 2D dictionary modes described herein (or reference picture Can be encoded using an inter pseudo 2D dictionary mode).

The encoder 340 represents an inter-frame coded and predicted block of the source frame 331 in terms of prediction from a reference frame. The motion estimator estimates the motion of a block for one or more reference frames 369. If multiple reference frames are used, the multiple reference frames may originate from different temporal directions or from the same temporal direction. The motion compensated prediction reference region is the region of the sample in the reference frame(s), which is used to generate a motion compensated prediction value for a block of samples of the current frame. The motion estimator outputs motion information such as motion vector information, and the motion information is entropy-coded. The motion compensator applies the motion vector to the reference frame 369 to determine the motion compensated prediction value.

The entropy coder of the encoder 340 compresses not only quantized transform coefficient values but also predetermined additional information (eg, motion vector information, QP value, mode determination, parameter selection). In particular, the entropy coder may compress data for an element of the index map using a coefficient coding syntax structure. Typical entropy coding techniques include Exp-Golomb coding, arithmetic coding, Huffman coding, run length coding, and variable-length- to-variable-length; "V2V") coding, variable-length-to-fixed-length ("V2F") coding, LZ coding, dictionary coding, probability interval partitioning entropy coding ( probability interval partitioning entropy coding ("PIPE"), and combinations of the above. Entropy coders can use different coding techniques for different kinds of information, and can choose from multiple code tables within a particular coding technique.

The coded frame 341 and MMCO/RPS information 342 are processed by the decoding process emulator 350. The decoding process emulator 350 implements some of the decoder's functionality, for example a decoding task to reconstruct a reference frame. The decoding process emulator 350, in order to determine whether a given coded frame 341 needs to be reconstructed and stored for use as a reference frame in inter-frame prediction of a subsequent frame to be encoded, MMCO/RPS information ( 342) is used. If the MMCO/RPS information 342 indicates that the coded frame 341 needs to be stored, the decoding process emulator 350 receives the coded frame 341 and generates the corresponding decoded frame 351. Model the decoding process to be done by the generating decoder. In doing so, when the encoder 340 uses the decoded frame(s) 369 stored in the decoded frame storage area 360, the decoding process emulator 350 also stores, as part of the decoding process. The decoded frame(s) 369 from region 360 are used.

The decoded frame temporary memory storage area 360 includes a plurality of frame buffer storage areas 361, 362, ..., 36n. The decoding process emulator 350 is configured to identify any frame buffers 361, 362, etc., which have frames that are no longer needed by the encoder 340 for use as reference frames. The MMCO/RPS information 342 is used to manage the contents of the. After modeling the decoding process, the decoding process emulator 350 stores the newly decoded frame 351 in the frame buffers 361, 362, etc., identified in this manner.

The coded frame 341 and MMCO/RPS information 342 are buffered in a temporary coded data area 370. The coded data aggregated in the coded data area 370, as part of the syntax of the basic coded video bitstream, includes encoded data for one or more pictures. The coded data aggregated in the coded data area 370 also includes media metadata related to the coded video data (eg, one or more supplemental enhancement information (“SEI”) messages or Video usability information ("VUI") as one or more parameters in the message).

The aggregated data 371 from the temporary coded data area 370 is processed by the channel encoder 380. The channel encoder 380 may packetize the aggregated data (eg, according to a media stream multiplexing format such as ISO/IEC 13818-1) for transmission as a media stream. In this case, the channel encoder 380 ) May add a syntax element as part of the syntax of the media transport stream. Alternatively, the channel encoder 380 may organize the aggregated data (eg, according to a media container format such as ISO/IEC 14496-12) for storage as a file. In this case, the channel encoder 380 ) Can add a syntax element as part of the syntax of the media storage file. Or, more generally, the channel encoder 380 may implement one or more media system multiplexing protocols or transport protocols, in which case the channel encoder 380 may add a syntax element as part of the syntax of the protocol(s). I can. Channel encoder 380 provides the output to channel 390, which represents another channel, storage, or communication connection to the output.

IV. Exemplary decoder system

4 is a block diagram of an exemplary decoder system 400 in which some described embodiments may be implemented in conjunction. The decoder system 400 may be a general purpose decoding tool capable of operating in any of a number of decoding modes, such as a low latency decoding mode for real-time communication and a typical decoding mode for media playback from a file or stream, or 400 may be a special purpose decoding tool adapted for one such decoding mode. The decoder system 400 may be implemented as an operating system module, as part of an application library, or as a standalone application. Taken together, the decoder system 400 receives coded data from channel 410 and generates a reconstructed frame as an output to an output destination 490. The coded data may include content encoded using a 1D dictionary mode, a pseudo 2D dictionary mode, and/or an inter pseudo 2D dictionary mode.

The decoder system 400 includes a channel 410, which may represent another channel, storage, or communication connection for coded data as an input. Channel 410 generates channel-coded coded data. The channel decoder 420 may process coded data. For example, the channel decoder 420 de-packetizes data that has been aggregated for transmission as a media stream (eg, according to a media stream multiplexing format such as ISO/IEC 13818-1). In this case, the channel decoder 420 may parse the syntax element added as part of the syntax of the media transmission stream. Alternatively, the channel decoder 420 separates the coded video data that has been aggregated for storage as a file (eg, according to a media container format such as ISO/IEC 14496-12). In this case, the channel decoder 420 ) Can parse the syntax element added as part of the syntax of the media storage file. Or, more generally, the channel decoder 420 may implement one or more media system demultiplexing protocols or transport protocols, in which case the channel decoder 420 is a syntax element added as part of the syntax of the protocol(s). Can be parsed.

The coded data 421 output from the channel decoder 420 is stored in the temporary coded data area 430 until a sufficient amount of such data is received. The coded data 421 includes a coded frame 431 and MMCO/RPS information 432. The coded data 421 in the coded data area 430, as part of the syntax of the basic coded video bitstream, includes coded data for one or more pictures. The coded data 421 of the coded data area 430 may also contain media metadata related to the encoded video data (e.g., as one or more parameters in one or more SEI messages or VUI messages). I can.

In general, coded data area 430 temporarily stores coded data 421 until such coded data 421 is used by decoder 450. At that time, the MMCO/RPS information 432 and coded data for the coded frame 431 are transmitted from the coded data area 430 to the decoder 450. As the decoding proceeds, newly coded data is added to the coded data area 430 and the oldest coded data remaining in the coded data area 430 is transmitted to the decoder 450.

The decoder 450 periodically decodes the coded frame 431 to generate a corresponding decoded frame 451. If appropriate, when performing its own decoding process, the decoder 450 may use one or more previously decoded frames 469 as reference frames for inter-frame prediction. The decoder 450 reads this previously decoded frame 469 from the decoded frame temporary memory storage area 460. In general, the decoder 450 includes a plurality of decoding modules that perform decoding tasks such as entropy decoding of tiles, inverse quantization, inverse frequency transform, intra prediction, motion compensation, and merging. The exact operation performed by the decoder 450 may vary depending on the compression format.

For example, the decoder 450 receives encoded data for a compressed frame or sequence of frames and generates an output comprising the decoded frame 451. At decoder 450, the buffer receives the encoded data for the compressed frame, and at the appropriate time, makes the received encoded data available to the entropy decoder. The entropy decoder entropy decodes entropy-coded quantized data as well as entropy-coded side information, and typically applies the inverse of entropy encoding performed by the encoder. The motion compensator applies motion information to one or more reference frames to form a motion compensated prediction value for any inter-coded block of the frame being reconstructed. The intra prediction module may spatially predict a sample value of a current block from a neighboring previously reconstructed sample value.

For the various dictionary coding modes described herein, the decoder can decode the current pixel value in the matching mode and/or the direct mode. In matching mode, the decoder is the current pixel predicted from a previously decoded picture value (e.g., a previously reconstructed pixel value), which may be stored in a 1D dictionary or another location (e.g., reconstructed picture). Decode the value. For example, the decoder may receive one or more codes indicating an offset (eg, within a dictionary) and a length (indicating the number of pixel values to be predicted from the offset). In direct mode, the decoder can directly decode pixel values without prediction.

In the non-dictionary mode, the decoder 450 also reconstructs the prediction residuals. The inverse quantizer inverse quantizes the entropy-decoded data. For example, the decoder 450 sets a value for QP for a picture, tile, slice, and/or other portion of the video based on the syntax element of the bitstream, and inverse quantizes the transform coefficient accordingly. The inverse frequency converter converts the quantized frequency domain data into spatial domain information. In the case of an inter-predicted block, the decoder 450 combines the reconstructed prediction residual with the motion compensated prediction value. Likewise, the decoder 450 may combine the prediction residual values with the predictions from intra prediction. The motion compensation loop in video decoder 450 includes an adaptive deblocking filter to smooth discontinuities across block boundary rows and/or columns of decoded frame 451.

The decoded frame temporary memory storage area 460 includes a plurality of frame buffer storage areas 461, 462, ..., 46n. The decoded frame storage area 460 is an example of a decoded picture buffer. The decoder 450 uses the MMCO/RPS information 432 to identify the frame buffers 461, 462, etc. that may store the decoded frame 451 therein. The decoder 450 stores the decoded frame 451 in its frame buffer.

The output sequencer 480 uses the MMCO/RPS information 432 to identify when the next frame to be generated in the output sequence is available in the decoded frame storage area 460. When the next frame 481 to be generated in the output sequence is available in the decoded frame storage area 460, the next frame 481 is read by the output sequencer 480 and output destination 490 (e.g., display ) Is displayed. In general, the order in which frames are output from the decoded frame storage area 460 by the output sequencer 480 may be different from the order in which the frames are decoded by the decoder 450.

V. Exemplary Video Encoder

5A and 5B are block diagrams of a generalized video encoder 500 in which some described embodiments may be implemented in conjunction. Encoder 500 receives a sequence of video pictures containing the current picture as input video signal 505 and generates encoded data as output in coded video bitstream 595.

The encoder 500 is block based and uses a block format that is implementation dependent. Blocks may be further sub-divided in different stages, for example in prediction, frequency transform and/or entropy encoding stages. For example, a picture may be divided into 64×64 blocks, 32×32 blocks or 16×16 blocks, which may eventually be divided into smaller blocks of sample values for coding and decoding. In an implementation of encoding for the HEVC standard, the encoder partitions the picture into CTU (CTB), CU (CB), PU (PB) and TU (TB).

The encoder 500 compresses a picture using intra picture coding and/or inter picture coding. Many of the components of encoder 500 are used for both intra picture coding and inter picture coding. The exact operation performed by these components can vary depending on the type of information being compressed.

The tiling module 510 optionally divides the picture into a plurality of tiles of the same size or of different sizes. For example, the tiling module 510 divides a picture along tile rows and tile columns that define horizontal and vertical boundaries of tiles within the picture, along with the picture boundary. In this case, each tile is a rectangular area. to be. The tiling module 510 may then group the tiles into one or more tile sets, in which case the tile set is one or more groups of tiles.

The general encoding control unit 520 receives not only a picture for the input video signal 505 but also feedback (not shown) from various modules of the encoder 500. In summary, the general encoding control unit 520 converts a control signal (not shown) to another module (e.g., a tiling module 510, a converter/scaler/quantizer 530) in order to set and change a coding parameter during encoding. , A scaler/inverse transformer 535, an intra picture estimator 540, a motion estimator 550, and an intra/inter switch). In particular, the general encoding control unit 520 may determine whether and how to use the dictionary mode during encoding. The general encoding control unit 520 may also evaluate intermediate results during encoding, for example, while performing rate distortion analysis. The general encoding control unit 520 generates general control data 522 representing decisions made during encoding, and as a result, the corresponding decoder can make a consistent decision. General control data 522 is provided by a header formatter/entropy coder 590.

When the current picture is predicted using inter picture prediction, the motion estimator 550 estimates a motion of a block of sample values of the current picture of the input video signal 505 based on one or more reference pictures. The decoded picture buffer 570 buffers one or more reconstructed previously coded pictures for use as a reference picture. When multiple reference pictures are used, the multiple reference pictures may originate from different temporal directions or from the same temporal direction. The motion estimator 550 generates motion data 552, such as motion vector data and reference picture selection data, as additional information. Motion data 552 is provided to a header formatter/entropy coder 590 as well as a motion compensator 555.

Motion compensator 555 applies the motion vector to the reconstructed reference picture(s) from decoded picture buffer 570. The motion compensator 555 generates a motion compensated prediction value for the current picture.

In a separate path within the encoder 500, the intra picture estimator 540 determines a method of performing intra picture prediction on a block of sample values of the current picture of the input video signal 505. The current picture may be wholly or partially coded using intra picture coding. In the case of intra spatial prediction using the value of the reconstruction 538 of the current picture, the intra picture estimator 540 spatially predicts the sample value of the current block of the current picture from the previously reconstructed sample value adjacent to the current picture. Decide how to do it.

For the various dictionary coding modes described herein, the encoder 500 is a grouping of previously reconstructed sample values (e.g., 1 pixel, 2 pixels, 4 pixels, 8 pixels, and so on). Hash values can be computed and these hash values can be compared against the hash value of the current pixel value being encoded. Matching of the length can be identified in one or more previously reconstructed sample values, based on a hash comparison, and the current pixel value (or values) is the various 1D dictionary modes and pseudo 2D dictionary modes described herein (or reference picture Can be encoded using an inter pseudo 2D dictionary mode).

The intra prediction estimator 540 includes information indicating whether intra prediction data 542, such as whether intra prediction uses spatial prediction or one of various dictionary modes (eg, per intra block or predetermined prediction A flag value per intra block in the mode direction) and a prediction mode direction (for intra spatial prediction) are generated as additional information. The intra prediction data 542 is provided not only to the header formatter/entropy coder 590 but also to the intra picture predictor 545. According to the intra prediction data 542, the intra picture predictor 545 spatially predicts the sample value of the current block of the current picture from the neighboring previously reconstructed sample values of the current picture.

In non-dictionary mode, the intra/inter switch selects a value of motion compensated prediction or intra picture prediction for use as a prediction value 558 for a given block. In non-dictionary mode, the difference between a block of prediction 558 and a corresponding portion of the original current picture of the input video signal 505 provides the value of the residual 518 (if any). During reconstruction of the current picture, the reconstructed residual value is combined with the predicted value 558 to produce a reconstructed value 538 of the original content from the video signal 505. In lossy compression, some information is still lost from the video signal 505.

In the converter/scaler/quantizer 530, in the case of a non-dictionary mode, the frequency transform converts spatial domain video information into frequency domain (ie, spectrum, transform) data. In the case of block-based video coding, the frequency converter, for a block of prediction residual data (or sample value data when prediction 558 is null), a discrete cosine transform (“DCT”), By applying the integer approximation or other type of forward block transform, a block of frequency transform coefficients is generated. The encoder 500 may also indicate that this transform step is skipped. The scaler/quantizer scales and quantizes the transform coefficients. For example, the quantizer applies non-uniform scalar quantization to frequency domain data with a quantization step size that varies on a frame-by-frame basis, on a tile basis, on a slice basis, on a block basis, or on another basis. The quantized transform coefficient data 532 is provided to a header formatter/entropy coder 590.

In the scaler/inverse transformer 535, in the case of a non-dictionary mode, the scaler/inverse quantizer performs inverse scaling and inverse quantization on the quantized transform coefficients. The inverse frequency converter performs inverse frequency transformation to generate a reconstructed prediction residual value or a block of sample values. The encoder 500 combines the reconstructed residual value with a value of the predicted value 558 (eg, a motion compensated predicted value, an intra picture predicted value) to form a reconstructed value 538.

In the case of intra picture prediction, the value of the reconstruction 538 may be supplied back to the intra picture estimator 540 and the intra picture predictor 545. Further, the value of reconstruction 538 may be used for motion compensated prediction of a subsequent picture. The value of reconstruction 538 may be further filtered. The filtering control unit 560 determines a method of performing deblocking filtering and sample adaptive offset (“SAO”) filtering on a value of the reconstruction 538 for a given picture of the video signal 505. . The filtering control unit 560 generates filter control data 562, which is provided to a header formatter/entropy coder 590 and a merger/filter(s) 565.

In merger/filter(s) 565, encoder 500 merges content from different tiles into a reconstructed version of the picture. The encoder 500 selectively performs deblocking filtering and SAO filtering according to the filter control data 562 in order to adaptively smooth a discontinuity across a boundary in a frame. The tile boundary may be selectively filtered or not filtered at all, depending on the settings of the encoder 500, and the encoder 500 may or may not have coded bits to indicate whether such filtering has been applied or not. You can also provide syntax in the stream. The decoded picture buffer 570 buffers the reconstructed current picture for use in subsequent motion compensated prediction.

The header formatter/entropy coder 590 includes general control data 522, quantized transform coefficient data 532, intra prediction data 542 and packetized index values, motion data 552, and filter control data ( 562) and/or entropy coding. For example, the header formatter/entropy coder 590 uses context-adaptive binary arithmetic coding ("CABAC") for entropy coding of various syntax elements of a coefficient coding syntax structure.

The header formatter/entropy coder 590 provides encoded data in the coded video bitstream 595. The format of the coded video bitstream 595 is HEVC format, window media video format, VC-1 format, MPEG-x format (e.g., MPEG-1, MPEG-2, or MPEG-4), H. It may be a 26x format (eg, H.261, H.262, H.263, H.264), or a variant or extension of another format.

Depending on the type and implementation of compression desired, modules of the encoder can be added, omitted, divided into multiple modules, combined with other modules, and/or replaced by similar modules. have. In alternative embodiments, encoders with different modules and/or different configurations of modules perform one or more of the described techniques. Certain embodiments of the encoder typically use a variant or supplemented version of the encoder 500. The relationships shown between the modules within the encoder 500 represent the general flow of information in the encoder; Other relationships are not shown for the sake of brevity.

VI. Exemplary video decoder

6 is a block diagram of a generalized decoder 600 in which some described embodiments may be implemented in conjunction. The decoder 600 receives the encoded data in the coded video bitstream 605 and produces an output containing the picture for the reconstructed video 695. The format of the coded video bitstream 605 is HEVC format, window media video format, VC-1 format, MPEG-x format (eg, MPEG-1, MPEG-2, or MPEG-4), H. It may be a 26x format (eg, H.261, H.262, H.263, H.264), or a variant or extension of another format.

The decoder 600 is block based and uses a block format depending on the implementation. Blocks may be further subdivided in different stages. For example, a picture may be divided into 64×64 blocks, 32×32 blocks or 16×16 blocks, which may eventually be divided into smaller blocks of sample values. In an implementation of decoding for the HEVC standard, the picture is partitioned into CTU (CTB), CU (CB), PU (PB) and TU (TB).

The decoder 600 decompresses a picture using intra picture decoding and/or inter picture decoding. Many of the components of decoder 600 are used for both intra picture decoding and inter picture decoding. The exact operation performed by these components can vary depending on the type of information being decompressed.

The buffer receives the encoded data in the coded video bitstream 605 and makes the received encoded data available to the parser/entropy decoder 610. The parser/entropy decoder 610 entropy decodes the entropy-coded data, and typically applies the inverse of the entropy coding performed by the encoder 500 (eg, context adaptive binary arithmetic decoding). For example, the parser/entropy decoder 610 uses context adaptive binary arithmetic decoding for entropy decoding of various syntax elements of a coefficient coding syntax structure. As a result of parsing and entropy decoding, the parser/entropy decoder 610 is configured with general control data 622, quantized transform coefficient data 632, intra prediction data 642 and packetized index values, motion data 652 ) And filter control data 662 are generated.

The general decoding control unit 620 receives the general control data 622 and converts a control signal (not shown) to another module (e.g., scaler/inverse converter 635, intra) in order to set and change the decoding parameter during decoding. A picture predictor 645, a motion compensator 655, and an intra/inter switch) are provided.

If the current picture is predicted using inter picture prediction, the motion compensator 655 receives motion data 652, such as motion vector data and reference picture selection data. Motion compensator 655 applies the motion vector to the reconstructed reference picture(s) from decoded picture buffer 670. The motion compensator 655 generates a motion compensated prediction value for the inter-coded block of the current picture. The decoded picture buffer 670 stores one or more previously reconstructed pictures for use as a reference picture.

In a separate path within decoder 600, intra prediction predictor 645 provides intra prediction data 642, such as information indicating whether intra prediction uses spatial prediction or one of dictionary modes (e.g. , A flag value per intra block or per intra block in a predetermined prediction mode direction), and a prediction mode direction (for intra spatial prediction) are received. In the case of intra-spatial prediction, using the value of the reconstruction 638 of the current picture, according to the prediction mode data, the intra picture predictor 645, from the neighboring previously reconstructed sample values of the current picture, the current picture Spatially predict the sample value of the block.

For the various dictionary coding modes described herein, the decoder can decode the current pixel value in the matching mode and/or the direct mode. In matching mode, the decoder is the current pixel predicted from a previously decoded picture value (e.g., a previously reconstructed pixel value), which may be stored in a 1D dictionary or another location (e.g., reconstructed picture). Decode the value. For example, the decoder may receive one or more codes indicating an offset (eg, within a dictionary) and a length (indicating the number of pixel values to be predicted from the offset). In direct mode, the decoder can directly decode pixel values without prediction.

In non-dictionary mode, the intra/inter switch selects a value of motion compensated prediction or intra picture prediction for use as a prediction value 658 for a given block. For example, if HEVC syntax is followed, the intra/inter switch may be controlled based on a syntax element encoded for a CU of a picture that may include an intra predicted CU and an inter predicted CU. The decoder 600 combines the predicted value 658 with the reconstructed residual value to generate a reconstructed value 638 of the content from the video signal.

In order to reconstruct the residuals, in the case of a non-dictionary mode, the scaler/inverse transformer 635 receives and processes the quantized transform coefficient data 632. In the scaler/inverse transformer 635, the scaler/inverse quantizer performs inverse scaling and inverse quantization on the quantized transform coefficients. The inverse frequency converter performs inverse frequency transformation to generate a reconstructed prediction residual value or a block of sample values. For example, the inverse frequency converter applies an inverse block transform to a frequency transform coefficient to generate sample value data or prediction residual data. The inverse frequency transformation may be an inverse DCT, an integer approximation of an inverse DCT, or another type of inverse frequency transformation.

In the case of intra picture prediction, the value of the reconstruction 638 may be supplied back to the intra picture predictor 645. In the case of inter-picture prediction, the value of the reconstruction 638 may be additionally filtered. In merger/filter(s) 665, decoder 600 merges the content from different tiles into a reconstructed version of the picture. The decoder 600 selectively performs deblocking filtering and SAO filtering according to the filter control data 662 and a rule for filter adaptation in order to adaptively smooth a discontinuity across a boundary in a frame. The tile boundary may be selectively filtered or may not be filtered at all, depending on the setting of the decoder 600 or the syntax indication in the encoded bitstream data. The decoded picture buffer 670 buffers the reconstructed current picture for use in subsequent motion compensated prediction.

The decoder 600 may also include a post-processing deblock filter. The post-processing deblocking filter optionally smoothes the discontinuities in the reconstructed picture. As part of post-processing filtering, other filtering (eg de-ring filtering) can also be applied.

Depending on the type and implementation of decompression desired, modules of the decoder can be added, omitted, divided into multiple modules, combined with other modules and/or replaced by similar modules. I can. In alternative embodiments, decoders having different modules and/or different configurations of modules perform one or more of the described techniques. Certain embodiments of the decoder typically use a variant or supplemented version of the decoder 600. The relationship shown between the modules within the decoder 600 represents the general flow of information in the decoder; Other relationships are not shown for the sake of brevity.

VII. 1D Dictionary Mode Korea Innovation Plan

This section presents various innovations for the one-dimensional (1D) dictionary mode. Some innovations relate to signaling pixel values using offset and length, while others relate to signaling pixel values directly. Another innovation concerns vertical and horizontal scanning.

In particular, using the 1D dictionary mode when encoding pixel values can improve performance and reduce the bits required when encoding video content, particularly screen content (e.g., when performing screen capture). I can. Screen content typically includes repeated structures (e.g., graphics, text characters), and the repeated characters improve performance by providing areas with the same pixel values that can be encoded through prediction.

A. 1D Dictionary mode - Introduction

In 1D dictionary mode, sample values (e.g. pixel values) are predicted with reference to previous sample values (e.g., previously reconstructed sample values) stored in the 1D dictionary (using offset and length). do. For example, a video encoder or image encoder may refer to a 1D dictionary that stores previous sample values (e.g., reconstructed or original sample values) used to predict and encode the current sample values. Can be encoded. The video decoder or image decoder may decode the current sample value with reference to a 1D dictionary that stores previously decoded (eg, reconstructed) sample values used to predict and decode the current sample value.

In the 1D dictionary mode, one or more current pixel values may be predicted from one or more previous pixel values (eg, in scan order). Prediction can be performed by matching the current pixel value with a previous pixel value so that the current pixel value can be accurately predicted (eg, without requiring any residuals). The term “matching mode” describes encoding and/or decoding using matching pixel values from a dictionary (or from another source, e.g. a reconstructed picture) (or in situations in which no matching pixel values are present (e.g., If no match is found at the beginning of the frame or in the dictionary of previous pixel values), one or more current pixel values may be directly coded The term “direct mode” directly encodes and/or decodes pixel values. Explain that.

In some implementations, pixel values are encoded and decoded as combined pixels (a combination of Y, U, and V for a pixel, or a combination of R, G, and B for a pixel are encoded/decoded together). In other implementations, pixel values are encoded and decoded as separate components (e.g., separate 1D dictionaries may be maintained for each of the Y, U, and V components or R, G, and B components). . Pixel values can be in various YUV data formats (e.g., YUV 4:4:4, YUV 4:2:2, YUV 4:2:0, etc.) or in various RGB data formats (e.g. RGB, GBR, BGR, etc.) and decoded.

Encoding and/or decoding of pixel values using the 1D dictionary mode may be applied to a separate area, such as video or image content divided into blocks. In general, blocks of any size can be used. In some implementations, video content (eg, a video picture or frame) is divided into coding units having a size of 64×64, 32×32, 16×16, or 8×8 sample values.

In some implementations, dictionary coding can be combined with other types of coding. For example, pixel values may be coded using one of the dictionary modes described herein (eg, 1D dictionary mode). The coded pixel values can then be coded using other coding techniques (eg, context based arithmetic coding or other coding techniques).

B. Offset and length Signaling

In the 1D dictionary mode, when a matching pixel value exists, an offset value and a length value are signaled to indicate a position in the 1D dictionary where the matching pixel value to predict the current pixel value is located. For example, one or more current pixel values may be stored in a 1D dictionary identified within the 1D dictionary by offset (position in the 1D dictionary from the current pixel value in the reverse direction) and length (number of pixel values predicted from the offset). It can be predicted from the previous pixel values. As should be understood, an offset of 5 means five pixels back in the 1D dictionary from the current pixel value (e.g., in some implementations, a negative sign is added, which in this example Will be an offset of -5).

In the 1D dictionary mode, in some implementations, the pixel values of the current block can be predicted from the pixel values of the previous block (eg, depending on the maximum size of the dictionary). For example, in a picture coded using a 64×64 block, the pixel values from the fourth block of the picture are from the pixel values from the first block of the picture stored in the 1D dictionary (e.g., offset and Can be predicted using length).

The offset may be signaled (eg, in a bit stream) encoded in a format that divides the possible offset values into multiple ranges and encodes the offset values by range. In this way, the offset may be encoded as a two-part code with a first part identifying the offset range and a second part indicating an offset value within that range.

In certain implementations, the offset values are coded using the following ranges. Also, in this embodiment, zero-based numbering is applied so that the offset value is decreased by 1 before the offset value is encoded and the offset value is increased by 1 after the offset value is decoded. . The range (with its own offset range code), the corresponding offset value, and the number of bits are represented by the following table (Table 1).

Using the implementation depicted in Table 1 above, the offset can be encoded, signaled, and decoded. As an example, an offset value of 415 (representing the original offset value of 416, decremented by 1 for encoding) would be encoded in range 4. Since range 4 starts with an offset value of 276, the value to be coded will be 415-276 = 139. The encoded offset will be generated by combining the offset range code of "0001" (representing range 4) followed by "0000000010001011" of the 16-bit value (16-bit binary value for decimal 139). Putting the two parts of the code together (offset range code and offset value code) is represented by the following combined code for the encoded offset: "00010000000010001011". As another example, an offset value of 45 (representing the original offset value of 46 that is reduced by 1 for encoding) would be encoded in range 3. Since range 3 starts with an offset value of 20, the value to be coded will be 45-20 = 25. The encoded offset will be generated by combining an offset range code of "001" (representing range 3) followed by an 8-bit value of "00011001" (an 8-bit binary value for decimal 25). Putting the two parts of the code together (offset range code and offset value code) is represented by the following combined code for the encoded offset: "00100011001".

As depicted in Table 1 above, range 5 uses N bits to represent an offset value greater than 65,811, where N represents the number of bits required to represent the maximum offset. In some implementations, the maximum offset value is determined from the current dictionary size. For example, if the current dictionary size is 300,000, N can be set to 18 (that is, 18 bits are required to represent the maximum offset value of 300,000), so an offset value between 65,811 and 300,000 is the offset value. We will use 18 bits to encode the value. The offset value for range 5 starts at 65,812, so to represent 300,000, only 18 bits are needed to represent an amount greater than 65,811 (i.e. 300,000-65,812 = only 18 bits to represent 234,188). It should be understood that only is needed). In other implementations, the maximum offset value is predetermined and does not depend on the current dictionary size. For example, if the predetermined maximum offset value is 800,000, N may be set to 20.

In other implementations, the offset values may be coded using different numbers of ranges and ranges covering different groupings of the offset values.

In certain implementations, length values, like offset values, are coded by range. Also, in this implementation, numbering starting at zero is applied such that the length value is decreased by 1 before the length value is encoded and the length value is increased by 1 after the length value is decoded. The range (with its own length range code), the corresponding length value, and the number of bits are represented by the following table (Table 2).

Using the implementation depicted in Table 2 above, the length can be encoded, signaled, and decoded. As an example, a length value of 2 (representing the original offset value of 3 decremented by 1 for encoding) would be encoded in range 1. The encoded length will be generated by concatenating a length range code of "1" (representing range 1) followed by "10" of a 2-bit value (2-bit binary value for decimal 2). Putting the two parts of the code together (Length Range Code and Length Value Code) is represented by the following combined code for the encoded length: "101". As another example, a length value of 56 (representing the original length value of 57 decremented by 1 for encoding) would be encoded in range 3. Since range 3 starts with an offset value of 20, the value to be coded will be 56-20 = 36. The encoded length will be generated by combining a length range code of "001" (representing range 3) followed by an 8-bit value of "00100100" (an 8-bit binary value for 36 decimal). Putting two parts of the code together (length range code and length value code) is represented by the following combined code for the encoded length: "00100100100".

As depicted in Table 2 above, range 4 uses N bits to represent length values greater than 275, where N represents the number of bits required to represent the maximum offset. In some implementations, the maximum length value is the number of pixels to the left in the current block being encoded or being decoded. For example, if the current pixel value being encoded or being decoded is the 3,000th pixel value in the current 64x64 block (a block with 4,096 pixel values), then the maximum length value is 1,096 (4,096-3,000). , This can be represented by 10 bits (N=10). The offset value for range 4 starts at 276, so, to represent 1,096, only 10 bits are needed to represent quantities over 275 (i.e., 1,096-276 = only 10 bits to represent 820. It should be understood that only is needed). In other implementations, the maximum length value is predetermined and does not depend on the current dictionary size. For example, if the predetermined maximum offset value is 4,096, N may be set to 12.

In other implementations, length values can be coded using different numbers of ranges and ranges covering different groupings of length values.

In some implementations, the maximum offset and/or maximum length is known. If the maximum offset and/or maximum length is known, the coding efficiency can be improved. For example, when coding the value of the matching offset, the maximum offset may be set to the current dictionary size (eg, if the current dictionary size is 10 pixels, the offset cannot be greater than 10). When encoding the value of the matching length, the maximum length may be set as the number of pixels to the left in the current block (eg, the current coding unit (CU). For example, being encoded or decoded) If the current pixel value being being is the 15th pixel in the 8x8 block, the maximum length can be set to 49. If the maximum value (for offset and/or length) is known, the maximum value will be signaled more efficiently. For example, the number of bits required to encode the maximum value can be determined by ceiling(log2(maximum value)), where ceiling(log2(maximum value)) is given in Tables 1 and 2 above. Can be used to define the "N" bit.

In some implementations, the minimum offset and length is 1, which can be coded as 0 when converted to numbering starting at zero.

The 1D dictionary mode can be applied to encode and/or decode pixel values within a block. For example, 1D dictionary mode (as well as other dictionary modes described herein) can be used for blocks of video frames (e.g., 4x4 blocks, 8x8 blocks, 16x16 blocks, 32x32 blocks, and 64x). It can be applied to encode and/or decode pixel values within blocks of various sizes, such as 64 blocks.

In some implementations, the offset and length may overlap the current pixel value being encoded/decoded. As an example, consider the pixel values [P-2, P-1, P0, P1, P2, P3], where P-2 and P-1 are the last two pixel values in the 1D dictionary, and P0 is the value being encoded. /Is the current pixel value being decoded, and P1 to P3 are the next pixel values to be encoded/decoded. In this situation, an offset of 1 and a length of 3 (unencoded offset value and length value) are valid conditions in which P0 is predicted from P-1, P1 is predicted from P0, and P2 is predicted from P1. As should be understood, an offset of 1 (unencoded value, which will be 0 when encoded) means one position before the current pixel value into the 1D dictionary (e.g., in some implementations, a negative The sign is added to the offset, which will be an offset of -1 in this example).

C. Horizontal and vertical scanning

The 1D dictionary mode supports horizontal and vertical scanning, which can be used to convert between a 1D dictionary and a two-dimensional representation of video or image content (e.g., a block of two-dimensional video or image content). For example, pixel values within a block of video content can be scanned horizontally when encoded and decoded. In horizontal scanning, pixel values are added to a 1D dictionary in horizontal scanning order (eg, from left to right in a row of pixels). Pixel values within a block of video content can be scanned vertically when encoded and decoded. In vertical scanning, pixel values are added to a 1D dictionary in vertical scanning order (eg, top to bottom in a column of pixels).

In some implementations, both horizontal and vertical scanning are supported. To support both horizontal and vertical scanning, two 1D dictionaries, one 1D dictionary that stores pixel values in horizontal scanning order (horizontal scanning 1D dictionary) and another 1D dictionary that stores pixel values in vertical scanning order ( Vertical scanning 1D dictionary) can be maintained. If a pixel value needs to be added, the pixel value can be added to both the horizontal scanning 1D dictionary and the vertical scanning 1D dictionary. The ordering of pixel values will be different in both dictionaries, as the ordering depends on which scanning order is used.

In some implementations, adding to the 1D dictionary is performed at different times. For example, when encoding or decoding blocks in the horizontal scanning mode, pixel values may be added to the horizontal scanning 1D dictionary when they are encoded or decoded. When the current block is encoded or decoded, pixel values can be added to the vertical scanning 1D dictionary.

In implementations that support both horizontal and vertical scanning, the scanning order can be changed (eg, on a block-by-block basis or on some other basis). For example, if one block of a picture uses horizontal scanning, the pixel values for that block will be added to the horizontal scanning 1D dictionary (in horizontal scanning order), and the pixel values for that block will also be added to (vertical scanning order). As) will be added to the vertical scanning 1D dictionary. If another block of the picture uses vertical scanning, the pixel values for that block will be added to the vertical scanning 1D dictionary (in vertical scanning order), and the pixel values for that block will also be added to the horizontal scanning 1D (in horizontal scanning order). It will be added to the dictionary.

D. Dictionary Size reduction

The size of a 1D dictionary can be limited (e.g., to balance the cost of maintaining the dictionary and the advantage of predicting pixel values).Reducing the size of the dictionary (e.g. pruning the dictionary) )) can be performed at various times. For example, the size of the dictionary can be checked when adding pixel values to the dictionary. If the dictionary is larger than the maximum size (eg, a predetermined maximum size, eg 500K), the dictionary can be reduced in size (eg, by removing the oldest entry in the dictionary).

In some implementations, a predefined maximum dictionary size is defined. If the dictionary is larger than the predefined maximum dictionary size, then a portion of the dictionary (eg, the oldest part of the dictionary) is removed. In certain implementations, if the dictionary is larger than the threshold size, 1/3 of the dictionary is removed. For example, in some implementations, the default dictionary size is defined as 1<<18. If the current dictionary size is equal to or 1.5 times larger than the base dictionary size, then the oldest 0.5 times base dictionary size element is removed from the dictionary.

In some implementations, the dictionary is only checked periodically (and pruned if necessary). For example, the dictionary may be checked after encoding and/or decoding a block, CU, or CTU. In certain implementations, the size of the dictionary is checked after encoding or decoding the CTU, and is reduced by 1/3 if it exceeds the maximum size. In such an implementation, it should be ensured that the maximum number of elements that may be added to the dictionary between two checks should not be larger than the dictionary buffer size minus the removing threshold. For example, the default dictionary size is defined as 1<<18, and the threshold for removal is defined as 1.5 times the default dictionary size, and 1.5 times should be 1<<18 + 1<<17. After encoding or decoding CTU (assuming that the CTU size is 4096), if the dictionary size is checked, the minimum buffer used for the dictionary should be 1<<18 + <<17 + 4096.

E. Reorganization of the scanning sequence

After decoding the pixel values, the pixel values are reconstructed to rewrite the video content in two dimensions. Reconstructing the pixel values in the scanning order can be performed at various points during the decoding process. For example, after pixel values for a specific region (eg, block, CU, or CTU) of the video content are decoded, the pixel values may be reconstructed in a scanning order.

In some implementations, the reconstruction is performed after the pixel values are decoded for the CU as follows. When horizontal scanning is used for a CU, the following equation (Equation 1) is used to reconstruct the pixel values for the CU in scanning order with a width "w" and a height "h" (rec[i][j] Is the reconstructed pixel in row "i" and column "j"; pixel[] is the decoded pixel):

If vertical scanning is used for a CU, the following equation (Equation 2) is used to reconstruct the pixel values for the CU in scanning order with width "w" and height "h":

F. direct mode

When using the 1D dictionary mode, there may be a situation in which a matching pixel value is not found. For example, during encoding, the encoder to determine if there is a pixel value (or a sequence of multiple pixel values) that matches the current pixel value being encoded (or matches multiple pixel values that are currently being encoded). You can look into the dictionary in the reverse direction. If a match is found, the current pixel value(s) can be encoded in the matching mode using the offset and length coding described above in this section. However, if a matching pixel value is not found in the dictionary, the current pixel value can be encoded using direct mode. In direct mode, the current pixel value can be directly coded (e.g., without reference to any other pixel values in the dictionary, the Y, U, and V components of the pixel value, or the R, G, and And the B component can be directly encoded).

In some implementations, an escape code or flag is used to indicate when direct mode is used for a pixel value. For example, the encoder can place an escape code or flag in the bitstream, along with the directly encoded pixel value, so that the decoder knows that the pixel value has been encoded using direct mode. In this way, the decoder can distinguish between a pixel value encoded in the direct mode and a pixel value encoded using the matching mode. In addition, coding in the 1D dictionary mode may support switching between a matching mode and a direct mode if necessary (eg, on a per-pixel basis).

G. Exemplary Encoding/Decoding

7 is a diagram illustrating a simplified example 700 of how pixel values can be encoded using a 1D dictionary mode. As depicted in example 700, three rows (first, second and last row) of an 8x8 block 710 of exemplary pixel values are depicted. An exemplary block of pixel values 710 is depicted using 3 bytes of YUV or RGB values. The pixel values of the block are labeled starting with pixel zero (P ₀ ) in the horizontal scanning order, for reference purposes.

As illustrated in example 700, pixel values are encoded 720 using a 1D dictionary mode. The first pixel value P ₀ is added to the 1D dictionary as a first entry (eg, the first pixel value may be the first pixel in the first block of the video frame). Since there is no previous pixel value in the 1D dictionary, the first pixel value P ₀ is encoded in direct mode and added to the encoded bit stream. The second pixel value P ₁ is also added to the 1D dictionary in direct mode because the second pixel value P ₁ does not match any previous pixel values in the dictionary. The third pixel value P ₂ is also added to the 1D dictionary in direct mode. The state of the encoded bit stream and 1D dictionary is described at 730. The encoded bit stream is depicted in a simplified format indicating that the first three pixels are encoded using direct mode (eg, direct mode may be represented by an escape code in the encoded bit stream).

When the fourth pixel value P3 is encoded, a match is found in the 1D dictionary. Specifically, P ₀ matches P ₃ and thus P ₃ can be encoded in the matching mode using an offset value and a length value with reference to P ₀ in the 1D dictionary. After the matching pixel P ₀ is identified in the 1D dictionary, the length of the matching pixel value may be determined. In this example, the two pixel values match (i.e., P ₃ and P ₄ match P ₀ and P ₁ ). To encode the offset and length, this example 700 uses the ranges described above (Table 1 and Table 2) in this section. First, the offset value and length value are decremented by 1 (to convert to numbering starting at zero) and encoded using the range. Specifically, the offset value of 2(3-1) is encoded as "110" (the first "1" represents the range 1 and "10" represents the offset value of 2) according to the first row of Table 1. do. The length value of 1(2-1) is encoded as "101" (the first "1" represents the range 1, and "01" represents the length value of 1) according to the first row of Table 2. Adding length and offset appears as a code of "110101". The state of the encoded bit stream and 1D dictionary is described at 740. The encoded bit stream is depicted in a simplified format indicating that the first three pixels are encoded using direct mode and the fourth and fifth pixel values are encoded in the matching mode and predicted from the first and second pixel values.

8 is a diagram illustrating a simplified example 800 of how pixel values can be decoded using a 1D dictionary mode. As depicted in example 800, the encoded bit stream resulting from encoding the block of FIG. 7 is decoded 810 using a 1D dictionary mode. The first three pixel values are decoded in direct mode and added to the dictionary, as depicted at 820.

The fourth and fifth pixel values are decoded using the matching mode. In this example, the encoded bit stream representation for the fourth and fifth pixel values is "110101", which is decoded using the offset and length range defined by Tables 1 and 2 above in this section. Specifically, the offset is decoded as 2 and the length is decoded as 1. Using the offset and length, the pixel values used for prediction are identified. In this example, an offset of 2 (three pixels before after adding 1 to compensate for numbering starting at zero) identifies the first pixel value in the dictionary. The length indicates that two pixel values are predicted (after adding 1 to the length to compensate for numbering starting at zero). Thus, the fourth and fifth pixel values are predicted from the first and second pixel values and added to the dictionary, as depicted at 830.

Once the 8x8 blocks are decoded, the 8x8 blocks are reconstructed in horizontal scanning order. The reconstructed 8x8 block is depicted in 840.

VIII. Doctor 2d Dictionary Mode Korea Innovation Plan

This section presents various innovations for the pseudo 2D dictionary mode. The pseudo 2D dictionary mode is similar to the 1D dictionary mode described in section VII, so the operation of the pseudo 2D dictionary mode is the same as the 1D dictionary mode, except for the differences described in this section.

While the 1D dictionary mode keeps a 1D dictionary of previous pixel values, the pseudo 2D dictionary mode does not keep a separate dictionary. Instead, in the pseudo 2D dictionary mode, all previous pixel values (eg, all of the pixel values previously reconstructed from the start of a picture or frame) may be used for prediction. For example, a video or image encoder or decoder usually keeps all reconstructed pixel values (e.g., for the current picture or frame) during encoding and decoding (e.g., for use during prediction). I can.

Since the pseudo 2D dictionary mode predicts the current pixel value from the pixel values of the two-dimensional picture (e.g., previously reconstructed pixel values), the pseudo 2D dictionary mode has two offset values, namely X offset values (offsetX) and Y Use an offset value (offsetY). The offsetX values and offsetY values can be signaled independently using the technique described above in the 1D dictionary section (eg, using the ranges described in Table 1). For example, if the pixel values at 100, 100 (X/Y from the top-left of the current picture) are predicted from the pixel values at 10 and 20, then offsetX can be set to 90 (picture Represents 90 pixels to the left in the reconstructed pixel value for, this can also be represented as -90) offsetY is 80 (represents 80 pixels up in the reconstructed pixel value for the picture, which is also represented as -80) Can be set).

In the pseudo 2D dictionary mode, the structure of the block is considered when performing prediction. For example, consider the current 8x8 block that is coded using horizontal scanning. If the pixel value of the current block is predicted from the previous 8×8 block, and if the length of the prediction is 9 (i.e., longer than one row of the 8×8 block), then the previous 8×8 block used for prediction The pixel values will wrap around two rows of a block (or from the last row of one block to the first row of the next block).

In some implementations, the following equation (Equation 3) is used to reconstruct the current pixel of the picture in the pseudo 2D dictionary mode. In this equation, the dimension of the current block is width (w) x height (h), the current pixel is the pixel at the current block's position "c" (counted from zero), and (x0, y0) is the upper left of the current block. Is the start position of, the offset is (oX, oY), the scanning order is horizontal, the matching length is 1, and pictureRec[] is the reconstruction of the current picture.

The remaining aspects of the pseudo 2D dictionary mode are described above with respect to the 1D dictionary mode (e.g., signaling of length, maximum number of bits to code length and offset, support for both horizontal and vertical scanning modes, Processing the pixel value components together (eg, Y, U, and V or R, G, and B), etc.).

IX. Inter Doctor 2d Dictionary Mode Korea Innovation Plan

This section presents various innovations for the inter pseudo 2D dictionary mode. The inter pseudo 2D dictionary mode is similar to the pseudo 2D dictionary mode described in section VIII, and therefore the operation of the inter pseudo 2D dictionary mode is the same as the pseudo 2D dictionary mode, except for the differences described in this section.

While the pseudo 2D dictionary model uses reconstructed pixel values of the current picture for prediction, the inter pseudo 2D dictionary mode uses the pixel values of a reference picture (or multiple reference pictures) for prediction. In some implementations, the reference picture used for prediction in the inter pseudo 2D dictionary mode is signaled (eg, by signaling a reference picture list and a reference picture index for the list).

Alternatively, a default reference picture can be used for prediction (eg, to avoid signaling overhead for a particular reference picture from multiple available reference pictures). In some implementations, the default reference picture is the first picture in reference picture list 0.

X. Dictionary Mode Example method for decoding pixel values using

A method for decoding pixel values using a 1D dictionary mode, a pseudo 2D dictionary mode, and/or an inter pseudo 2D dictionary mode may be provided.

9 is a flowchart of an example method 900 for decoding pixel values using a dictionary mode. At 910, encoded data is received in the bit stream. For example, the encoded data may be encoded video data and/or encoded image data.

At 920, the one or more current pixel values are decoded using the dictionary mode. For example, the dictionary mode may be a 1D dictionary mode, a pseudo 2D dictionary mode, or an inter pseudo 2D dictionary mode. One or more current pixel values may be decoded for a block of video content. Decoding the one or more current pixel values includes performing operations 930 to 950.

At 930, an offset representing the offset position within the previously decoded pixel value is decoded. For example, decoding the offset may include an offset range code to obtain an offset value that identifies the offset position within a 1D dictionary of previously decoded (e.g., previously reconstructed) pixel values of the current picture. And decoding the offset value code. Decoding the offset may also include decoding a two-dimensional offset with X and Y offset values to identify previous pixel values using a pseudo 2D dictionary mode or an inter pseudo 2D dictionary mode. In addition, when the inter pseudo 2D dictionary mode is used, reference picture information may be decoded (eg, independently from an offset).

At 940, a length representing the number of pixels being predicted from the offset that was decoded at 930 is decoded. For example, decoding the length may include decoding the length range code and the length value code.

At 950, one or more current pixel values are predicted from one or more previous pixel values at the offset. One or more current pixel values can be accurately predicted using the same pixel values (eg, YUV or RGB component values) as one or more previous pixel values, without any residuals or other corrections. The number of predicted pixel values is represented by the length.

The one or more current pixel values may be used to reconstruct a two-dimensional video picture or image (eg, using a horizontal or vertical scanning order for the current picture) after being predicted.

10 is a flowchart of an exemplary method 1000 for decoding pixel values using a 1D dictionary mode. At 1010, encoded data is received in the bit stream. For example, the encoded data may be encoded video data and/or encoded image data.

At 1020, a number of current pixel values are decoded using the 1D dictionary mode. The 1D dictionary mode stores previously decoded pixel values (eg, previously reconstructed pixel values in the current picture) in the 1D dictionary. Decoding the multiple current pixel values includes performing operations 1030 to 1070.

At 1030, the offset range code is decoded. The offset range code represents the number of bits for the offset value code. For example, possible offset values can be divided into multiple ranges (e.g., as depicted in Table 1 above), where the offset range code represents the number and range of bits used for the offset value code. .

At 1040, the offset value code is decoded (using the number of bits indicated at 1030) to generate an offset value. The offset value identifies the position within the 1D dictionary of previously decoded pixel values. If both the horizontal scanning 1D dictionary and the vertical scanning 1D dictionary are used, the offset value may identify a position within the dictionary corresponding to the scanning order of the current pixel (e.g., the scanning order of the current block).

At 1050, the length range code is decoded. The length range code represents the number of bits for the length value code. For example, possible length values can be divided into multiple ranges (e.g., as depicted in Table 2 above), the length range code representing the number and range of bits used for the length value code. .

At 1060, the length value code is decoded (using the number of bits indicated at 1050) to produce a length value. The length value specifies the number of pixels being predicted.

At 1070, the current pixel value is predicted from the pixel value in at least one dictionary using an offset value and a length value. The current pixel value may be predicted from a corresponding pixel value in a 1D dictionary storing previous pixel values in an order corresponding to the current pixel value (eg, horizontal or vertical scanning order). The position in the 1D dictionary is identified by an offset value with the number of predicted current pixels represented by the length value. The current pixel value can be accurately predicted using the same pixel value (eg, YUV or RGB component value) as the previous pixel value, without any residuals or other corrections.

The current pixel value can be used to reconstruct a two-dimensional video picture or image after it is predicted (eg, using a horizontal or vertical scanning order for the current picture).

XI. 1D and pseudo 2D Dictionary Mode Innovation for encoding

This section presents various innovations for encoding applicable to 1D dictionary mode, pseudo 2D dictionary mode, and/or inter pseudo 2D dictionary mode. Some innovations relate to finding matching pixel values within a dictionary and/or previously reconstructed pixel values, while others relate to early termination and the cost of signaling in matching mode.

A. 1D Dictionary Mode Hash-based matching

In some implementations, the video or image encoder uses a hash-based search technique to identify matching pixel values. In certain implementations of the hash-based search technique, the hash value is every single pixel (e.g., all components of a pixel such as Y, U, and V components, or R, G, and B components, which are treated together. Combined pixels), every 2 pixels, every 4 pixels, and every 8 pixels. For example, the hash value is when the pixel value is added to the dictionary for each combination of 1, 2, 4, and 8 pixels whose current pixel is part of it (e.g., to be added to the 1D dictionary). When) can be created. For example, a first pixel value may be encoded and added to the 1D dictionary. A hash value for the first pixel value may be determined and added (eg, to a hash table). The second pixel value may be encoded and added to the 1D dictionary. A hash value for the second pixel value may be determined and added. Also, a hash value for a two-pixel combination (first pixel value and second pixel value) may be calculated and added, and may continue as additional pixel values are added to the 1D dictionary.

Then, matching is performed to see if the pixel value (or pixel values) in the hash matches the current pixel value (or current pixel value) being encoded. First, for a match every 1 pixel value using the hashed pixel value (e.g. by building a hash of 1 current pixel value and comparing it to the hash of the previous 1 pixel value in the dictionary). A check is made. If one pixel match is found, the encoder can check how many pixels can match from the current pixel to determine the length (number of matching pixels from the current pixel). If a match length of 2 is found (e.g., if the current pixel value matches the pixel value at a specific offset of length 2 in the dictionary), the match will no longer check the hash of 1 pixel for the current pixel. Without need, you can proceed with two pixels and more (eg, a pixel value at a different offset in a dictionary with a length of 2 or more may match the current pixel). Likewise, if a match length of 4 is found, the hash checking starts with 4 pixels and more, and so on for 8 pixels. In some implementations, the hash search is implemented with 1, 2, 4, and 8 pixels. In other implementations, the hash search may use more or fewer pixels.

As an example, consider a dictionary that ends with the following eight pixel values (with the indicated values and positions, e.g. p-3 is the pixel before three pixels in the dictionary with a pixel value of 3) ):

The current pixel will be encoded by the encoder:

Encoding starts in hash encoding mode by checking the hash value for one pixel p0. The hash value for p0 matches the 1 pixel hash value of p-3 (and both p0 and p-3 have a pixel value of 3). Hash matching only determines the starting position of the check. From its starting position, the encoder also needs to check the actual number of matching pixel values. Thus, the encoder checks the length of the matching pixel value. In this example, the encoder checks whether p0 == p-3 (both p0 and p-3 have a pixel value of 3, so if yes), then whether p1 == p-2 Check (both have a pixel value of 4, so if yes), then check whether p2 == p-1 (the pixel values do not match, i.e. 7 != 5, so the encoder stops And determine that the matching length is 2). Next, the encoder starts checking from the hash values for the two pixel values (since a match with a length of 2 has already been found, the encoder no longer checks the hash match of one pixel). The hash value for p0p1 matches the two pixel hash values of p-7p-6. Then, the encoder checks the length of the matching pixel value. In this example, the encoder checks whether p0p1 == p-7p-6 (both has pixel values of 3, 4, so if yes), then checks whether p2 == p-5 (If both have a pixel value of 7 and thus yes), then check whether p3 == p-4 (if both have a pixel value of 1 and thus yes), then p4 == Check if it is p-3 (the pixel value does not match, i.e. 6 != 3, so the encoder stops and determines that the match length is 4). The encoder may then proceed to check a hash match of 4 pixels (and eventually with a hash match of 8 pixels) to see if a longer match length can be found. When the encoder finishes checking, the current pixel value will be encoded with the largest match length found.

If a pixel value (or multiple pixel values) in a dictionary (e.g., a 1D dictionary) has the same hash value as the current pixel value, then matching is still performed to see if the pixel value in the dictionary can be used for prediction. do. For example, the hash value for one pixel in the 1D dictionary may be the same as the hash value for the current pixel. The pixel values in the 1D dictionary still need to be compared to determine if the pixel values of the current pixel are the same (ie different pixel values may have the same hash value).

In some implementations, even if a match is found for one or more current pixels, the cost of encoding one or more current pixels (e.g., in terms of number of bits) in a matching mode using offset and length is (e.g. For example, it may be higher than the cost of directly encoding one or more current pixels) in terms of the number of bits. In this situation, one or more current pixels may be directly coded (e.g., the encoder may switch from matching mode to direct mode for one or more current pixels, which is identified in the bit stream by an escape code or flag. Can be). The encoder can switch between matching mode and direct mode as needed (eg, on a per-pixel basis, on a per-block basis, or on some other basis).

In some implementations, early termination is performed by the encoder. For example, enough pixel values (e.g., N pixel values) have been processed, and the average matching length (for direct mode, the matching length can be considered as 1) is the threshold (e.g., the threshold of T). Value), the dictionary mode estimation can be terminated early (e.g., on a block-by-block basis). For example, the dictionary mode can be terminated and the picture can be re-encoded using a different encoding mode, or the dictionary mode can be terminated for the remainder of the picture or part of the picture (e.g., the current block). . Early termination may be performed when the average matching length is sufficiently small so that the dictionary mode is less efficient than other encoding modes (eg, when it is less efficient than normal intra mode, normal inter mode, etc.). For example, in some implementations, the average match length threshold T may be 2 or 3.

B. Doctor 2D Dictionary Mode Hash-based matching

The hash-based matching during encoding may be performed in the pseudo 2D dictionary mode (and in the inter pseudo 2D dictionary mode), similar to the hash-based matching described above for the 1D dictionary mode.

Like the 1D dictionary mode, a hash value is created for the previous pixel value in the grouping of 1, 2, 4, and 8 pixel values. However, if they match, the pseudo 2D dictionary mode (and inter pseudo 2D dictionary mode) starts checking with a hash value of 8 pixels (rather than starting with a 1 pixel hash match). If a match of length 8 is found, the maximum length should not be less than 8 and there is no need to check hash values of 4 pixels or less. However, if a match of length 8 is not found, then checking for a match of 4 pixels starts, and continues up to 1 pixel that way. If 8 pixel matches are not found by hash matching, the current matching length is 7 (e.g., a hash match for 4 pixels is found and from its starting position, the encoder finds that 7 matching pixels actually exist). ), and the encoder can end here because there is no match for 8 pixels.

C. Dictionary Mode Example method for encoding pixel values using

A method may be provided for encoding pixel values using a 1D dictionary mode, a pseudo 2D dictionary mode, and/or an inter pseudo 2D dictionary mode. Encoding may include calculating a hash value of a previous pixel value (eg, a reconstructed pixel value) and comparing the hash value to a hash value of a current pixel value to be encoded. Matches can be identified and encoded by offset and length (eg, in a previously encoded value of the picture or in a 1D dictionary). The encoding can be performed in direct mode if no match is found.

11 is a flowchart of an example method 1100 for encoding pixel values using a dictionary mode. At 1110, the one or more current pixel values are encoded in a dictionary mode (eg, 1D dictionary mode, pseudo 2D dictionary mode, or inter pseudo 2D dictionary mode). Decoding the current pixel value includes performing an operation according to 1120 to 1150.

At 1120, a hash value is calculated for a previously encoded pixel value (e.g., a reconstructed pixel value). For example, the hash value is 1 pixel, 2 pixels, 4 pixels, and 8 pixels. Can be calculated for combinations.

At 1130, a hash value is computed for one or more current pixel values to be encoded.

At 1140, the hash values for the one or more current pixel values are compared to the hash value of the previously encoded pixel value to determine if a match is found. If a match is found (eg, for one pixel value), the length of the matching pixel can be determined.

At 1150, if a match is found, the one or more current pixel values are encoded using the offset and length. For example, the offset and length can be reconfigured before or at the position in the 1D dictionary where the current pixel value is predicted (e.g., using X and Y offset values for pseudo 2D dictionary mode or inter pseudo 2D dictionary mode) It can indicate the position within the picture.

Claims (24)

A method in a computing device having a video decoder or an image decoder, the method comprising:
Receiving encoded data for a picture in a bit stream; And
Decoding one or more current pixel values from the encoded data, the decoding of the one or more current pixel values,
Decoding an offset representing an offset position in at least one one-dimensional dictionary of previously decoded pixel values from the encoded data, the at least one one-dimensional dictionary being a horizontal scanning one-dimensional dictionary, and decoding the offset :
Decoding an offset range code indicating a range of offset values and a number of bits to decode for the offset value; And
Decoding the offset value from the number of bits for the offset value based on the offset range code, wherein the offset position in the one-dimensional dictionary is identified by the offset value
Including, decoding the offset;
Decoding a length from the encoded data; And
Predicting the one or more current pixel values from one or more corresponding pixel values in the previously decoded pixel values at the offset location, the number of predicted pixels represented by a length value;
Decoding the one or more current pixel values including;
Adding the decoded one or more current pixel values to the horizontal scanning one-dimensional dictionary in a horizontal scanning order; And
Adding the decoded one or more current pixel values to a vertical scanning one-dimensional dictionary in a vertical scanning order. The method of claim 1,
Wherein the one or more current pixel values and the one or more corresponding pixel values are combined YUV pixel values. The method of claim 1,
Wherein the one or more current pixel values are decoded according to a 1D dictionary mode. The method of claim 3,
Determining the size of the one-dimensional dictionary; And
If the size of the one-dimensional dictionary is larger than a predetermined maximum value, reducing the size of the one-dimensional dictionary
The method in a computing device further comprising. The method of claim 1,
The step of decoding the length,
Decoding a length range code indicating the range of length values and the number of bits to be decoded for the length value; And
Decoding the length value from the number of bits for the length value based on the length range code
Including,
Wherein the predicted number of pixels is identified by the length value. The method of claim 1,
The one or more current pixel values are decoded in a matching mode that predicts the one or more current pixel values from the one or more corresponding pixel values in the previously decoded pixel values,
The above method,
Decoding one or more other current pixel values from the encoded data
Including more,
Decoding one or more other current pixel values from the encoded data,
Decoding the one or more other current pixel values using a direct mode in which the one or more other current pixel values are directly encoded without prediction
A method in a computing device comprising a. The method of claim 1,
Reconstructing at least a portion of the picture in one of a horizontal scanning order and a vertical scanning order using at least partially the decoded one or more current pixel values
The method in a computing device further comprising. The method of claim 1,
Wherein the offset represents an X/Y offset position within a current picture of previously decoded pixel values, the one or more current pixel values being decoded according to a pseudo 2-D dictionary mode, Method on a computing device. The method of claim 8,
The step of decoding the offset,
Decoding an X offset value from a first offset range code indicating a range of offset values and a number of bits to decode for the X offset value; And
Decoding a Y offset value from a second offset range code indicating a range of offset values and the number of bits to be decoded for the Y offset value.
Including,
Wherein the X/Y offset position within the current picture of the previously decoded pixel values is identified by the X offset value and the Y offset value. A method in a computing device having a video decoder or an image decoder, the method comprising:
Receiving encoded data for a picture in a bit stream; And
A step of decoding a plurality of current pixel values from the encoded data using a 1D dictionary mode, the step of decoding the plurality of current pixel values,
Decoding an offset range code, the offset range code representing the number of bits for an offset value code;
The offset value-the offset value identifies a position in at least one dictionary of previously decoded pixel values, and the at least one dictionary includes a horizontal scanning one-dimensional dictionary and a vertical scanning one-dimensional dictionary-indicated above to generate Decoding the offset value code from a lower number of bits;
Decoding a length range code, the length range code indicating the number of bits for a length value code;
Decoding the length value code from the indicated number of bits to produce a length value;
From the corresponding pixel values at the position in the at least one dictionary, identified by the offset value, with the number of predicted current pixel values-the number of current pixel values being represented by the length value- Predicting the current pixel values
Decoding the plurality of current pixel values;
Adding the plurality of decoded current pixel values to the horizontal scanning one-dimensional dictionary in a horizontal scanning order; And
Adding the plurality of decoded current pixel values to the vertical scanning one-dimensional dictionary in a vertical scanning order
A method in a computing device comprising a. The method of claim 10,
Reconstructing at least a part of the picture in one of a horizontal scanning order and a vertical scanning order by using at least partially the decoded plurality of current pixel values
The method in a computing device further comprising. A method in a computing device having a video encoder or an image encoder, comprising:
Encoding data for the picture, including using a dictionary mode to encode one or more current pixel values.
Including,
The encoding step,
Calculating hash values for previously encoded pixel values;
Calculating a hash value for the one or more current pixel values to be encoded;
Determining whether a hash value for the at least one current pixel value matches a hash value for the previously encoded pixel values;
If a match is found, encoding the one or more current pixel values using a length and an offset that predicts the one or more current pixel values from matching previously encoded pixel values.
Including, the step of encoding the one or more current pixel values,
Encoding an offset range code, the offset range code indicating a number of bits for an offset value code;
Encoding the offset value code from the indicated number of bits to generate an offset value, the offset value identifying a location within at least one dictionary of previously encoded pixel values, and wherein the at least one dictionary Includes a horizontal scanning one-dimensional dictionary and a vertical scanning one-dimensional dictionary -;
Encoding a length range code, the length range code indicating a number of bits for a length value code;
Encoding the length value code from the indicated number of bits to produce a length value;
The number of current pixel values-the number of current pixel values to be encoded is represented by the length value-and the corresponding pixel values at the location in the at least one dictionary, identified by the offset value, are used. Encoding the one or more current pixel values
Including,
The encoded current pixel values of the number are added in the horizontal scanning order to the horizontal scanning one-dimensional dictionary,
The encoded current pixel values of the number are added in the vertical scanning order to the vertical scanning one-dimensional dictionary,
Method on a computing device. The method of claim 12,
Wherein the one or more current pixel values and the previously encoded pixel values are one of combined YUV pixel values, combined RGB pixel values, and combined GBR pixel values. The method of claim 12,
Computing hash values for the previously encoded pixel values,
Calculating hash values for each one of the previously encoded pixel values;
Calculating hash values for each of the two pixel values of the previously encoded pixel values;
Calculating hash values for each of the four pixel values of the previously encoded pixel values; And
Calculating hash values for each of the eight pixel values among the previously encoded pixel values
A method in a computing device comprising a. The method of claim 12,
Calculating an average matching length;
When the average matching length is less than a threshold value, switching to an encoding mode other than the dictionary mode for the current block
The method in a computing device further comprising. A computing device comprising a processing unit and a memory and adapted to perform the method of claim 1. In the one or more computer-readable storage media storing computer-executable instructions,
Wherein the computer-executable instructions cause a computing device programmed by the computer-executable instructions to perform the method of any one of claims 1-15. media. delete delete delete delete delete delete delete

Images