US20220312029A1 - Video encoding/decoding using detected pattern of pixel intensity differences - Google Patents
Video encoding/decoding using detected pattern of pixel intensity differences Download PDFInfo
- Publication number
- US20220312029A1 US20220312029A1 US17/215,347 US202117215347A US2022312029A1 US 20220312029 A1 US20220312029 A1 US 20220312029A1 US 202117215347 A US202117215347 A US 202117215347A US 2022312029 A1 US2022312029 A1 US 2022312029A1
- Authority
- US
- United States
- Prior art keywords
- pixels
- block
- reconstructed
- predicted
- prediction
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 claims abstract description 56
- 239000013598 vector Substances 0.000 claims abstract description 39
- 230000002123 temporal effect Effects 0.000 claims description 46
- 230000006835 compression Effects 0.000 claims description 42
- 238000007906 compression Methods 0.000 claims description 42
- 238000012360 testing method Methods 0.000 claims description 20
- 238000013211 curve analysis Methods 0.000 claims description 12
- 238000001514 detection method Methods 0.000 description 20
- 238000010586 diagram Methods 0.000 description 11
- 238000012545 processing Methods 0.000 description 6
- 238000013139 quantization Methods 0.000 description 5
- 230000002093 peripheral effect Effects 0.000 description 3
- 238000013213 extrapolation Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 230000003190 augmentative effect Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000006837 decompression Effects 0.000 description 1
- 238000009795 derivation Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000002620 method output Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 239000000758 substrate Substances 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
- H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
- H04N19/51—Motion estimation or motion compensation
- H04N19/513—Processing of motion vectors
- H04N19/517—Processing of motion vectors by encoding
- H04N19/52—Processing of motion vectors by encoding by predictive encoding
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
- H04N19/136—Incoming video signal characteristics or properties
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/103—Selection of coding mode or of prediction mode
- H04N19/107—Selection of coding mode or of prediction mode between spatial and temporal predictive coding, e.g. picture refresh
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
- H04N19/146—Data rate or code amount at the encoder output
- H04N19/147—Data rate or code amount at the encoder output according to rate distortion criteria
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
- H04N19/172—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a picture, frame or field
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
- H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/182—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a pixel
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
- H04N19/593—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving spatial prediction techniques
Abstract
Description
- Image frames are often compressed through a coding scheme to reduce the amount of data transmitted in video systems. A video codec is a device or firmware/software program that enables video compression and/or decompression for digital video. Certain video, such as gaming applications that use graphics processors to create graphics content employ shading algorithms to apply shading to objects and scenes for a realistic image quality. For example, cloud gaming platforms employ cloud servers that output streams of video gaming content at high resolution and employ video encoders to compress image frames that are then decoded by a decoder in a client device such as a gaming console, laptop, handheld device, high definition television or other suitable decoding device. Also, live video from cameras are also compressed such as when a user employs a camera during a multi-player gaming event. Having proper encoding schemes to allow throughput of high quality content is paramount in the use of cloud platforms.
- Prediction techniques are used in video compression systems to enable efficient encoding. In block-based video compression, the closer the predicted block to the original one, the more efficient the encoding is. Contemporary video encoding technologies rely on two main types of prediction, namely intra and inter prediction. In intra prediction, reconstructed left and/or top spatially neighboring lines of pixels are used to predict the original block in different angular modes. On the other hand, in inter prediction, a temporally neighboring reconstructed block of pixels is used to make the prediction. In many cases, the reference pixels and the pixels of the current frame may belong to areas that are shaded differently, and hence it becomes difficult to predict the current from the reference without spending too many bits on residual data.
- In regions of video with motion, a number of pixels change from one image frame to the next, and the video compression scheme must send more data to keep up with the larger number of pixels that are changing. In extreme cases of high-frequency detail changes, the video quality may decrease. In order to maintain video quality, yet reduce the amount of data that is transmitted, various different compression techniques have been developed. For example, MPEG-based video compression typically operates on groups of pixels called macroblocks. Macroblocks are compared within a frame (intra) and/or compared from one image frame to the next image frame (inter). For inter-prediction the video compression the codec sends macroblock motion vectors and residual information about differences between the macroblocks. Areas of video that have no motion thus require very little transmitted data.
- The temporal prediction technique used in MPEG video, for example, is based on motion estimation. Motion estimation is based on the premise that, in most cases, consecutive image frames will be similar except for changes caused by objects moving within the frames. A motion vector is the key element in the motion estimation process. The motion vector is used to represent a macroblock in a frame based on the position of the macroblock (or a similar one) in another image frame, called the reference frame. The H.264/MPEG-4 AVC standard defines a motion vector as a two-dimensional vector used for inter prediction that provides an offset from the coordinates in the decoded image frame to the coordinates in a reference frame. In general, motion estimation is the process of determining the motion vectors that describe the transformation from one image frame to another image frame, where the image frames are usually adjacent in a video sequence. Motion vectors may relate to the whole image (global motion estimation) or specific parts, such as rectangular blocks or arbitrary shaped patches. The motion vectors may be represented by a translational model or other models that can approximate the motion of a real video recording system.
- Applying the motion vectors to an image frame to synthesize the transformation to the next image frame is called motion compensation. The combination of motion estimation and motion compensation is a key part of the video compression method used by many video codecs.
- Some systems employ both intraframe prediction and interframe prediction and select a best candidate for use in encoding the macroblock of the image, after a rate distortion optimization (RDO) process is performed. The RDO process is a method for performing a compression quality test on inter-predicted candidates and different angular modes of intra-predicted candidates for blocks of pixels and the best candidate is used to encode a pixel or block of pixels. Where differently shaded objects or sections of a frame occur, having an inferior compression technique can result in poor output image quality at comparable bitrates, or require higher bit rates for comparable image quality that is provided from a decoder.
- Some known systems may employ interframe prediction to determine a temporal linear trend among temporal blocks of pixels but require using spatially neighboring pixels to the corresponding temporal block in a single temporal frame and the spatially neighboring pixels to the block being predicted. Such systems may not provide needed accuracy in detecting temporal trends.
- The implementations will be more readily understood in view of the following description when accompanied by the below figures, wherein like reference numerals represent like elements, and wherein:
-
FIG. 1 is a block diagram of an apparatus for encoding one or more images in accordance with one example set forth in the disclosure; -
FIG. 2 is a flowchart illustrating a method for encoding one or more images in accordance with one example set forth in the disclosure; -
FIG. 3 is a diagrammatic illustration of predicting pixels for a block of reconstructed pixels based on a detected spatial pattern of pixel intensity differences in accordance with one example set forth in the disclosure; -
FIG. 4 is a block diagram of an apparatus for encoding one or more images in accordance with one example set forth in the disclosure; -
FIG. 5 is a flowchart illustrating a method for encoding one or more images in accordance with one example set forth in the disclosure; -
FIG. 6 is a block diagram of an apparatus for encoding one or more images in accordance with one example set forth in the disclosure; -
FIG. 7 is a block diagram illustrating a decoding device for decoding one or more image frames in accordance with one example set forth in the disclosure; -
FIG. 8 is a block diagram illustrating a decoding device for decoding one or more image frames in accordance with one example set forth in the disclosure; -
FIG. 9 is a block diagram of a system for encoding and decoding one or more images in accordance with one example set forth in the disclosure; and -
FIG. 10 diagrammatically illustrates temporal pixel intensity trend detection in accordance with one example set forth in the disclosure. - In the following description, the use of the same reference numerals in different drawings indicates similar or identical items. Unless otherwise noted, the word “coupled” and its associated verb forms include both direct connection and indirect electrical connection by means known in the art, and unless otherwise noted any description of direct connection implies alternate embodiments using suitable forms of indirect electrical connection as well.
- Briefly, a method and apparatus process reconstructed pixels, also referred to as reference pixels, within a reconstructed frame or among temporal reconstructed frames and detect patterns of pixel intensity differences among neighboring reconstructed pixels. The detected pattern of pixel intensity differences, also referred to as a shading trend, is then used to extrapolate pixels to predict pixels for blocks of the reconstructed frame. The predicted pixels are then used to encode a block of pixels of an image frame which then may be decoded and displayed or further processed.
- In some implementations, the encoder and if desired the decoder, detects patterns of pixel intensity differences from shading differences in graphics content such as in gaming applications and also detects shading differences for video produced by cameras. In certain implementations, a conventional prediction process is augmented to include a process that analyzes the reconstructed pixels (e.g., reference pixels) to detect patterns in pixel intensity differences in reconstructed pixels resulting in predicted reconstructed pixels that represents a better predictor of the current block. In certain implementation, the parameters of the shading model used by a graphics processor to create graphics images for example can be detected based on the trend in neighboring reconstructed pixels and/or it can be signaled in the bitstream to the decoder.
- Stated another way, in some implementations, the pattern of pixel intensity differences is an additional process used by an encoder to present candidate reconstructed pixels that are compared with other candidate reconstructed pixels generated by other prediction techniques. In some examples, the encoder chooses the best candidate prediction technique and applies it to a given block. The pattern of pixel intensity differences detection is employed in both intra-prediction operations and/or inter-prediction operations.
- In certain implementations, a method in an encoding device for encoding at least one image frame includes providing intra-frame prediction by generating a predicted block from reconstructed pixels for a block of current pixels, based on a detected spatial pattern of pixel intensity differences among a plurality of spatially neighboring reconstructed pixels to the block of current pixels; and encoding the block of current pixels of the image frame using the predicted block of reconstructed pixels.
- In some examples, the method provides inter-frame prediction by generating the predicted block from reconstructed pixels for the block of current pixels, based on a detected temporal pattern of pixel intensity differences among temporal blocks of pixels in temporally neighboring reconstructed frames corresponding to a candidate motion vector of the block of current pixels and produces predicted blocks for a reconstructed frame based on the determined pattern of pixel intensity difference among temporally neighboring frames.
- In certain examples, the method performs a compression quality test on the intra-predicted reconstruction pixels and inter-prediction reconstructed pixels. Based on the compression quality test, a compression mode is selected that uses at least one of the intra-predicted reconstructed pixels or inter-predicted reconstructed pixel to encode a block of pixels in the frame.
- In some examples, the method outputs data for a decoder indicating that an intensity difference pattern prediction operation was used in the encoding process. In certain examples data representing a formula defining the pattern is output for a decoder.
- In certain examples, the method performs a best fit curve analysis on differences among the neighboring reconstructed pixels and applies the determined best fit curve to predict pixel intensities for pixels in the predicted block of reconstructed pixels. In some implementations the best fit curve analysis includes detecting linear pixel intensity differences among the plurality of neighboring reconstructed pixels for a plurality of differing angular prediction modes.
- In some implementations, an encoding device for encoding at least one image frame includes a block-based predictor that provides intra-frame prediction by generating a predicted block from reconstructed pixels for a block of current pixels, based on a detected spatial pattern of pixel intensity differences among a plurality of neighboring reconstructed pixels to the block of current pixels. The encoding device encodes the block of current pixels of the image frame using the predicted block of reconstructed pixels.
- In certain examples, the block-based predictor performs a best fit curve analysis on differences among the neighboring reconstructed pixels and apply the determined best fit curve to predict pixel intensities for pixels in the predicted block of reconstructed pixels.
- In some examples, the block-based predictor detects linear pixel intensity differences among the plurality of neighboring reconstructed pixels for a plurality of differing angular prediction modes and apply the detected linear differences to predict pixels for the block of reconstructed pixels.
- In certain examples, the block-based predictor includes a compression mode determinator that performs a compression quality test on the predicted reconstruction pixels as compared to predicted pixels from other prediction modes, and based on the compression quality test, selects a compression mode that uses the predicted reconstructed pixels to encode a block of pixels in the frame.
- In some examples, the block-based predictor provides inter-frame prediction by generating the predicted block from reconstructed pixels for the block of current pixels, based on a detected temporal pattern of pixel intensity differences among temporal blocks of pixels in temporally neighboring reconstructed frames corresponding to a candidate motion vector of the block of current pixels, and produces predicted blocks for a reconstructed frame based on the determined pattern of pixel intensity difference among temporally neighboring frames. In certain examples, the encoder encodes blocks of the current input frame using the predicted blocks.
- In some examples, the block-based predictor includes a compression mode determinator that performs a compression quality test on the intra-predicted reconstruction pixels and inter-prediction reconstructed pixels, and based on the compression quality test, selects a compression mode that uses at least one of the intra-predicted reconstructed pixels or inter-predicted reconstructed pixel to encode a block of pixels in the frame.
- In certain examples, the encoder outputs data for a decoder indicating that an intensity difference pattern prediction operation was used in the encoding process.
- In some implementations, an apparatus for decoding an encoded image frame includes a block-based decoder that receives a stream of encoded video, and decodes an encoded block of pixels that is encoded using a predicted block of reconstructed pixels that is predicted based on a detected spatial pattern of pixel intensity differences from neighboring reconstructed pixels in a reconstructed frame.
- In certain examples, the block-based decoder receives data indicating that a spatial pattern of pixel intensity differences was used to encode the encoded video and decodes the encoded block of pixels based on the data.
- In some examples, the block-based decoder performs intra-prediction based decoding by predicting pixels for a decoded block of reconstructed pixels, based on a detected spatial pattern of pixel intensity differences among a plurality of neighboring reconstructed pixels of a reconstructed frame in a spatial direction that is identified by a decoded intra-prediction mode, and outputting the decoded block that includes the predicted pixels.
- In certain examples, the decoder performs inter-prediction based decoding by generating a plurality of reconstructed temporally neighboring frames corresponding to a motion vector, determining whether blocks of pixels in the temporally neighboring reconstructed frames corresponding to the motion vector have a pattern of pixel intensity differences among the blocks from temporally neighboring frames, and producing predicted blocks for a reconstructed frame based on the determined pattern of pixel intensity difference among temporally neighboring frames.
- In some examples, the block-based decoder performs a best fit curve analysis on differences among the neighboring reconstructed pixels and apply the determined best fit curve to predict pixel intensities for pixels in the predicted block of reconstructed pixels.
- In certain examples, the block-based decoder detects linear pixel intensity differences among the plurality of neighboring reconstructed pixels for a plurality of differing angular prediction modes and apply the detected linear differences to predict pixels for the block of reconstructed pixels.
- In some examples, the block-based decoder receives data in the encoded video indicating a type of pixel intensity difference pattern that was used to encode data, and uses the indicated type of pixel intensity difference pattern for prediction of pixel data in a reconstructed frame when decoding the encoded video.
-
FIG. 1 is a block diagram illustrating one example of anencoding device 100 employing detection of spatial patterns of pixel intensity differences among neighboring reconstructed pixels that are neighboring to a block of current pixels to be predicted. A block-based prediction unit generally shown as 102 has anintra-prediction detector 104 that provides candidate predicted pixels for a block of reconstructed pixels, based on detected spatial patterns of pixel intensity differences among neighboring reconstructed pixels.Inter-prediction detector 106 provides predicted pixels for a block of reconstructed pixels based on detected temporal patterns of pixel intensity differences among neighboring reconstructed pixels that are in neighboring temporal frames. Theencoding device 100 includes memory that stores one or morecurrent frames 108,reference frames 110 and reconstructedframes 112. As known in the art, reference frames are a subset of a reconstructed frames. Once a reconstrued frame is generated, it can be used as reference frame if desired but is not always used as such. In this example, theencoding device 102 receives an input image frame such ascurrent frame 108 and encodes the frame through block-based encoding and outputs encoded blocks ofpixels 116 as a video stream. Theinter-prediction detector 106 serves as a motion estimation unit that receives as input thecurrent frame 108 and/or reconstructedframe 112. The output of theinter-prediction detector 106 andintra-prediction detector 104 are provided to a compressionmode determination unit 118 that decides which mode is to be used as the final prediction mode for a given block of pixels to be encoded through atransform stage 120, aquantization stage 122 andentropy encoder 124, as known in the art. Thetransform stage 120 can be, for example, a forward discrete cosine transform (FDCT) module. Thequantization stage 122 can be any suitable quantization unit. The output of thedecision module 118 is the predicted block of reconstructed pixels shown as 128. Acombiner 130, as known in the art, combines the predictedpixels 128 with a corresponding current pixels from thecurrent frame 108 whoseoutput 132 is provided to thetransform stage 120. - The encoding unit also includes an
inverse quantization stage 134, aninverse transform stage 136, acombiner 138, adeblocking stage 140 as known in the art. Thecombiner 138 outputs non-filteredreconstructed pixels 142 which are input to theintra-prediction detector 104. It will be recognized that reconstructed pixels from reconstructedframe 112 orreference frame 110 may also serve as input to theintra-prediction detector 104 as desired. It will be recognized that theencoder device 100 although shown to have both intra-prediction and inter-prediction may be configured to provide either type or both types. For purposes of illustration, theencoder unit 100 will be described as including both intra-prediction and inter-prediction operations. - In some implementations, the
intra-prediction detector 104 andinter-prediction detector 106 perform conventional prediction operations such as those compliant with H.264/MPEG 4 standards or any other suitable video encoding standards. For example, in some implementations, theintra-prediction detector 104 performs in-frame angular mode prediction to provide prediction candidates associated with desired angular modes shown as predictedpixels 150. Theinter-prediction detector 106 performs inter-prediction among differing frames to provide candidates of predictedpixels 152. Unlike conventional encoders, theencoding unit 102 in some implementations, also produces intra-frame predictedpixels 154 for differing angular modes wherein the intraframe predicted pixels are for blocks of reconstructed pixels that are based on a detected spatial pattern of pixel intensity differences among a plurality of neighboring reconstructed pixels that are neighboring to the block of reconstructed pixels to be predicted. Theinter-prediction detector 106 produces predicted blocks ofpixels 156 wherein the predicted blocks are for a reconstructed frame and are based on a determined pattern of pixel intensity differences among blocks of temporally neighboring frames. - The
inter-prediction detector 106 provides inter-frame prediction by determining whether blocks of pixels in temporally neighboring reconstructed frames that correspond to a given motion vector, have a pattern of pixel intensity differences among the corresponding blocks associated with the motion vectors, where the blocks are from temporally neighboring frames. For example, where a motion vector is associated with an object in a game, such as a tank or airplane that is moving from frame to frame, the motion vector identifies, as known in the art, the amount of movement a block makes from one temporal frame to the next. Theinter-prediction detector 106 detects pixel intensity differences and determines if a pattern exists among candidate blocks referred by motion vectors across frames. Inter-frame trend detection is performed without using spatially adjacent pixels to a current block to be predicted and spatially adjacent pixels to a corresponding block in a temporally adjacent frame. Instead reconstructed pixels of multiple corresponding blocks in multiple temporally associated frames are used to predict the temporal pixel intensity trend. Theinter-prediction detector 106 andintra-prediction detector 104 have, in some implementations, programmable registers that are set with pixel intensity thresholds (e.g., luminance thresholds) that have been determined to be at a difference level that is useful for attempting to detect a pattern. It will be recognized however that the encoder may be preset so that any suitable pixel intensity difference will be stored to use as a point value to determine whether a pattern exists. - The compression
mode determination unit 118 in some implementations, evaluates candidate predicted pixels and/or blocks 150, 152, 154 and 156 and performs a compression quality test on the differing intra-predicted and inter-predicted pixels to choose a best candidate to use for encoding the current frame (e.g., block in a frame). One example of a compression quality test is an RDO test. In one example, the compressionmode determination unit 118 performs a compression quality test on at least theintra-predicted reconstruction pixels 154 and inter-predictionreconstructed pixels 156 and based on the compression quality test, selects a compression mode as the final compression mode and outputs the selected reconstructed pixels that are predicted from a detected spatial or temporal pattern of pixel intensity differences as predictedpixels 128. The predictedpixels 128 are combined (e.g., subtracted) atcombiner 130 with original pixels to obtain the residuals which get encoded and sent to the decoder. During decoding, the reconstructed pixels get added to the residuals in the decoder to get reconstructed. In some implementations, the standard generatedintra-predicted pixels 150 and standard generatedinter-predicted pixels 152 are also considered during mode decision when presented. For some cases, a standard mode gives better predictors than a shading-based mode and will get selected as the best candidate. - Referring also to
FIG. 2 , which illustrates a flowchart of a method for encoding an image frame as carried out, for example, by theencoding device 100 ofFIG. 1 . As shown inblock 200, the method includes providing intra-frame prediction by generating a predicted block from reconstructed pixels for a block of current pixels, based on a detected spatial pattern of pixel intensity differences among a plurality of neighboring reconstructed pixels to the block of current pixels. This is carried out, for example, by theintra-prediction detector 104. The predicted pixels that are based on detected spatial pattern of pixel intensity differences is shown as predictedpixels 154. - As shown in
block 202, the method includes providing inter-frame prediction by determining whether blocks of pixels in temporally neighboring reconstructed frames that correspond to a candidate motion vector, have a pattern of pixel intensity differences among the blocks from temporally neighboring frames. This is carried out, for example, by theinter-prediction detector 106. The method includes as shown inblock 204, producing predictedblocks 156 based on the determined pattern of pixel intensity difference among temporally neighboring frames. As shown inblock 206, the method includes encoding a block of pixels of the image frame using the predicted block ofreconstructed pixels mode decision 118 chooses an intra-predicted block as a best candidate or an inter-prediction block as a best candidate. - The spatial pattern of pixel intensity differences in one example are detected by performing a best fit curve analysis on differences among neighboring reconstructed pixels and apply the determined best fit curve to predict pixel intensities for pixels for a predicted block of reconstructed pixels. One example of a best fit curve is the detection of a linear pixel intensity differences among neighboring reconstructed pixels whether the reconstructed pixels are neighboring within the frame or among temporally neighboring frames. In one example, the linear pixel intensity differences are detected for differing angular prediction modes. The detected linear differences are applied to predict pixels for the block of reconstructed pixels. Also, in some implementations, a set of typical pre-determined models is examined until one of them is identified as a good prediction. In some cases, such as synthesized graphics content, the shading model can be passed by the application to the encoder.
-
FIG. 3 illustrates diagrammatically an implementation where a detected spatial pattern of pixel intensity differences is detected to be a linear pattern. This example illustrates theintra-prediction detector 104 that provides spatial based intensity pattern detection. Although the example shown is a best fit curve for a linear relationship, it will be recognized that best fit curve analysis that analyzes differences among neighboring reconstructed pixels such as non-linear relationships and any other suitable patterns may be employed to allow prediction of reconstructed pixels to encode a block of pixels in a frame. - In this example, a two-dimensional angle is being assessed to generate the predicted reconstructed pixel values during intra prediction, n lines of neighboring pixels will be used to detect the suitable shading process (if any). For instance, if an almost linear increase in pixel intensity is detected between neighboring pixels along the same angle, such linear increase is applied while generating the predicted pixels.
- For example, assuming n=2. If pixels CDE are to be horizontally predicted from pixel B in the classical way, they will all get the pixel value of B (i.e., A0). However the encoder uses a different approach where reconstructed pixels A and B are used to detect a trend in pixel intensity differences between the pixels in the horizontal direction (e.g., linear increase based on the delta A0-A1), in which case, the same trend is followed to predict pixels CDE whose values continue increasing linearly leading to pixel values (P0, P1, and P2) respectively.
- In the case for inter-prediction, while assessing a candidate motion vector, blocks in reconstructed frames that represent the trajectory of the motion are inspected to see if there is a shading trend. If a trend is detected among blocks in temporally neighboring frames, a formula defining the trend is applied during the inter-prediction process to come up with a better-predicted block. For example, instead of pixels being used in
FIG. 3 , thedetector 106 uses blocks of pixels. So A and B will be the temporal neighboring block of pixels and C will be the current block of pixels. The trend can be formulated as scaling and/or offsetting operation. For example, where a linear trend y is defined as y=aX+b, a is scaling factor and b is the offset. In some implementations, non-linear trends are detected. The shading operation can be implicit (self-derived) as the shown in the intra prediction example above, or it can be explicitly defined with shading parameters provided and signaled into the bitstream (e.g., a flag at the block level). For example, for a fitting curve, a X3+b X2+cX+d, then a, b, c, d will be the parameters sent in the bit stream to the decoder. - For example, referring also to
FIG. 10 , theinter-prediction detector 106 receivescurrent frame 108 and multiple reconstructedframes FIG. 10 , thetemporal trend detector 410 compares pixel intensities for pixels that are in temporally adjacent frames forblocks Blocks current block 1006, that are temporally neighboring blocks defined by the trajectory of the candidate motion vector (MVxy). For example, where a plane serves as an object that is tracked by motion vectors, the plane moving from one frame to another may have a lighter shading in one frame and darker shading in the next frame, and pixels between the temporally adjacent frames are compared for the moving block to determine if there is a pattern in pixel intensity differences for pixels that in blocks of each neighboring frame. With the shading assisted prediction, true motion can be captured, since the shading compensation will bias the selection process towards the direction of the motion. - The shading-assisted predication operation in some implementations does not replace regular (also referred to as standard) intra or inter prediction operations. It provides additional candidates to assess during an RDO process. Hence in some implementations, the predicted pixel or block will only be chosen when it has the smallest rate-distortion cost.
-
FIG. 4 is a block diagram illustrating aspects of theencoding device 100. As shown in this example, theintra-prediction detector 104 includes aspatial trend detector 400 andpixel predictor 402. Thespatial trend detector 400 receives thereconstructed pixels 142, such as non-filtered reconstructed pixels, and providesintensity trend data 404, such as projected points or extrapolated points along a trend line based on neighboring reconstructed pixels such as points A0 and A1 fromFIG. 3 . Thepixel predictor 402 produces the predicted reconstructed pixels by extrapolating thetrend data 404 and in this example, produces projected values P0, P1 and P2 to generate the predicted pixels C, D and E ofFIG. 3 . Theintra-prediction detector 104 also includes a standard intra-prediction unit 408 that performs conventional intra-prediction pixel generation which are output as predictedpixels 150 to themode decision unit 118 as with conventional systems. - The
inter-prediction detector 106 includes atemporal trend detector 410 and apixel predictor 412. Thetemporal trend detector 410 as noted above, while assessing a candidate motion vector and associated pixel blocks 1002 and 1004 among multiple neighboring frames, that represents the trajectory of motion to see if there is a pattern of pixel intensity differences, also referred to as a shading trend. If a trend is detected, a formula that represents the trend such as a linear formula or non-linear formula is applied during the inter-prediction process to generate predicted blocks based on the pattern of pixel intensity differences among blocks from temporally neighboring frames that are identified by the candidate motion vector. The trend can be formulated as a scaling and/or an offsetting operation as set forth above. - If a disruption of the projection process is observed due to sudden change in pixel values, the case is considered a different object (for intra prediction) or scene change (for inter prediction), and hence the projection process is limited to only legitimate pixels that represent a good reference. In other words, the
mode decision unit 118 eliminates a subset of the neighboring pixels from the shading detection process if (for example) their intensity is significantly higher/lower than the surrounding trend, and hence it is considered an outlier. - In some implementations, the detected spatial pattern of pixel intensity differences is implicit and is determined by the encoder (and the decoder) by analyzing neighboring reconstructed pixels. In other implementations, the spatial pattern of pixel intensity differences is explicitly defined through shading parameters or data representing formulas that are provided and signaled into the bitstream to an encoder, such as by setting an “explicit” flag at the block level indicating that a formula is being sent with the encoded block.
- The
inter-prediction detector 106 also includes a conventional (standard)inter-prediction unit 424 that provides candidate temporal predictedblocks 152 for differing angular modes as known in the art. Thebitstream data 416 in one implementation, is produced by the encoder to include encodingmode type data 430 such as data indicating whether a block is an intra-predicted block versus an inter-predicted block as well as data representing for the intra-predicted block type, the angular mode that has been used for that particular block.Data 432 is data that is output by the encoder in a bitstream indicating that an intensity difference pattern prediction operation was used in the encoding process for the block. This is referred to as an intensity pattern based prediction flag. The data encoded using the detected pattern of pixel intensity differences is shown as encodeddata 434. In this example, the prediction of pixels is done in an implicit manner using the detected spatial or temporal pattern of pattern of pixel intensity differences such that the decoder will also include corresponding spatial or temporal trend detectors and pixel predictors as well as a reconstructed frame decoder so that the decoder can carry out the same operations as the encoder in terms of producing the reconstructed block. - In other examples, the bitstream includes
data 436 representing the trend extrapolation formula so that the decoder need not implicitly determine the spatial pattern of pixel intensity differences that was employed by the encoder and instead is provided with the linear formula or best fit curve formula directly in the bitstream. This includes sending more data with each block so that there is a tradeoff between the complexity of decoder and bandwidth being used to communicate the encoded information. -
FIG. 5 illustrates an example of a method for encoding an image frame according to one example. The method as shown includes both intra-prediction with spatial based intensity difference pattern detection and inter-prediction detection with temporal based intensity difference pattern detection. The method in one example is carried out by the system as shown inFIG. 1 . As shown inblock 500, themode decision unit 118 performs a quality test on theintra-predicted reconstruction pixels 154 for all angular modes and the inter-predictionreconstructed pixels 156 for all candidate motion vectors that are provided by thedetectors - As shown in
block 502, the method includes determining which of the predictedblocks - For example, the
mode decision unit 118 performs RDO analysis on all predicted blocks that are presented to it and outputs as the final candidate predictedblock 128, the block with the best encoding characteristics. For example, if the intra or inter-frame intensity pattern difference detection based predicted blocks meet the quality threshold, the corresponding trend based compression mode is selected as the mode to use to encode the block from the current frame or if a standard inter-prediction or standard intra-prediction block is the best candidate one of them is chosen and as shown inblock 206 the encoder encodes the block of pixels using the predicted block of reconstructed pixels by for example encoding the residuals resulting between the current block of pixels and the best candidate predicted block using any suitable encoding technique. - As such, the appropriate predicted block and mode is selected by the
mode decision unit 118 as the final predicted mode to use for the block of interest from the current frame and the block is encoded as shown in 206 based on the prediction mode. -
FIG. 6 is a block diagram illustrating one example of aspects of theencoder 100. Theintra-prediction detector 104 includes neighboringspatial pixel selector 600 which for example receives the pixel coordinate 602 of the neighboring lines in the appropriate angular mode and in some cases can include any suitable number of lines that are used to determine whether a pattern exists. For example, referring toFIG. 2 , the predicted pixel coordinateinformation 602 includes the coordinate for pixels A and B and thecorresponding pixel intensities 604 are provided to the neighboring pixel spatialintensity trend detector 400. The pixel intensities 604 can be stored in any suitable memory if desired. Similarly, theinter-prediction detector 106 also receives a predicted pixel coordinates 608 corresponding to pixel coordinates in temporally neighboring frames. Any suitable number of frames can be employed. The correspondingpixel intensities 610 are provided to the neighboring pixel temporalintensity trend detector 400 which attempts to determine whether a pattern exists by applying best fit curve analysis for example. - Also shown is angular mode select data 614 which is provided by the encoder to allow for predicted blocks to be generated for each angular mode as desired. For example, the encoder attempts to detect spatial patterns of pixel intensity information for blocks in each of a plurality of angular modes. The results if any are then output as candidate predicted blocks 154.
- The encoding device can be implemented in a computing environment that includes multiple central processing (CPU) cores, multiple GPU cores or a hybrid multi-core, CPU/GPU system such as in a cloud server, laptop computer, desktop computer, handheld device, HD television or any suitable device. In some implementations, the encoder includes a processor executing firmware, in other implementations the encoder is implemented by the GPU with driver and API communication with an operating system as known in the art. In other implementations, the encoder is implemented as discrete logic including one or more state machines, program processors, field programmable gate arrays, or any suitable combination of processors executing software in combination with hardware logic.
-
FIG. 7 is a block diagram illustrating one example of adecoder 700 that attempts to perform implicit detection of spatial pattern of pixel intensity differences from reference frames or reconstructedframes decoder 700 receives the encodeddata 116 as a bitstream and in this example, receives thedata 416. The encoded data is passed through anentropy decoder 704 aninverse quantization stage 706 andinverse transform stage 708 and provided to acombiner 710 as known in the art. The encodingmode type data 430 informs the decoder as to whether to select 712 intra-frame decoding or inter-frame decoding as well as the angular mode for decoding intra-predicted pixels or motion vector for inter-predicted pixels. Thedata 432 informs the decoder that the decoder must undergo spatial/temporal pattern of pixel intensity difference detection since the encoded block has been encoded using this type of prediction. In some implementations, theinformation 416 is provided on a per block basis. As such, as a block is being decoded, the decoder selects the appropriate intra-prediction or inter-prediction spatial/temporal pattern pixelintensity difference detectors intra-prediction detector 104 andinter-prediction detector 106 in operation. The decoder includes adeblocking stage 720 as known in the art to provide filtering. - As such, the decoder includes similar spatial trend and temporal trend detection logic. If a
flag bit 432 relating to the intensity pattern based prediction flag is set on a per-block basis (which includes groups of blocks) the decoder detects the trend and performs the extrapolation fromreconstructed pixels 711. The intensity pattern basedprediction flag 432 in some examples indicates whether a neighboring temporal trend detection should be used on a particular encoded block or if a spatial pattern of pixel intensity differences detection should be performed on a particular block. In some implementations, thespatial trend detector 400,pixel predictor 402, thetemporal trend detector 410 andpixel predictor 412 are duplicated as corresponding units in the decoder. -
FIG. 8 is a block diagram illustrating one example of thedecoder 700 that includes aspatial trend detector 400,pixel predictor 402,temporal trend detector 410,pixel predictor 412 and adecoder unit 800 that produces decodedblocks 802 using reconstructedframes 702 and the predictedblocks 154/156 to produce decoded blocks 802. The decoder performs the corresponding same operations as the encoder as needed. For example, it will be recognized that the decoder need not perform angular mode predictions and choose among them since the decoder need only perform spatial trend detection for the angular mode signaled by the encoder in the bitstream. Similarly, the decoder performs temporal trend detection for only one motion vector (the one that the encoder chose and signaled into the bitstream). - In certain implementations, the
decoder 700 performs inter-prediction-based decoding by generating reconstructed temporally neighboring frames corresponding to a motion vector as reconstructedframes 702 and also performs intra-prediction based decoding on encoded blocks that were encoded using spatial intensity pattern differences by theencoder 100. For inter-prediction encoded blocks, the decoder determines whether blocks of pixels in the temporally neighboring reconstructed frames correspond to a motion vector that in some implementations is provided asinformation 416 for a block to determine if the blocks have a pattern of pixel intensity differences among the blocks from the temporally neighboring frames, in a similar manner as the encoder. The decoder produces the predictedblocks 156 for a reconstructed frame based on the determined pattern of pixel intensity differences. - In some implementations, the
decoding device 700 includes a block-based decoder that receives a stream of encodedvideo 116 and decodes an encoded block of pixels that is encoded using a predicted block of reconstructed pixels that is predicted based on a detected spatial pattern of pixel intensity differences from neighboring reconstructed pixels in a reconstructed frame and/or predicted based on detected temporal patterns of reconstructed blocks among temporally neighboring frames. - In this example, the block-based decoder receives data 432 (e.g., on a per block, per group of blocks, per slice or per frame as desired) indicating that a spatial pattern of pixel intensity differences was used to encode a block or blocks of the encoded video, and decodes the encoded block of pixels based on the data. For example, when the
data intra-prediction detector 714 and in particular thespatial trend detector 400 andpixel predictor 402 in the decoder, and outputs the decodedblock 802 that includes the predicted pixels. The predicted pixel is added with the residual data sent by the encoder to generate the reconstructed pixel. - When the
data inter-prediction detector 716 and in particular thetemporal trend detector 410 andpixel predictor 412. The decoder produces predicted blocks for a reconstructed frame based on the determined pattern of pixel intensity difference among temporally neighboring frames. - In certain examples, the block-based decoder performs a best fit curve analysis on differences among the neighboring reconstructed pixels and apply the determined best fit curve to predict pixel intensities for pixels in the predicted block of reconstructed pixels. In some examples, the block-based decoder detects linear pixel intensity differences among the plurality of neighboring reconstructed pixels for a plurality of differing angular prediction modes and apply the detected linear differences to predict pixels for the block of reconstructed pixels. In certain examples, the block-based decoder receives data in the encoded video indicating a type of pixel intensity difference pattern, such as formula defining the pattern that was used to encode data, and uses the indicated type of pixel intensity difference pattern for prediction of pixel data in a reconstructed frame when decoding the encoded video.
- In some implementations, the decoder tests all possible models until it finds the most descriptive one (like the encoder). In other implementations, bits are sent in the bit stream to the decoder to signal in the bitstream the model that was used (e.g., linear), so that the decoder does not have to do a lot of compute for all possible models. This model type is different than the model parameters. The model parameters can be either explicit (parameters sent in the bitstream), or implicit (decoder calculates it the same way encoder did). In contrast to some prior systems in some implementations, different sets of pixels are used to calculate the temporal trending model such as corresponding blocks of pixels that correspond to the trajectory of the motion vector for the candidate block of pixels to be predicted—as opposed to using spatially adjacent pixels to the candidate block and corresponding spatially located pixels to a temporal block corresponding to the candidate block to be predicted. Also, unlike some other systems, the disclosed prediction process is scalable with block size so that eh the size of the pixels blocks used for trend derivation can scale with the size of compute units. It will be recognized that subsampled version of pixels can be used to determine trends as well.
-
FIG. 9 illustrates an embodiment of acomputing system 900 such as a cloud platform that provides encoded video. In this example the system uses GPU 192 and associated drivers to provide theencoder 100 as described herein. In general, thecomputing system 900 is embodied as any of a number of different types of devices, including but not limited one or more cloud servers, or other devices that includes aCPU 912,GPU 922 andmemory 906 and may be a system on-chip, integrated circuits, multipackage device etc. In this example, thecomputing system 900 includes a number of components that communicate with each other through abus structure 912. Incomputing system 900, each of the components is capable of communicating with any of the other components either directly through thebus 912, or via one or more of the other components. The components incomputing system 900 in some implementations are contained within a single physical enclosure, such as a server chassis, laptop or desktop chassis, or a mobile phone casing or in some implementations the display is remote from a system on-chip or other configuration. In alternative embodiments, some of the components ofcomputing system 900 are embodied asperipheral devices 908 such that theentire computing system 900 does not reside within a single physical enclosure. - In some implementations, the
computing system 900 also includes user interface devices for receiving information from or providing information to a user. Specifically, thecomputing system 900 includes aninput device 902, such as a keyboard, mouse, touch-screen, or other device for receiving information from the user. Thecomputing system 900 in some implementations displays information to the user via aphysical monitor 911, such as light-emitting diode (LED) display, liquid crystal display, or other output device. However, such devices need not be employed. The GPU usesframe buffer memory 910 to store frames of created graphics images or other image data for encoding by the encoder. - In certain implementations,
computing system 900 additionally includes anetwork adapter 907 for transmitting and receiving data over a wired or wireless network to a device that includes adecoder 700. The device can be any suitable device.Computing system 900 also includes one or moreperipheral devices 908. Theperipheral devices 908 may include mass storage devices, location detection devices, sensors, input devices, or other types of devices used by thecomputing system 900. - The
processing unit 912 receives and executes instructions that are stored in amemory system 906 such as gaming applications or other applications. In one embodiment, theprocessing unit 912 includes multiple processing cores that reside on a common integrated circuit substrate.Memory system 906 includes memory devices used by thecomputing system 900, such as random-access memory (RAM) modules, read-only memory (ROM) modules, hard disks, and other non-transitory computer-readable media. Some of the memory devices are used asmemory 906 for theprocessing unit 912. - Some embodiments of
computing system 900 may include fewer or more components than the embodiment as illustrated inFIG. 9 . For example, certain embodiments are implemented without anydisplay 911 orinput devices 902. Other embodiments have more than one of a particular component, for example, an embodiment ofcomputing system 900 could havemultiple processing units 912, buses,network adapters 907,memory systems 906, etc. - Although features and elements are described above in particular combinations, each feature or element can be used alone without the other features and elements or in various combinations with or without other features and elements. The apparatus described herein in some implementations use software or firmware incorporated in a non-transitory computer-readable storage medium for execution by a general-purpose computer or a processor. Examples of computer-readable storage mediums include a read only memory (ROM), a random-access memory (RAM), a register, cache memory, semiconductor memory devices, magnetic media such as internal hard disks and removable disks, magneto-optical media, and optical media such as CD-ROM disks, and digital versatile disks (DVDs).
- In the preceding detailed description of the various embodiments, reference has been made to the accompanying drawings which form a part thereof, and in which is shown by way of illustration specific preferred embodiments in which the invention may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the invention, and it is to be understood that other embodiments may be utilized, and that logical, mechanical and electrical changes may be made without departing from the scope of the invention. To avoid detail not necessary to enable those skilled in the art to practice the invention, the description may omit certain information known to those skilled in the art. Furthermore, many other varied embodiments that incorporate the teachings of the disclosure may be easily constructed by those skilled in the art. Accordingly, the present invention is not intended to be limited to the specific form set forth herein, but on the contrary, it is intended to cover such alternatives, modifications, and equivalents, as can be reasonably included within the scope of the invention. The preceding detailed description is, therefore, not to be taken in a limiting sense, and the scope of the present invention is defined only by the appended claims. The above detailed description of the embodiments and the examples described therein have been presented for the purposes of illustration and description only and not by limitation. For example, the operations described are done in any suitable order or manner. It is therefore contemplated that the present invention covers any and all modifications, variations or equivalents that fall within the scope of the basic underlying principles disclosed above and claimed herein.
- The above detailed description and the examples described therein have been presented for the purposes of illustration and description only and not for limitation.
Claims (20)
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US17/215,347 US11831900B2 (en) | 2021-03-29 | 2021-03-29 | Video encoding/decoding using detected pattern of pixel intensity differences |
US18/493,153 US20240056600A1 (en) | 2021-03-29 | 2023-10-24 | Video encoding/decoding using detected pattern of pixel intensity differences |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US17/215,347 US11831900B2 (en) | 2021-03-29 | 2021-03-29 | Video encoding/decoding using detected pattern of pixel intensity differences |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US18/493,153 Continuation US20240056600A1 (en) | 2021-03-29 | 2023-10-24 | Video encoding/decoding using detected pattern of pixel intensity differences |
Publications (2)
Publication Number | Publication Date |
---|---|
US20220312029A1 true US20220312029A1 (en) | 2022-09-29 |
US11831900B2 US11831900B2 (en) | 2023-11-28 |
Family
ID=83365198
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US17/215,347 Active US11831900B2 (en) | 2021-03-29 | 2021-03-29 | Video encoding/decoding using detected pattern of pixel intensity differences |
US18/493,153 Pending US20240056600A1 (en) | 2021-03-29 | 2023-10-24 | Video encoding/decoding using detected pattern of pixel intensity differences |
Family Applications After (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US18/493,153 Pending US20240056600A1 (en) | 2021-03-29 | 2023-10-24 | Video encoding/decoding using detected pattern of pixel intensity differences |
Country Status (1)
Country | Link |
---|---|
US (2) | US11831900B2 (en) |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20070171978A1 (en) * | 2004-12-28 | 2007-07-26 | Keiichi Chono | Image encoding apparatus, image encoding method and program thereof |
US20140112392A1 (en) * | 2011-07-01 | 2014-04-24 | Sk Telecom Co., Ltd. | Apparatus and method for coding and decoding image |
US20170251227A1 (en) * | 2014-11-14 | 2017-08-31 | Huawei Technologies Co., Ltd. | Systems and methods for mask based processing of a block of a digital image |
US20180160113A1 (en) * | 2015-06-05 | 2018-06-07 | Intellectual Discovery Co., Ltd. | Method and device for encoding and decoding intra-frame prediction |
-
2021
- 2021-03-29 US US17/215,347 patent/US11831900B2/en active Active
-
2023
- 2023-10-24 US US18/493,153 patent/US20240056600A1/en active Pending
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20070171978A1 (en) * | 2004-12-28 | 2007-07-26 | Keiichi Chono | Image encoding apparatus, image encoding method and program thereof |
US20140112392A1 (en) * | 2011-07-01 | 2014-04-24 | Sk Telecom Co., Ltd. | Apparatus and method for coding and decoding image |
US20170251227A1 (en) * | 2014-11-14 | 2017-08-31 | Huawei Technologies Co., Ltd. | Systems and methods for mask based processing of a block of a digital image |
US20180160113A1 (en) * | 2015-06-05 | 2018-06-07 | Intellectual Discovery Co., Ltd. | Method and device for encoding and decoding intra-frame prediction |
Also Published As
Publication number | Publication date |
---|---|
US20240056600A1 (en) | 2024-02-15 |
US11831900B2 (en) | 2023-11-28 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US10051273B2 (en) | Video decoder and video decoding method | |
JP7104188B2 (en) | Block size limit for DMVR | |
CN110612553B (en) | Encoding spherical video data | |
US20180054613A1 (en) | Video encoding method and apparatus with in-loop filtering process not applied to reconstructed blocks located at image content discontinuity edge and associated video decoding method and apparatus | |
KR100772576B1 (en) | Method for encoding and decoding video information, a motion compensated video encoder and a corresponding decoder | |
US8098731B2 (en) | Intraprediction method and apparatus using video symmetry and video encoding and decoding method and apparatus | |
JP2020526109A (en) | Motion vector refinement for multi-reference prediction | |
KR20210072064A (en) | Inter prediction method and apparatus | |
TW201902214A (en) | Sub-prediction unit temporal motion vector prediction for video codec | |
KR20230043079A (en) | Methods of decoding using skip mode and apparatuses for using the same | |
GB2487261A (en) | Motion compensated image coding using diverse set of motion predictors | |
KR20200002036A (en) | Optical Flow Estimation for Motion Compensated Prediction in Video Coding | |
JP7462740B2 (en) | Image encoding/decoding method and device performing PROF, and method for transmitting bitstream | |
CN112291571A (en) | Video decoding method and device, computer equipment and storage medium | |
CN110741641B (en) | Method and apparatus for video compression | |
JP2023107799A (en) | Method and device for decoding and encoding video picture | |
US20230199175A1 (en) | Method and device for subpicture-based image encoding/decoding, and method for transmitting bitstream | |
JP2023179684A (en) | Picture prediction method and device, and computer readable storage medium | |
WO2020247577A1 (en) | Adaptive motion vector resolution for affine mode | |
US10979704B2 (en) | Methods and apparatus for optical blur modeling for improved video encoding | |
US11831900B2 (en) | Video encoding/decoding using detected pattern of pixel intensity differences | |
US11729424B2 (en) | Visual quality assessment-based affine transformation | |
CN117501694A (en) | Motion refinement with bi-directional matching for affine motion compensation in video codec | |
US11089308B1 (en) | Removing blocking artifacts in video encoders | |
US20210185323A1 (en) | Inter prediction method and apparatus, video encoder, and video decoder |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: ATI TECHNOLOGIES ULC, CANADA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:LIU, HAIBO;AMER, IHAB;REEL/FRAME:055751/0483 Effective date: 20210324 |
|
FEPP | Fee payment procedure |
Free format text: ENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: BIG.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: FINAL REJECTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: PUBLICATIONS -- ISSUE FEE PAYMENT VERIFIED Free format text: AWAITING TC RESP, ISSUE FEE PAYMENT VERIFIED |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |