Classifications

• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
  • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
  • H04N19/51—Motion estimation or motion compensation
  • H04N19/533—Motion estimation using multistep search, e.g. two-dimensional [2D]-log search or one-at-a-time search [OTS]
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
  • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
  • H04N19/103—Selection of coding mode or of prediction mode
  • H04N19/105—Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
  • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
  • H04N19/154—Measured or subjectively estimated visual quality after decoding, e.g. measurement of distortion
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
  • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
  • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
  • H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
  • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
  • H04N19/51—Motion estimation or motion compensation
  • H04N19/573—Motion compensation with multiple frame prediction using two or more reference frames in a given prediction direction
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
  • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
  • H04N19/51—Motion estimation or motion compensation
  • H04N19/577—Motion compensation with bidirectional frame interpolation, i.e. using B-pictures
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
  • H04N19/61—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding in combination with predictive coding

Definitions

• the invention relates to the general field of image coding and more particularly to that of inter-image prediction.
• Inter-image prediction is about taking advantage of the time redundancies that exist between consecutive frames of a video to achieve high compression rates of that video.
• the principle of inter-image prediction is to divide a current image into a block or macroblock. Then, the coder finds a similar block in another image (previous or future) of the video. This other image is usually called a reference image. The encoder then encodes a motion vector that defines the position of the block found in said reference image (s) from the block to be predicted. The encoder then calculates the difference between these two blocks and codes the prediction error. The motion vector and the prediction error are then transmitted to the decoder which can thus reconstruct the block.
• Video coding / decoding schemes that use this type of method. These include MPEG-2 standards (ISO / IEC JTC1 / SC29 / WG1 1 MPEG00 / October 2000, Coding of moving pictures and audio), MPEG-4 / AVC (T. Wiegand, GJ Sullivan, G. Bjontegaard , A. Luthra, "Overview of the H.264 / AVC” Circuits and Systems for Video Technology, IEEE Transactions, Vo 13.7, 560-576, July 2003, or HEVC (ITU-T Q.6 / SG and ISO / IEC Moving Picture Experts Group (ISO / IEC JTC 1 / SC 29 / WG 1 1).
• MPEG-2 standards ISO / IEC JTC1 / SC29 / WG1 1 MPEG00 / October 2000, Coding of moving pictures and audio
• MPEG-4 / AVC T. Wiegand, GJ Sullivan, G. Bjontegaard , A. Luthra, "Over
• blocks or more generally zones
• the definition of blocks (or more generally zones) for predicting a block is preponderant for the coding efficiency. Indeed, if the contents of the current block and the prediction block are very different, the prediction error will be significant which will induce a large number of bits to code this prediction error. It is therefore necessary to minimize the risks of choosing remote prediction zones in terms of the content of the block to be predicted.
• the cost of coding the syntax elements required by the remote decoder to reconstruct a predicted image is relatively high.
• the reference images are grouped into two lists: one grouping together images (decoded or reconstructed) temporally prior to an image to which the block to be predicted belongs and that grouping together images (decoded or reconstructed) posterior temporally. Subsequently, when we speak of images before and / or later temporally we will imply that these images are decoded and reconstructed.
• the invention aims to overcome at least one of the disadvantages of the prior art and in particular to improve the efficiency of interimage coding processes.
• the invention relates to a method for predicting a block of pixels of an image which comprises:
• a step of defining a causal neighborhood of said block to be predicted a step of searching for candidate patches in the course of which a set of candidate patches is formed of at least one patch belonging to an image other than the image to be which belongs the block to predict, each patch being formed of a block and a neighborhood which is causal of this block and
• the method reduces the elements of syntax that it is necessary to transmit to the decoder, such as those relating to the MPEG-4 / AVC standard.
• the block is predicted from a block of a patch of said set of candidate patches, said block being close, in terms of content, of the block to to predict.
• the block prediction step comprises
• a block prediction step during which the block of pixels of the image is predicted by a weighted linear combination of the pixels of the blocks of the patches of a dictionary, the weighting parameters of said linear combination being the optimal ones which have determined during the neighborhood prediction step.
• the prediction of the block is determined by a linear combination of blocks belonging to a dictionary.
• the weighting parameters of this linear combination are those which make it possible to obtain the best prediction of the causal neighborhood of the block to be predicted in the sense of a distance.
• the prediction error of the block is reduced because the weighting parameters are defined to minimize a prediction error of a zone (neighborhood) located around the block to be predicted and not directly a prediction error of this block thus favoring a continuity of the content of the images.
• the present invention also relates to a coding / decoding method which implements this method as well as an image sequence coding / decoding device and apparatus which comprises means for implementing the method. It also relates to a signal whose frame is particular because it carries specific information that influences the operation of said device and / or coding device and / or decoding.
• FIG. 1 shows a diagram of the steps of the method of predicting a block of pixels of an image according to the present invention
• FIG. 3 illustrates an embodiment of the block prediction step of the method
• FIG. 4 illustrates another embodiment of the block prediction step of the method
• FIG. 5 illustrates the case where the patches of a dictionary belong to the same image
• FIG. 6 illustrates the case where the patches of a dictionary do not all belong to the same image
• FIG. 7 illustrates the case where dictionaries defined from the first patch
• FIG. 8 illustrates the case where patches of a dictionary do not all belong to the same image
• FIG. 9 schematically represents an exemplary architecture of a device embodying the invention. 5. Detailed description of the invention
• Figure 1 shows a diagram of the steps of the method of predicting a block of pixels of an image according to the present invention.
• the method comprises a step 1 of defining a neighborhood V causal of a block s to predict a current image l c .
• Figure 2 gives an example of a definition of causal neighborhood.
• the neighborhood V is formed, for example, of three blocks to the left and above the block to be predicted B.
• a neighborhood is a vector of N values, each value corresponding to the value of a pixel belonging to this neighborhood.
• the invention is in no way limited to this definition of neighborhood but on the contrary extends to any definition of causal neighborhood that is to say to any neighborhood that is available at a decoder before the decoding of the block current to predict.
• patch denoted by X k
• X k the term patch, denoted by X k
• the neighborhood V k is causal of this block B k and has a shape identical to that of the neighborhood V situated around the block to be predicted B.
• the causality of a neighborhood with respect to a block of pixels indicates that the pixel values are known. prior to the prediction of this block.
• the patch X to designate the grouping of the pixels of the block to predict B and pixels of the neighborhood V.
• the method also comprises a step 2 of searching for candidate patches during which a set of candidate patches PS is formed of at least one patch x k belonging to an image other than the image to which belongs the block to predict.
• the method further includes a block prediction step 3 in which block B is predicted from, at least, the block of at least one patch of said PS set.
• each patch x k of the set of candidate patches is such that its neighborhood V k is close, in terms of content, to the neighborhood V of the patch X.
• a patch X k of the set of candidate patches is such that it verifies equation (1):
• the set PS groups the N candidate patches that minimize the Euclidean norm given in equation (1). Distances other than Euclidean norm may be used without departing from the scope of the invention.
• each said other image is considered a predetermined number of candidate patches.
• This embodiment is advantageous because by limiting the number of candidate patches per image and the number of images, the computational complexity of the neighborhood (and block) prediction step is greatly reduced while preserving the homogeneity of the content often present in an image or in consecutive images.
• only one candidate patch is thus chosen per image.
• patches that belong to a predetermined zone SW defined on one or more images are considered as candidate patches.
• the predetermined zone (or part of it called the sub-zone) is defined around a patch.
• the predetermined zone SW can be defined on a single image in the form of a spatial region but can also have a temporal character, that is to say that this predetermined zone SW is defined on several images, images which can or not be temporally consecutive.
• this zone is defined in FIG. 2, on the one hand, by a sub-area of an image / c-1 anterior temporally to the image 1c (to which the block to predicting), said sub-area being centered around the patch X (whose co-localized is represented on this dashed image) and, secondly, by a sub-zone of another image l c + 1 posterior temporally in the picture I c .
• the predetermined zone SW can be composed of one or more sub-zone (s), each sub-zone can be defined around a co-localized of this patch X, possibly centered around it, and that these sub-areas may be located in different images prior to and / or subsequent to the image to which the block to be predicted belongs.
• the block B is predicted from the block B opt of a patch of said set of candidate patches, said block being close, in terms of content, the block to predict.
• this prediction of the block to be predicted B consists of determining a patch of the set PS which minimizes a distance between the values of the pixels of the block B k of this patch and the values of the pixels of the block B (mapping of block).
• This embodiment is particularly advantageous because the coding cost of the index of the block B opt or the patch to which it belongs in the set PS is reduced compared to the coding cost of the syntax elements of the coding systems and / or usual decoding.
• this distance is expressed in Euclidean space by minimization, in the least squares sense, expressed by equation (2):
• an information (index) designating the patch to which belongs the prediction block B opt must be known from a remote decoder. Indeed, such a decoder can reconstruct all PS candidate patches and can, from this information, find what is the patch of this set to which this block B opt . For this purpose, a signal carrying this designation information of this patch is sent to a decoder intended to use this prediction block.
• the block prediction step 3 comprises a dictionary definition sub-step 31 during which L D 1 dictionaries are formed (L is greater than or equal to 1).
• Each dictionary consists of at least one patch of the set of PS candidate patches.
• the number L of dictionaries and the number of patches per dictionary are known values a priori.
• the number K of patches of each dictionary is common to all the dictionaries.
• the number K is variable according to the block to be predicted.
• this number K can be optimized for each block to be predicted. It is then necessary, in a transmission context between transmitter / receiver to transmit this number to the receiver for each block to be predicted.
• a dictionary can group randomly selected patches from those in the PS candidate patch set.
• the block prediction step also includes a substep 32 of neighborhood prediction. During this sub-step, for each dictionary D 1, a prediction of the causal neighborhood V of the block to be predicted B is determined by a weighted linear combination of the neighborhoods V k of the patches X k of this dictionary, weighting parameters that optimize the prediction are then retained.
• the prediction of the causal neighborhood V of the block to predict B by a weighted linear combination of the neighborhoods V k of the patches X k of a dictionary Di consists in determining weighting parameters W m with m G ⁇ 0; K - 1 ⁇ which minimize a distance between the weighted values of the pixels of the neighborhoods V k of the patches of this dictionary D l and the values of the pixels of the neighborhood V.
• this distance is expressed in Euclidean space by minimization, in the least squares sense, expressed by equation (3):
• the weighting parameters W used to predict the block B are those which provide the closest prediction, in the sense of a criterion , said block to predict.
• this criterion is a quadratic error between the reconstructed predicted block (after coding and decoding) and the block to be predicted.
• the criterion used is a rate-distortion (Rate-distortion) criterion particularly adapted to the context of video compression.
• SSE 1 a least squares measurement of the reconstruction error between the block to be predicted and the reconstructed predicted block (decoded block), R 1 the block coding cost (prediction error and other syntax elements), and ⁇ the Lagrangian.
• the block prediction step includes a block prediction sub-step 33 during which the block B is predicted by a weighted linear combination of the pixels of the blocks B k of the patches X k of the dictionary D 1 , the weighting parameters (W if several dictionaries formed or w vt in the case of a single dictionary) being those which were determined during the step of predicting the causal neighborhood of the block to be predicted.
• the prediction B of block B is given by equation (7):
• A is a matrix of dimension PxK which group together the P values of the pixels of K blocks B k , and W the weighting parameters.
• no particular information is to be transmitted to the receiver (decoder) to predict the block B in the case where the number of parameters to be used is previously known to the decoder and in the case of 'a single dictionary built on the basis of neighborhood only.
• the prediction method can be implemented by the receiver without particular information because, on the one hand, the neighborhoods used by the prediction are causal, which allows the receiver to find the blocks of the patches to reconstruct the matrix A and, on the other hand, by implementing the prediction of the neighborhood V, the K weighting parameters then obtained are identical to those (W) obtained during the neighborhood prediction sub-step then implemented by the transmitter (encoder).
• a coding method implementing this prediction method provides significant coding gains compared to traditional inter-picture coding techniques such as those used for example in H.264 / AVC.
• the K patches X k with k G ⁇ 0; K-1 ⁇ of a dictionary D l are all located in the same image I f other than the current image I c .
• the image I f may be anterior or posterior temporally to image 1c when these two images belong to the same sequence of images.
• the K patches X k with ke ⁇ 0; K - l ⁇ of a dictionary D l are located in different images. According to the example of FIG.
• the dictionary D 1 has (K-1) patches ⁇ ⁇ , ..., X K -i in an image l c _ x anterior temporally to the current image I c and a patch X 0 in an image I c + 1 posteriorly temporal to the current image l c .
• This embodiment is advantageous because it makes it possible to multiply the possibilities of patches in the same dictionary which can thus belong to different images. This makes it possible to further reduce the prediction error of the block to be predicted because the method then takes advantage of temporal redundancies between images of the same video.
• FIGS. 5 and 6 in no way limit the dictionary definition. They have been given to illustrate that a dictionary may be formed by patches located in one or more image other than that to which the block to be predicted belongs.
• each dictionary D 1 it is determined, for each dictionary D 1 to be defined, firstly a first patch X 0 among the patches of the set PS, said first patch is close, in terms of content, patch X and, secondly, (K-1) patches X k among the patches of the set PS, each of them being close, in terms of content, of this first patch X 0 .
• the dictionary D l then groups the first patch X 0 and the (K-1) patches X k .
• the proximity of the contents of two patches is quantified by a calculated distance between the pixel values of the patches. This distance is, for example, the sum of the absolute differences between the pixels of these two patches.
• the predetermined zone SW is defined by at least one sub-zone which is located around a first patch. It can, for example, be centered around this first patch.
• the position of a first patch X 0 of a dictionary D 1 in an image is given by a displacement information d defined from the patch X.
• the displacement information d can, according to one embodiment, be obtained by a block matching method which makes it possible to determine a displacement of each first patch with respect to the X patch.
• block matching is similar to that described in relation to Figure 3 considering that the blocks are formed of all the pixels of the patches and not only the pixels of the blocks of these patches as described in the performing step 3 of block prediction.
• the displacement information In a transmission context between transmitter and decoder, the displacement information must be transmitted to the decoder and this in order that this decoder can determine what was the first patch used. It is not necessary to transmit other information to determine the other (K-1) patches of the dictionary because the decoder is able to determine them by implementing operations similar to those described above.
• FIG 7 illustrates the case where L dictionaries have been defined, each from a first patch.
• Each displacement is expressed as a vector d.
• FIG. 8 shows the case where dictionaries patches do not all belong to the same image. It may be noted that an image l c + 1 does not temporally precede the current image l c during the decoding of this current image. .
• the dictionary D 0 is formed of a first patch X% belonging to the image l c + 1 and patches that do not all belong to the image I c + 1 .
• a patch X ⁇ _ belongs to the image / c _ 2 .
• the dictionary D L - 1 is formed of a first patch X ⁇ '1 belonging to the image / c _ 2 and patches that do not all belong to the image / c _ 2 .
• a patch x ⁇ x belongs to the image / c - 1 .
• the distance that quantifies the proximity of the contents of two patches is to be taken in the broad sense because it can be defined to quantify the similarity between patches that do not necessarily belong to the same image.
• Figure 9 shows an exemplary architecture of a device comprising means configured to implement the invention described in relation to Figures 1 to 8.
• the device 900 comprises the following elements, interconnected by a digital addressing and data bus 901:
• a computing unit 903 also called Central Processing Unit in English
• a memory 905 A memory 905;
• the computing unit 903 can be implemented by a microprocessor, possibly dedicated, a microcontroller also possibly dedicated, etc.
• the memory 905 can be implemented in a volatile and / or nonvolatile form such as a RAM (Random Access Memory), a hard disk, an EPROM (Erasable Programmable ROM), etc.
• the means 903, 905 and possibly 904 cooperate with each other to define a causal neighborhood of a block to be predicted, to search for a set of candidate patches which is formed of at least one patch belonging to an image other than the image to which the block to be predicted, each patch being formed of a block and a neighborhood that is causal of that block.
• the means 903, 905 and possibly 904 cooperate with one another to predict a block starting, at least, from the block of at least one patch of said set of candidate patches.
• the means of the device are configured, according to one embodiment, to implement a method described in connection with Figures 1 to 8.
• the means 904 are configured to transmit and / or receive a signal whose frame is particular.
• the frame of this signal carries information which designates the patch to which the block of prediction of the block to be predicted belongs and, according to a variant which corresponds to the case where the means for predicting are configured to implement. implement a neighborhood prediction step as described in relation to Figure 4, the frame of this signal carries specific information that identifies the dictionary from which is derived the prediction of the block to predict.
• the frame of this signal carries a displacement information relative to the position of this first patch in an image, said displacement information being defined from the patch formed by the block to be predicted and its neighborhood.
• the invention also relates to a method for encoding and / or decoding an image sequence in which a prediction block is calculated from a reference image image block.
• the method is characterized in that the prediction block is calculated according to a method described in connection with Figures 1 to 8.
• the invention also relates to an apparatus for encoding and / or decoding an image sequence which is characterized in that it comprises a device described in relation with FIG. 9.
• the represented modules are functional units, which may or may not correspond to physically distinguishable units.
• these modules or some of them may be grouped into a single component or circuit, or constitute functionalities of the same software.
• some modules may be composed of separate physical entities.
• inter-image prediction devices compatible with the invention are implemented in a purely hardware ("hardware") embodiment, for example in the form of a dedicated component (for example in an ASIC or FPGA or VLSI) ( respectively "Application Specifies Integrated Circuit” in English, meaning “Integrated Circuit for a specific application”, “Field Programmable Gate Array” in English, meaning “In-Situ Programmable Gate Network”, “Very Large Scale Integration” in English, meaning “very large-scale integration") or several electronic components integrated in a device or in the form of a mixture of hardware and software elements ("software” in English).
• a dedicated component for example in an ASIC or FPGA or VLSI

Landscapes

• Engineering & Computer Science (AREA)
• Multimedia (AREA)
• Signal Processing (AREA)
• Compression Or Coding Systems Of Tv Signals (AREA)
• Compression, Expansion, Code Conversion, And Decoders (AREA)

Priority Applications (5)

Applications Claiming Priority (4)

Publications (1)

Family

ID=49261523

Family Applications (1)

Country Status (6)

Cited By (1)

Families Citing this family (3)

Citations (4)

Family Cites Families (11)

• 2013
  • 2013-09-25 JP JP2015533563A patent/JP6309010B2/ja not_active Expired - Fee Related
  • 2013-09-25 CN CN201380053391.7A patent/CN104756496B/zh not_active Expired - Fee Related
  • 2013-09-25 EP EP13770443.3A patent/EP2901697A1/fr not_active Withdrawn
  • 2013-09-25 KR KR1020157007678A patent/KR20150060730A/ko not_active Withdrawn
  • 2013-09-25 WO PCT/EP2013/069903 patent/WO2014048946A1/fr not_active Ceased
  • 2013-09-25 US US14/431,292 patent/US9374597B2/en not_active Expired - Fee Related

Patent Citations (4)

Non-Patent Citations (3)

Also Published As

Similar Documents

Legal Events