Classifications

• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T9/00—Image coding
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
  • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
  • H04N19/12—Selection from among a plurality of transforms or standards, e.g. selection between discrete cosine transform [DCT] and sub-band transform or selection between H.263 and H.264
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
  • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
  • H04N19/154—Measured or subjectively estimated visual quality after decoding, e.g. measurement of distortion
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/20—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using video object coding
  • H04N19/27—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using video object coding involving both synthetic and natural picture components, e.g. synthetic natural hybrid coding [SNHC]
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
  • H04N19/59—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
  • H04N19/61—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding in combination with predictive coding

Definitions

• the invention lies in the context of image synthesis and more particularly in the field of video compression.
• the synthesis method applies to the encoder and the decoder.
• the method consists of synthesizing the content of an image from texture patches, the patches in question being: • blocks of images of reduced dimensions,
• the rendering of the synthesis thus obtained is compared with the coded source; the parts of the reconstructed image that do not meet a quality level judged to be acceptable by the criterion are then encoded by a more conventional technique; as examples:
• the metric can be the ssim
• Figure 1 represents the principle of the algorithm. It has two inputs, a texture patch and an image with the desired dimensions, initialized by a noise to avoid periodicities. It outputs a synthesized image from the texture.
• the neighborhood consists of the pixels surrounding the current pixel, it is included in a given square of dimension [dxd]. It is called "causal" when it contains only the pixels already synthesized in the current image. It is therefore here causal neighborhoods that are used since the non-causal part of the neighborhood in the current image comprises only noise pixels and is not interesting for the comparison.
• Figure 2 shows such causal neighborhoods.
• the output image is periodised so the pixels taken into account are on the other side of the image as shown for the first corner pixel (x) and its neighborhood located at the four corners of the image.
• the main problem raised by the exhaustive approach remains the computation time necessary to synthesize images of reasonable size. This computing time is correlated to the size of the neighborhood, this multi-resolution approach will improve performance.
• the main idea introduced in [1] is to use images of lower resolutions so that 5x5 or 3x3 neighborhoods extend on the texture like 15x15 neighborhoods in single resolution. For that, we start by creating pyramids, one for the patch and one for the synthesized image using a sub-sampler filter, as shown in Figure 3.
• the algorithm then synthesizes the pyramid of the current image, from the lowest resolution to the highest resolution, as follows: • The lowest resolution image is synthesized in the same way as in the case of simple technical resolution.
• the other images are synthesized in the same way, except that the neighborhoods do not only contain the pixels of the current resolution, but also pixels of the neighborhood of the pixel corresponding to the current at the lower resolution.
• the last image is the output image synthesized from the patch and lower resolution images.
• Figure 4 shows a multi-resolution neighborhood.
• This neighborhood contains the pixels of the causal neighborhood of the current resolution of level n, represented in dark in the diagram on the left, plus the pixels contained in the non-causal neighborhood of the higher resolution of level n + 1, pixels represented in dark plus the parent in the center shown in more light, in the diagram on the right.
• Figure 5 shows the order of the multi-resolution synthesis.
• the upper image, level 2 corresponds to the synthesis of the first level, causal neighborhood.
• the lower images, level 1 and level 0 correspond to the synthesis of the second level, causal neighborhood.
• the object of the invention is to synthesize an image via texture patches for the purpose of image compression, it is of course necessary to estimate the quality of reproduction of the parts of synthesized images in comparison with the source image. (encoder side).
• These synthesis-based reconstruction techniques tend to implicitly generate a reconstructed signal that moves away from the original signal in terms of classical sse (sum of squared differences) distortion, but on the other hand offer a visual rendering that can be quite acceptable; it is here that we come up against quality metrics.
• SSIM Structural Similarity
• the SSIM is applied by 8x8 block in the image, relative to each pixel of the image.
• One of the aims of the invention is to overcome the aforementioned drawbacks. It relates to an image decoding method using a technique of image synthesis and image regions using a synthesis algorithm that operates on a set of patches, this operation being done by via a low resolution image, characterized in that it comprises the following steps:
• the synthesis technique is of the pyramidal type.
• the low resolution image is in a form of spatial scalability type so that the synthesis algorithm is punctually guided to pyramid levels other than the lower resolution level.
• the synthesis algorithm operates on an RGB image signal, a YUV image signal or a luminance signal Y alone, the U and V signals being subjected to the same processing as the applied luminance processing.
• the subject of the invention is also an image compression method using a technique for image synthesis and image regions using a synthesis algorithm that operates on a set of patches, this operation being carried out via a low resolution image, characterized in that it comprises the following steps:
• the synthesis technique is of the pyramidal type.
• the low resolution image is in a form of spatial scalability type so that the synthesis algorithm is punctually guided to pyramid levels other than the lower resolution level.
• the synthesis algorithm operates on an RGB image signal, a YUV image signal or a luminance signal Y alone, the U and V signals being subjected to the same processing as the applied luminance processing.
• the quality metric is the SSIM (Structural SIMilarity).
• the invention makes it possible to improve the synthesis of images and image regions by using a synthesis algorithm that operates on a set of patches, this operation being done via a low resolution image.
• the target application is video compression, a quality metric intervenes to classically code the areas of the poorly reconstructed image or leave the areas in question.
• a first advantage of the invention is thus to allow an acceptable visual rendering (based on quality metrics) of reconstructed image regions via a synthesis algorithm, this synthesis being guided to the encoder and decoder by a transmitted image of low resolution, in order to in the end, to reduce the bit rate with a given visual quality, and vice versa.
• this technique does not require a segmentation map as such to be transmitted to the decoder, the synthesis algorithm naturally operating the distribution of the information contained in the different patches via the guidance image. .
• the rendering imperfections by the synthesis technique are corrected by conventional coding which areas of imperfection are detected by a quality metric, this metric may be ssim.
• a second advantage of the invention is the scalability of the representation, which makes it possible to decode the signal at a chosen resolution.
• Another advantage is the possibility of coding the low resolution image according to an existing coding technique, for example H.264, thus ensuring backward compatibility with these coding techniques.
• the algorithm sub-samples the reference image as many times as there are stages in the Gaussian pyramid used in the multi-resolution algorithm. 2) This low resolution image is then copied as initialization of the synthesized image, replacing the proposed white initialization noise in the approach of LY Wei and M. Levoy. 3) Several patches corresponding to the different textured parts of the image are provided to the algorithm. 4) The low resolution image is then synthesized with a (non-causal) square neighborhood: the non-causal part of the neighborhood computed on the image being constructed then rests on the subsampled reference image. The exhaustive algorithm then tests all the neighborhoods of all the patches provided. The non-causal part of the current neighborhood will then guide the synthesis to the patch that has the characteristics closest to the current part of the subsampled image. 5) The algorithm keeps in memory which patch comes from each synthesized pixel.
• the synthesized image of dimensions 768x512, represented in FIG. 9, is obtained by this algorithm with the following characteristics: • Voisinages of current resolution: 5 ⁇ 5 pixels
• Associated metric In order to measure whether the texture synthesis is relevant to the regions of the image produced, a quality metric is used that can reveal the rendering of the structure.
• Figure 11 shows the general block diagram of the coding method.
• the applications concerned are those related to video compression. More specifically, very low and low speed applications (ex HD for mobile) as well as super resolution (HD and +).

Landscapes

• Engineering & Computer Science (AREA)
• Multimedia (AREA)
• Signal Processing (AREA)
• Physics & Mathematics (AREA)
• General Physics & Mathematics (AREA)
• Discrete Mathematics (AREA)
• Theoretical Computer Science (AREA)
• Compression Or Coding Systems Of Tv Signals (AREA)
• Image Processing (AREA)
• Compression Of Band Width Or Redundancy In Fax (AREA)

Applications Claiming Priority (2)

Publications (1)

Family

ID=41152012

Family Applications (1)

Country Status (6)

Families Citing this family (9)

Family Cites Families (8)

• 2009
  • 2009-06-04 WO PCT/EP2009/056903 patent/WO2009147224A1/fr active Application Filing
  • 2009-06-04 CN CN200980120456.9A patent/CN102047663A/zh active Pending
  • 2009-06-04 EP EP09757605A patent/EP2281396A1/de not_active Withdrawn
  • 2009-06-04 US US12/737,034 patent/US20110081093A1/en not_active Abandoned
  • 2009-06-04 JP JP2011512136A patent/JP2011522496A/ja not_active Withdrawn
  • 2009-06-04 KR KR1020107027301A patent/KR20110020242A/ko not_active Application Discontinuation

Non-Patent Citations (1)

Also Published As

Similar Documents

Legal Events