JP5941097B2 - Method and apparatus for texture compression using patch-based sampling texture synthesis - Google Patents

Description

  The present invention relates generally to video coding and decoding, and more particularly to a texture compression method and apparatus.

  Although there are many image and video coding standards, recommendations, and extensions (hereinafter collectively referred to as video coding standards), texture compression is a technique that has not been fully developed. . First, most current video coding standards encode video pictures using a deformation-based method. Such a method requires a very large number of bits to preserve the texture details of a particular video picture. This is especially true when the image contains a large texture background.

  In general, an image area can be classified into a flat region, an edge region, or a texture region. Many current video coding standards employ transformation-based methods that are generally applied to video pictures, regardless of the type of region contained within such pictures. When such a transform-based method is applied to a texture region of a video picture, a series of high frequency components are generated that are later disposed of through operations such as filtering. Therefore, it is very difficult to preserve texture details during compression. This is because such a region is usually associated with such a high frequency component and reflects such a texture region.

  Texture synthesis is used in both computer vision and computer graphics applications, and because of its high complexity, texture synthesis is not frequently used for image and video compression operations. According to the method for texture synthesis, a texture can be defined as several visual patterns on a finite two-dimensional (2D) plane with some fixed distribution. The purpose of texture synthesis is explained below: given a particular texture sample, it is generated by the same latent probabilistic processing as that texture sample when received by a human observer. A new texture appears.

  One way to perform a texture synthesis operation is to directly determine global statistics in the feature space and sample images from the texture ensemble. This texture synthesis operation is a form of model-based coding. In order to perform such a texture synthesis operation at the encoder, the image is first analyzed with respect to the texture and the parameters of the texture are evaluated and coded. At the decoder, these parameters are extracted from the data bitstream and used to reconstruct the modeled texture. In particular, the texture is reconstructed at the decoder by determining the probability distribution from the extracted parameters. The disadvantage of this method of texture synthesis is the high complexity required to analyze and reconstruct the texture in both the encoder and decoder.

  These and other deficiencies and disadvantages of the prior art are overcome by the principles of the present invention relating to a method and apparatus for texture compression using patch-based sampled texture synthesis.

  In accordance with an aspect of the principles of the present invention, an apparatus is provided. The apparatus includes an encoder that encodes a texture for a picture by synthesizing the texture. The encoder uses information to move the decoding complexity operation to the encoder during texture synthesis using the patch-based sampling method.

  According to another aspect of the present principles, a method is provided. The method includes encoding the texture for the picture by synthesizing the texture. The encoding step uses information for moving the decoding complexity operation to the encoder during texture synthesis using the patch-based sampling method.

  In accordance with yet another aspect of the present principles, an apparatus is provided. The apparatus includes a decoder that decodes the texture for a picture by synthesizing the texture. The decoder uses information configured to move the decoding complexity operation to the encoder during texture synthesis using the patch-based sampling method.

  According to a further aspect of the present principles, a method is provided. The method includes the step of decoding the texture for the picture by synthesizing the texture. The decoding step uses information for moving the decoding complexity operation to the encoder during texture synthesis using the patch-based sampling method.

  These and other aspects, features and advantages of the principles of the present invention will become apparent from the following detailed description of exemplary embodiments, read in conjunction with the accompanying drawings.

The principles of the present invention will be better understood with reference to the following illustrative figures.
It is a figure of the general patch base sampling method. FIG. 3 is a diagram of a patch-based sampling method according to an embodiment of the present principles. 1 is a block diagram of an exemplary video encoder to which the principles of the present invention are applied, in accordance with an embodiment of the principles of the present invention. FIG. 3 is a block diagram of an exemplary video decoder to which the principles of the present invention are applied, in accordance with an embodiment of the principles of the present invention. FIG. 3 is a flow diagram of an exemplary video texture encoding method using patch-based sampling texture synthesis, in accordance with an embodiment of the present principles. FIG. 3 is a flow diagram of an exemplary video texture decoding method using patch-based sampling texture synthesis in accordance with an embodiment of the present principles.

  The principles of the present invention relate to a method and apparatus for texture compression using sampled texture synthesis.

This description illustrates the principles of the invention. Accordingly, those skilled in the art who have read the detailed description of the invention will appreciate the various configurations encompassing the principles of the invention even though not explicitly described and illustrated herein. It is understood that it can be devised.
All statements herein reciting principles, aspects and embodiments of the principles of the invention, as well as specific examples of the principles of the invention, include both their structural and functional equivalents. Is intended. Furthermore, such equivalents are intended to include presently known equivalents and future equivalents, i.e., all elements developed that perform the same function regardless of structure.

  Thus, for example, it will be appreciated by those skilled in the art that the block diagrams depicted herein illustrate a conceptual view of an exemplary circuit that includes the principles of the present invention. Similarly, flowcharts, flow diagrams, state transition diagrams, pseudocode, etc., represent various processes that may be substantially represented in a computer-readable medium and executed by a computer or processor, and are described herein by one of ordinary skill in the art. It will be appreciated that computer code can be developed from various disclosures in the document.

  The functionality of the various elements shown in the drawings may be brought about by the use of dedicated hardware and hardware capable of executing software associated with the appropriate software. If provided by a processor, the functionality may be provided by a single dedicated processor, by a single shared processor, or by multiple separate processors, some of which may be shared. Furthermore, the explicit use of “processor” or “controller” should not be understood as referring only to hardware capable of executing software, but without limitation, storing digital signal processor (DSP) hardware, software ROM (read only memory), RAM (random access memory), graphics processor (GPU), and non-volatile storage may be included implicitly.

  Other hardware, conventional and / or custom, may also be included. Similarly, the switches shown in the drawings are merely conceptual. These functions can be performed through the operation of program logic, through dedicated logic, through program controlled instructions and dedicated logic, or manually, and certain techniques are better understood from context. As such, it can be selected by the implementer.

  In the claims of this application, any element represented as a means for performing a particular function can be, for example, a) a combination of circuit elements that perform that function, or b) It is intended to include any method of performing the function, including firmware, any form of software including microcode, etc. combined with appropriate circuitry to perform the function. The principles of the invention defined by the claims lie in the fact that the functions provided by the various means disclosed can be combined and organized together in the manner recited in the claims. Exists. Accordingly, all means capable of providing these functions are equivalent to those shown herein.

  References in the specification to “one embodiment” or “an embodiment” of the principles of the invention refer to at least one embodiment of the principles of the invention in particular features, structures, characteristics, etc. described with the embodiment. Means it is included. Thus, the appearances of the phrases “in one embodiment” or “in an embodiment” appearing in various places throughout the specification are not necessarily all referring to the same embodiment.

  For example, the use of “/”, “and / or” and “at least one of” in “A / B”, “A and / or B” and “at least one of A and B” is It is understood that it is intended to encompass the selection of only the first listed option (A), the second selected option (B) only, or the selection of both options (A and B) It should be. As another example, when “A, B, and / or C” and “at least one of A, B, and C” are used, the expression is a choice of only the first listed option (A), second Selection of only the listed option (B), selection of only the third listed option (C), selection of only the first and second listed options (A and B), listed first and third The selection of only the alternatives (A and C), the selection of only the second and third choices (B and C), or the selection of all three alternatives (A, B and C) Is intended. This can be extended to a large number of items, as will be readily understood by those skilled in the art and related arts.

  Further, although one or more embodiments and / or implementations of the principles of the present invention are described herein in connection with the MPEG-4 AVC (ISO / IEC 14496-10) standard, the principles of the present invention are described. Is not limited to this standard, but other video coding standards, recommendations, and extensions thereof (including extensions of the MPEG-4 AVC standard, VC-1 (SMPTE)) while maintaining the spirit of the principles of the present invention. Can be used in connection with.

  In this specification, “high level syntax” refers to syntax that exists hierarchically above the macroblock layer in the bitstream. For example, the high level syntax herein includes, but is not limited to, slice header level, supplemental enhancement information (SEI) level, picture parameter set (PPS) level, sequence parameter set (SPS) level, and network abstraction. It may be a layer (NAL) unit header level syntax.

  The present application discloses and improves upon an alternative method of performing texture modeling called “patch-based sampling” that differs from prior art statistical modeling methods. This texture video coding method involves the use of a texture patch of the input sample texture as a component for the composition of the synthesized texture. Referring to FIG. 1, an overview of a prior art patch-based sampling method is indicated generally by the reference numeral 100.

In each of the steps involved in the patch-based sampling method of FIG. 1, a patch of the input sample texture I _in is pasted into the composite texture I _out . In order to avoid mismatching characteristics across the patch boundary, the new patch B _k is applied to the already applied patch {B ₀ ,. . . B _k−1 } is selected. For simplicity, only square patches of a predetermined size ω _B × ω _B are used. The texture patch boundary zone is defined as a band of width ω _E along the patch boundary. The new patch overlaps the old patch in the border zone. A new patch is selected by searching the input sample texture I _in for all patches Ψ _B whose boundary zone matches the old patch using a predetermined condition. The synthesizer randomly selects one patch from Ψ _B and places the selected patch in the synthesized texture I _out . The order of patch generation for images is from bottom to top and left to right. Note that due to the random nature of the texture, the patch-based sampling method 100 does not care which order is used as long as it is the normal scan order. However, the encoder and decoder should use the same order.

  The patch-based sampling method of FIG. 1 can also be described as follows.

(A) A texture patch B ₀ is randomly selected from the input sample texture I _in . Paste B ₀ in the lower left corner of I _out . Set k = 1.

(B) As the boundary zone with respect to each of the texture patch [psi _B is matched to the boundary zone of the old adjacent patches, it generates a [psi _B that is determined from all of the texture patches from I _in.

(C) An element is randomly selected from Ψ _B as the k-th texture patch B _k . Pasting _{B k} on I _out. Set k = k + 1.

(D) Repeat steps (b) and (c) until I _out is completely covered.

  (E) Perform blending in the boundary zone.

(E) blending step in, after I _out has been completely covered with the texture patch, resulting in a smooth transition between adjacent texture patches.

Note that the above algorithm is for forced texture synthesis. In order to synthesize a texture with a boundary, the boundary area adjacent to the texture can be considered as a boundary zone when searching for Ψ _B.

If a patch-based sampling method is used for texture compression, only input texture samples are coded at the encoder. At the decoder, the texture is reconstructed according to the algorithm described above associated with the patch-based sampling method of FIG. Patch-based sampling methods are more efficient than statistical model-based methods in speed and quality. Nevertheless, patch-based sampling methods are not limited: (1) the decoder is more complex than the encoder; (2) the speed of the decoder is by searching for Ψ _B , a time-consuming scan. And (3) codec performance has several drawbacks, including being dependent on a search algorithm that is dependent on the decoder configuration for the encoder.

  The principles of the present invention relate to a method and apparatus for texture compression using patch-based sampled texture synthesis that is better than the patch-based method understood in conjunction with FIG. In contrast to many image and video coding standards, the present invention can code textures using texture synthesis with patch-based sampling. Advantageously and in contrast to the prior art, the principles of the present invention use information to implement real-time texture synthesis at the decoder.

  According to the present invention, complexity is moved from the decoder to the encoder. Such a policy eliminates the need for a search in the decoder, and the encoder substantially determines the performance (eg, compression efficiency).

  Turning to FIG. 2, a schematic diagram of an exemplary patch-based sampling method is indicated generally by the reference numeral 200. Note that the method of FIG. 2 uses information regarding the patch-based sampling method of the method.

In one embodiment, the information used is a “displacement vector” that describes the placement of the new patch _in the input sample texture I _in . This displacement vector is defined as dv as shown in FIG. In the embodiment of FIG. 2, dv is the distance from the upper left point of the _{B k} to the upper left point of _{I in.} In accordance with the principles of the present invention, this “displacement vector” is different from the conventional “motion vector” _in that the reference picture is the input texture I _in and generally refers to temporal displacement. Is different from the motion vectors used in motion interpolation in current video compression standards and recommendations. Accordingly, spatial displacement vectors described herein in accordance with the principles of the present invention are used interchangeably herein with “input texture spatial displacement vectors”. In the encoder, both the input sample texture and information indicative of the spatial displacement vector dv are coded. At the decoder, the “displacement vector” is used directly to generate a new patch.

Input sample texture and information indicating a spatial displacement vector may be coded in any way or loss is a loss is not Na. In one embodiment, the input sample texture is coded in certain embodiments of the loss by using a vector quantization and entropy coding, the displacement vector has name aspect of differential pulse code modulation (DPCM) and loss using entropy coding Coding. Advantageously, embodiments of the principles of the present invention apply to any method based on patch-based sampling, including but not limited to multi-resolution patch-based sampling texture synthesis where displacement vectors can be generated at each of the resolutions. Can be done.

  Referring to FIG. 3, a video encoder capable of performing video encoding in accordance with the MPEG-4 AVC standard is indicated generally by the reference numeral 300.

  Video encoder 300 includes a frame ordering buffer 310 having an output connected to the non-inverting input of combiner 385. The output of the combiner 385 is connected to the first input of the converter and quantizer 325. The output of the transformer and quantizer 325 is connected to the first input of the entropy coder 345 and the first input of the inverse transformer and inverse quantizer 350. The output of entropy coder 345 is connected to the first non-inverting input of combiner 390. The output of the combiner 390 is connected to the first input of the output buffer 335.

  The first output of the encoder controller 305 is the second input of the frame ordering buffer 310, the second input of the inverse transformer and inverse quantizer 350, the input of the picture type determination module 315, and the macroblock type (MB type). Input of decision module 320, second input of intra prediction module 360, second input of deblocking filter 365, first input of motion compensator 370, first input of motion evaluator 375, reference picture buffer 380 , A first input of the texture synthesizer 333, and a first input of the displacement vector extractor 334.

  The second output of the encoder controller 305 is the first input of the supplemental enhancement information (SEI) inserter 330, the second input of the transformer and quantizer 325, the second input of the entropy coder 345, and the output buffer 335. , And a sequence parameter set (SPS) and picture parameter set (PPS) inserter 340.

  A first output of the picture type determination module 315 is connected to a third input of the frame ordering buffer 310. A second output of the picture type determination module 315 is connected to a second input of the macroblock type determination module 320.

  The output of the sequence parameter set (SPS) and picture parameter set (PPS) inserter 340 is connected to the third non-inverting input of combiner 390.

  The output of the inverse quantizer and inverse transformer 350 is connected to the first non-inverting input of the combiner 319. The output of the combiner 319 is connected to the first input of the intra prediction module 360 and the first input of the deblocking filter 365. The output of the deblocking filter 365 is connected to the first input of the reference picture buffer 380. A first output of the reference picture buffer 380 is connected to a second input of the motion evaluator 375 and a third input of the motion compensator 370. The second output of the reference picture buffer 380 is connected to the second input of the texture synthesizer 333. The first output of the texture synthesizer 333 is connected to the third input of the deblocking filter 365. The second output of the texture synthesizer 333 is connected to the second input of the displacement vector extractor 334. The output of the displacement vector extractor 334 is connected to the third input of the entropy coder 345 to provide a motion vector. The first output of the motion evaluator 375 is connected to the second input of the motion compensator 370. A second output of motion evaluator 375 is connected to a fourth input of entropy coder 345 to provide a displacement vector. Note that in one embodiment, the motion and displacement vectors can be similarly coded to include both magnitude and direction.

  The output of the motion compensator 370 is connected to the first input of the switch 397. The output of the intra prediction module 360 is connected to the second input of the switch 397. The output of the macroblock type determination module 320 is connected to the third input of the switch 397. The third input of switch 397 determines whether the “data” input of the switch should be provided by motion compensator 370 or intra-prediction module 360 (as compared to the control or third input). The output of switch 397 is connected to the second non-inverting input of combiner 319 and the inverting input of combiner 385.

  The input of the frame ordering buffer 310 and the encoder controller 305 can be used as the input of the encoder 300 for receiving the input picture 301. Furthermore, the input of the supplemental extension information (SEI) inserter 330 can be used as an input of the encoder 300 for receiving metadata. The output of the output buffer 335 can be used as an output of the encoder 300 for outputting a bit stream.

  Referring to FIG. 4, a video decoder capable of performing video decoding according to the MPEG-4 AVC standard is indicated generally by the reference numeral 400.

  Video decoder 400 includes an input buffer 410 having an output connected to a first input of entropy decoder 445. A first output of entropy decoder 445 is connected to a first input of inverse transformer and inverse quantizer 450. The output of the inverse transformer and inverse quantizer 450 is connected to the second non-inverting input of the combiner 425. The output of the combiner 425 is connected to the second input of the deblocking filter 465 and the first input of the intra prediction module 460. The second output of the deblocking filter 465 is connected to the first input of the reference picture buffer 480. The first output of the reference picture buffer 480 is connected to the second input of the motion compensator 470. The second output of the reference picture buffer 480 is connected to the first input of the texture synthesizer 433. The output of the displacement vector decoder 434 is connected to the second input of the texture synthesizer 433.

  A second output of entropy decoder 445 is connected to a third input of motion compensator 470, a first input of deblocking filter 465, and a second input of intra prediction module 460. The third output of the entropy decoder 445 is connected to the input of the decoder controller 405. The fourth output of the entropy decoder 445 is connected to the second input of the displacement vector decoder 434. The first output of the decoder controller 405 is also connected to the second input of the entropy decoder 445. The second output of the decoder controller 405 is connected to the second input of the inverse transformer and inverse quantizer 450. The third output of the decoder controller 405 is the third input of the intra prediction module 460, the first input of the motion compensator 470, the second input of the reference picture buffer 480, the third input of the texture synthesizer 433, And a first input of a displacement vector decoder 434. The fourth output of the decoder controller 405 is connected to the third input of the deblocking filter 465.

  The output of the motion compensator 470 is connected to the first input of the switch 497. The output of the intra prediction module 460 is connected to the second input of the switch 497. The output of switch 497 is connected to the first non-inverting input of combiner 425.

  The input of the input buffer 410 can be used as an input of the decoder 400 for receiving the input bit stream. The first output of the deblocking filter 465 can be used as the output of the decoder 400 for outputting the output picture.

  Referring to FIG. 5, an exemplary video texture encoding method using patch-based sampling texture synthesis is indicated generally by the reference numeral 500.

  Method 500 includes a start block 505 that passes control to function 510. The function block 510 performs image segmentation on the input image and passes control to a decision block 515. Decision block 515 determines whether the current image segmentation unit (eg, image block or image region) is a texture region. If so, control is passed to function block 520. Otherwise, control is passed to function block 545.

The function block 520 signals the texture region (eg, using one or more high-level syntax elements), selects an input sample texture I _in from the texture region, and codes and decodes the input sample texture. Then, control is passed to the function block 525. The function block 525 performs texture synthesis by reconstructing the texture region B _k using patch-based sampling from the decoded sample texture and passes control to the function block 530 and the function block 550. It is understood that patch-based sampling can be performed using any patch-based sampling method including, but not limited to, the prior art patch-based sampling described above with respect to steps (a) through (d). Should be.

  The function block 530 calculates a corresponding information space displacement vector dv for each of the patches in the output texture and passes control to the function block 535. The function block 535 codes all the spatial displacement vectors dv and passes control to the function block 540.

  The function block 540 outputs the corresponding data (eg, coded displacement vector) and passes control to the end block 599.

  The function block 545 uses other coding methods (not the method specified in connection with the blocks 520 to 535) to code the non-texture region and passes control to the function block 540.

  A function block 550 that performs a decoding operation during encoding performs blending between boundary zones and passes control to a function block 545.

  With reference to FIG. 6, an exemplary video texture decoding method using patch-based sampling texture synthesis is indicated generally by the reference numeral 600.

  Method 600 includes a start block 605 that passes control to function block 610. The function block 610 determines the header of the current bitstream and / or the header (s) of the packet (s) associated with the bitstream (to determine whether the texture region is signaled in association with the bitstream). ) Extract and pass control to decision block 615. Decision block 615 determines whether the current image segmentation unit (eg, image block or image region) is a texture region. If so, then control is passed to function block 620. Otherwise, control is passed to function block 645.

The function block 620 decodes all of the input sample texture I _in and the spatial displacement vector dv and passes control to the function block 625. The function block 625 reconstructs the output texture region by pasting the patch B _k obtained from the input sample texture using information corresponding to the spatial displacement vector dv, and passes control to the function block 630. The function block 630 performs blending within the boundary zone up to all patches and passes control to the function block 635.

  The function block 635 combines the texture regions and passes control to the function block 640. The function block 640 outputs the corresponding data (decoded texture region) and passes control to the end block 699.

  The function block 645 uses other decoding methods (not the method specified with respect to the blocks 620 to 630) to decode the non-texture regions and passes control to the function block 635.

  It should be understood that texture mode signaling according to the principles of the present invention may be performed using any signaling technique. For example, as described above with respect to functional block 520 of FIG. 5, such signaling can be performed using one or more high-level syntax elements. Such syntax elements or other techniques may simply indicate and / or otherwise identify related modes, such as texture modes. Of course, the principles of the present invention are not limited to the above-described method for signaling texture, and other methods may be used in accordance with the principles of the present invention while maintaining the spirit of the principles of the present invention.

  It should further be appreciated that the blending described above with respect to function block 630 can be performed using any blending technique. For example, in one embodiment, several blending methods can be applied to smooth transitions between different patches in the boundary zone. As a further example, in one embodiment, the deblocking filter concept can be used in conjunction with the MPEG-4 AVC standard and / or blurring method. In one embodiment, the blurring method includes the use of bilinear interpolation (weighted bilinear interpolation) from the two closest boundaries or current boundaries and / or processed boundaries. The same blending method is preferably applied to both the decoding part and the decoder of the encoder to avoid mismatching. Of course, the principles of the present invention are not limited to the blending method described above, and other methods may be used in accordance with the principles of the present invention while maintaining the spirit of the principles of the present invention.

  Some description of the many attendant advantages / features of the present invention, some of which are described above, will now be made. For example, one advantage / feature is an apparatus that includes an encoder that encodes the texture of a picture by texture synthesis. When the encoder synthesizes a texture using a patch-based sampling method, it uses information that moves the normal decoding complexity to the encoder.

  Another advantage / feature is an apparatus having the above-described encoder, wherein the information includes at least one input texture space displacement vector that eliminates a corresponding texture region search in a corresponding decoder that later decodes the texture of a picture. It is a device including.

  Yet another advantage / feature is an apparatus having the above-described encoder, wherein the picture includes a plurality of regions, and the encoder signals which of the plurality of regions is coded as a texture.

  Yet another advantage / feature is an apparatus having the above-described encoder, wherein the texture is an input sample texture obtained from the picture, and the encoder encodes corresponding information and the input sample texture. is there.

  Yet another advantage / feature is an apparatus having the above-described encoder, wherein the information is configured to support pasting of texture patches directly to a texture region of a picture at a corresponding decoder. Device.

  Yet another advantage / feature is an apparatus having the encoder described above, wherein the texture is an input sample texture obtained from the picture, and the encoder selects a plurality of texture patches from the input sample texture; The encoder is an apparatus for smoothing transitions in overlapping boundary zones of individual texture patches among a plurality of texture patches using at least one of deblocking filtering and blurring.

  These and other features and advantages of the principles of the present invention will be readily ascertainable by one of ordinary skill in the pertinent art based on the teachings herein. It should be understood that the teachings of the present principles may be implemented in various forms of hardware, software, firmware, application specific processors, or combinations thereof.

  Most preferably, the teachings of the present principles are implemented as a combination of hardware and software. Furthermore, the software may be implemented as an application program that is explicitly included in the program storage unit. This application program can be uploaded to and executed by a machine containing any suitable architecture. In the presently preferred embodiment, the machine may be implemented on a computer platform having hardware such as one or more central processing units (CPU), random access memory (RAM), input / output (I / O) interfaces, and the like. The computer platform may also include an operating system and microinstruction code. The various processes and functions described herein can be part of microinstruction code or part of an application program, or any combination thereof, which can be performed by a CPU. Can be a thing. In addition, various other peripheral device units may be connected to the computer platform such as an additional data storage unit and a printing unit.

  Since some of the constituent system elements and methods shown in the accompanying drawings are preferably implemented in software, the actual connections between multiple system components or multiple processing functional blocks are the principles of the present invention. It should be further understood that can vary depending on how it is programmed. In view of the teachings herein, one of ordinary skill in the related art will be able to contemplate these and similar implementations or configurations of the principles of the present invention.

Although illustrative embodiments are described herein with reference to the accompanying drawings, the principles of the invention are not limited to these embodiments themselves, and various modifications and changes may be made to the principles of the invention. These embodiments can be brought to these embodiments by one skilled in the relevant art without departing from the scope or spirit. All of these variations and modifications are intended to be included within the scope of the present principles as set forth in the appended claims.
The present invention includes the following aspects.
(Appendix 1)
An apparatus comprising an encoder (300) for encoding a texture of a picture by synthesizing a texture, wherein the encoder performs a decoding operation when synthesizing the texture using a patch-based sampling method The device using information to be moved to.
(Appendix 2)
The apparatus of claim 1, wherein the information includes at least one input texture space displacement vector that eliminates a decoding operation that performs a corresponding texture region search at a decoder capable of decoding the texture of the picture.
(Appendix 3)
The picture includes a plurality of regions;
The apparatus of claim 1, wherein the encoder signals which of the plurality of regions is coded as a texture region.
(Appendix 4)
The texture is an input sample texture obtained from the picture;
The apparatus of claim 1, wherein the encoder (300) encodes the information and the input sample texture.
(Appendix 5)
The apparatus of claim 1, wherein the information is configured to support direct pasting of a texture patch into a texture region of the picture in a corresponding decoder.
(Appendix 6)
The texture is an input sample texture obtained from the picture;
The encoder (300) selects a plurality of texture patches from the input sample texture and uses at least one of deblocking filtering and blurring to overlap boundary zones of individual texture patches of the plurality of texture patches The apparatus of claim 1, wherein the apparatus smooths the transitions.
(Appendix 7)
Encoding a texture of a picture by synthesizing a texture, wherein the encoding step moves a decoding operation to an encoder when the texture is synthesized using a patch-based sampling method The method (500), wherein information to be used is used.
(Appendix 8)
The method of claim 7, wherein the information includes at least one input texture space displacement vector (530) that causes the encoder to perform a texture region search.
(Appendix 9)
The picture includes a plurality of regions;
The method of claim 7, wherein the method further comprises signaling (520) which of the plurality of regions is coded as a texture.
(Appendix 10)
The texture is an input sample texture obtained from the picture;
The method of claim 7, wherein the encoding step includes encoding (520, 535) the information and the input sample texture.
(Appendix 11)
The method of claim 7, wherein the information is configured to support direct pasting (530) of a texture patch into a texture region of the picture at a corresponding decoder.
(Appendix 12)
The texture is an input sample texture obtained from the picture;
The encoding step further comprises:
Selecting (525) a plurality of texture patches from the input sample texture;
Smoothing transitions within overlapping boundary zones of individual texture patches of the plurality of texture patches using at least one of deblocking filtering and blurring (550);
The method according to appendix 7, comprising:
(Appendix 13)
An apparatus comprising a decoder (400) for decoding a texture of a picture by synthesizing a texture,
The apparatus, wherein the decoder uses information to move a decoding operation from the decoder to the encoder when synthesizing the texture using a patch-based sampling method.
(Appendix 14)
The apparatus of claim 13, wherein the information includes at least one input texture space displacement vector that eliminates a corresponding texture region search in the decoder.
(Appendix 15)
The picture includes a plurality of regions;
The apparatus of claim 13, wherein the decoder (400) determines which of the plurality of regions is coded as the texture based on a received signal.
(Appendix 16)
The texture is an input sample texture obtained from the picture;
The apparatus of claim 13, wherein the decoder (400) decodes the information and the input sample texture.
(Appendix 17)
The apparatus of claim 13, wherein the information is configured to support direct pasting of a texture patch into a texture region of the picture at the decoder.
(Appendix 18)
The texture is an input sample texture obtained from the picture;
The decoder (400) selects a plurality of texture patches from the input sample texture and uses at least one of deblocking filtering and blurring to overlap boundary zones of individual texture patches of the plurality of texture patches The apparatus of claim 13 wherein the transition in is smoothed.
(Appendix 19)
A method comprising the step of decoding the texture of a picture by synthesizing the texture, comprising:
The method (600) wherein the decoding step uses information generated from an operation in which a decoding operation is moved to an encoder when the texture is synthesized using a patch-based sampling method.
(Appendix 20)
The method of claim 19, wherein the information includes at least one input texture space displacement vector (620) that removes a corresponding texture region search.
(Appendix 21)
The picture includes a plurality of regions;
The method of claim 19, wherein the method further comprises the step of determining (610) which of the plurality of regions is coded as a texture region based on the information.
(Appendix 22)
20. The method of clause 19, wherein the information is configured to support pasting (625) a texture patch directly into a texture region of the picture during the decoding step.
(Appendix 23)
The texture is an input sample texture obtained from the picture;
The decoding method further comprises:
Selecting a plurality of texture patches from the input sample texture (625);
Smoothing (630) transitions within overlapping boundary zones of individual texture patches of the plurality of texture patches using at least one of deblocking filtering and blurring;
The method according to appendix 19, comprising:

Claims (10)

An apparatus comprising encoder for encoding texture for a picture by synthesizing the texture,
The texture is an input sample texture obtained from the picture;
The encoder synthesizes the texture using a patch-based sampling method ;
Calculating an input texture space displacement vector for each texture patch in the synthesized texture;
The input texture space displacement vector represents a position of a texture patch in the input sample texture;
The encoder encodes the input texture space displacement vector and the input sample texture;
The apparatus, wherein the input sample texture is coded in a lossy manner and the input texture spatial displacement vector is coded in a lossless manner . The input texture spatial displacement vector, in the decodable decoder texture of the picture, to remove the operations that perform corresponding texture region search for, according to claim 1. The picture includes a plurality of regions;
The apparatus of claim 1, wherein the encoder signals which of the plurality of regions are coded as texture regions. The apparatus of claim 1, wherein the input texture space displacement vector is configured to support direct pasting of a texture patch into a texture region of the picture at a corresponding decoder. The encoder selects a plurality of texture patches from the input sample texture, using at least one of deblocking filtering and blurring in the boundary zone of overlap individual texture patches of the plurality of texture patches The apparatus of claim 1, wherein the transition is smoothed. A method performed by an encoder,
Look including the step of encoding the texture of a picture by synthesizing the texture,
The texture is an input sample texture obtained from the picture, the step of encoding, using patch-based sampling method combining the texture, and the input for each texture patch of the synthesized in the texture Calculating a texture space displacement vector, wherein the input texture space displacement vector represents a position of a texture patch in the input sample texture;
The encoding step includes encoding the input texture space displacement vector and the input sample texture;
The method, wherein the input sample texture is coded in a lossy manner and the input texture spatial displacement vector is coded in a lossless manner. The method of claim 6, wherein the input texture space displacement vector removes an operation to perform a corresponding texture region search at a decoder capable of decoding the texture of the picture. The picture includes a plurality of regions;
Said method which of said plurality of regions further comprises automatic answering step to either the signaling is coded as texture A method according to claim 6. The input texture spatial displacement vector is configured to support only with direct bonding of the texture patch into the texture region of the picture at a corresponding decoder method of claim 6. Said encoding step comprises :
And the steps for selecting a plurality of texture patches from the input sample texture,
And steps of deblocking filtering and using at least one of blurring to smooth transitions in the boundary zone of overlap individual texture patches of the plurality of texture patches,
The method of claim 6 , further comprising:

Images