KR101467601B1 - Tiling in video encoding and decoding - Google Patents

Tiling in video encoding and decoding Download PDF

Info

Publication number
KR101467601B1
KR101467601B1 KR1020097023043A KR20097023043A KR101467601B1 KR 101467601 B1 KR101467601 B1 KR 101467601B1 KR 1020097023043 A KR1020097023043 A KR 1020097023043A KR 20097023043 A KR20097023043 A KR 20097023043A KR 101467601 B1 KR101467601 B1 KR 101467601B1
Authority
KR
South Korea
Prior art keywords
view
function block
delete delete
id
control
Prior art date
Application number
KR1020097023043A
Other languages
Korean (ko)
Other versions
KR20100016212A (en
Inventor
푸어빈 비브하스 팬디트
펭 인
동 티안
Original Assignee
톰슨 라이센싱
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority to US92301407P priority Critical
Priority to US60/923,014 priority
Priority to US92540007P priority
Priority to US60/925,400 priority
Application filed by 톰슨 라이센싱 filed Critical 톰슨 라이센싱
Publication of KR20100016212A publication Critical patent/KR20100016212A/en
Application granted granted Critical
Publication of KR101467601B1 publication Critical patent/KR101467601B1/en

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/20Image signal generators
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/46Embedding additional information in the video signal during the compression process
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00Stereoscopic video systems; Multi-view video systems; Details thereof
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00Geometric image transformation in the plane of the image
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T9/00Image coding
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/10Processing, recording or transmission of stereoscopic or multi-view image signals
    • H04N13/106Processing image signals
    • H04N13/111Transformation of image signals corresponding to virtual viewpoints, e.g. spatial image interpolation
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/10Processing, recording or transmission of stereoscopic or multi-view image signals
    • H04N13/106Processing image signals
    • H04N13/161Encoding, multiplexing or demultiplexing different image signal components
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/10Processing, recording or transmission of stereoscopic or multi-view image signals
    • H04N13/194Transmission of image signals
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/172Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a picture, frame or field
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/174Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a slice, e.g. a line of blocks or a group of blocks
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/177Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a group of pictures [GOP]
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/182Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a pixel
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/20Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using video object coding
    • H04N19/23Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using video object coding with coding of regions that are present throughout a whole video segment, e.g. sprites, background or mosaic
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/44Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/597Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding specially adapted for multi-view video sequence encoding
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/60Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
    • H04N19/61Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding in combination with predictive coding
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/70Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by syntax aspects related to video coding, e.g. related to compression standards
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/85Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N2213/00Details of stereoscopic systems
    • H04N2213/003Aspects relating to the "2D+depth" image format

Abstract

Implementations are provided for, for example, view tiling of video encoding and decoding. The method may include accessing (826) a video picture comprising multiple pictures combined into a single picture, accessing (806, 808, 822) information indicating how multiple pictures of the accessed video picture have been combined, Decode (824, 826) the video picture to provide a decoded representation, and provide (824, 826) the accessed information and decoded video picture as an output. Some other implementations format or process information indicating how multiple pictures contained in a single video picture are combined into a single video picture and format or process the encoded representation of the combined multiple pictures.
Figure R1020097023043
View tiling, video encoding, video decoding MVC, pseudo-view

Description

[0001] TILING IN VIDEO ENCODING AND DECODING [0002]

Cross reference of related application

This application is related to (1) United States Provisional Application Serial No. 60 / 923,014 (filed on April 12, 2007) and "Multiview information" (Attorney Docket No. PU070078) and (2) US Provisional Application No. 60 / 925,400 (Filed April 20, 2007) and entitled " View Tiling in MVC Coding "(Attorney Docket No. PU070103). Each of these two applications is incorporated herein by reference in its entirety.

These principles generally relate to video encoding and / or decoding.

Video display manufacturers can use the framework to arrange or tile different views on a single frame. These views can then be extracted and rendered from their respective locations.

According to a general aspect, a video picture comprising multiple pictures combined into a single picture is accessed. Information indicating how multiple images of the accessed video image are combined is accessed. The video picture is decoded to provide a decoded representation of the combined multiple pictures. The accessed information and the decoded video picture are provided as an output.

According to another general aspect, information is generated that indicates how multiple images included in a video image are combined into a single image. The video picture is encoded to provide an encoded representation of the combined multiple pictures. The generated information and the encoded video picture are provided as an output.

According to another general aspect, the signal or signal structure includes information indicating how multiple pictures included in a single video picture are combined into a single video picture. The signal or signal structure also includes an encoded representation of the combined multiple images.

According to another general aspect, a video picture comprising multiple pictures combined into a single picture is accessed. Information indicating how multiple images of the accessed video image are combined is accessed. The video picture is decoded to provide at least one decoded representation of the multiple pictures. The accessed information and the decoded representation are provided as output.

According to another general aspect, a video picture comprising multiple pictures combined into a single picture is accessed. Information indicating how multiple images of the accessed video image are combined is accessed. The video picture is decoded to provide a decoded representation of the combined multiple pictures. A user input for selecting at least one of multiple pictures for display is received. A decoded output of at least one selected picture, a decoded output provided based on the accessed information, a decoded representation and a user input are provided.

The details of one or more implementations are set forth in the accompanying drawings and the description below. It should be understood that implementations may be constructed or implemented in various ways, even when described in one particular manner. For example, an implementation may be implemented as a method, or may be implemented as an apparatus configured to perform a set of operations or as a signal. Other aspects and features will become apparent from the following detailed description considered in conjunction with the accompanying drawings and the claims.

Figure 1 shows an example of four views tiled on a single frame.

Figure 2 shows an example of four views flipped and tiled on a single frame.

Figure 3 is a block diagram of a video encoder to which these principles may be applied, in accordance with an embodiment of the present principles;

4 is a block diagram of a video decoder to which these principles may be applied, in accordance with an embodiment of the present principles;

Figures 5A and 5B are flow diagrams of a method of encoding images for a plurality of views using the MPEG-4 AVC standard, in accordance with an embodiment of the present principles.

6A and 6B are flow diagrams of a method of decoding pictures for a plurality of views using the MPEG-4 AVC standard, in accordance with an embodiment of the present principles.

Figures 7A and 7B are flow diagrams of a method of encoding images for multiple views and depths using the MPEC-4 AVC standard, in accordance with embodiments of the present principles.

8A and 8B are flow diagrams of a method for decoding pictures for multiple views and depths using the MPEC-4 AVC standard, in accordance with embodiments of the present principles.

9 is a diagram illustrating an example of a depth signal, in accordance with an embodiment of the present principles;

10 illustrates an example of a depth signal added as a tile, in accordance with an embodiment of the present principles;

11 is a YELLOW illustration of five views tiled on a single frame, in accordance with an embodiment of the present principles.

Figure 12 is a block diagram of an exemplary multi-view video coding (MVC) encoder to which these principles may be applied, in accordance with an embodiment of the present principles.

Figure 13 is a block diagram of an exemplary multi-view video coding (MVC) decoder to which the present principles may be applied, in accordance with an embodiment of the present principles.

14 is a flow diagram of a method for processing images for a plurality of views against encoding of images using a multi-view video coding (MVC) extension of the MPEG-4 AVC standard, in accordance with an embodiment of the present principles.

15A and 15B are flow diagrams of a method of encoding images for multiple views using a multi-view video coding (MVC) extension of the MPEG-4 AVC standard, in accordance with an embodiment of the present principles.

Figure 16 is a flow diagram of a method for processing images for a plurality of views against decoding of images using a multi-view video coding (MVC) extension of the MPEG-4 AVC standard, in accordance with an embodiment of the present principles.

17A and 17B are flow diagrams of a method for decoding pictures for multiple views using a multi-view video coding (MVC) extension of the MPEG-4 AVC standard, in accordance with an embodiment of the present principles.

Figure 18 is a flow diagram of a method for processing images for multiple views and depths in preparation for encoding images using multiple view video coding (MVC) extensions of the MPEG-4 AVC standard, in accordance with embodiments of the present principles.

Figures 19A and 19B are flow diagrams of a method of encoding images for multiple views and depths using a multi-view video coding (MVC) extension of the MPEG-4 AVC standard, in accordance with embodiments of the present principles.

Figure 20 is a flow diagram of a method for processing images for multiple views and depths in preparation for decoding images using a multi-view video coding (MVC) extension of the MPEG-4 AVC standard, in accordance with embodiments of the present principles.

Figures 21A and 21B are flow diagrams of a method for decoding pictures for multiple views and depths using a multi-view video coding (MVC) extension of the MPEG-4 AVC standard, in accordance with embodiments of the present principles.

22 illustrates tiling examples at a pixel level, in accordance with an embodiment of the present principles.

Figure 23 is a block diagram of a video processing device to which these principles may be applied, in accordance with an embodiment of the present principles.

Various implementations are directed to a method and apparatus for view tiling in video encoding and decoding. Accordingly, those skilled in the art will understand that, although not explicitly described or shown herein, it is contemplated that various devices may be devised that implement these principles and are within the spirit and scope of the principles.

All examples and conditional language cited herein are intended for educational purposes to assist the reader in understanding the principles and concepts of the present invention provided by the inventor (s) above the art, Are to be construed as being without limitation to the examples and conditions.

Moreover, all statements herein reciting principles, features, embodiments of the principles, and specific embodiments thereof are intended to include structural and functional equivalents. Additionally, such equivalents are intended to encompass presently known equivalents, as well as any equivalents developed in the art, that is, any elements developed that perform the same function, regardless of structure.

Thus, for example, those skilled in the art will understand that the block diagrams presented herein represent conceptual views of exemplary circuitry embodying the principles. Similarly, any of the flowcharts, flowcharts, state transition diagrams, pseudocode, etc. may be actually represented in computer readable media and may be stored on a computer or processor, whether or not such a computer is explicitly shown Lt; RTI ID = 0.0 > and / or < / RTI >

The functions of the various elements shown in the figures may be provided through use of software executable hardware and dedicated hardware associated with appropriate software. The functions may be provided by a single dedicated processor, a single shared processor, or a plurality of separate processors, some of which may be shared, when provided by the processor. The explicit use of the term " processor "or" controller "should not be construed to refer exclusively to software executable hardware, "), Random access memory (" RAM "), and non-volatile storage media.

Other conventional and / or customary hardware may also be included. Similarly, any of the switches shown in the figures are merely conceptual. Their functions may be performed through the operation of the program logic, through dedicated logic, through the interaction of program control and dedicated logic, or even manually, and the specific techniques may be performed by the implementer as is more specifically understood from the context Selectable.

In the claims, any element expressed as a means for performing a particular function may be, for example, a) a combination of circuit components performing the function or b) a firmware, microcode, etc., Including any type of software in combination with the appropriate circuitry for executing the software. The principles set forth in these claims exist to the extent that the functions provided by the various recited means are combined and grafted in the manner required by the claims. Thus, any means capable of providing these functions is considered equivalent to that shown herein.

Reference in the specification to "one embodiment" (or "one embodiment") or "embodiment" (or "implementation") of the present principles means that a particular feature, structure, Quot; is included in at least one embodiment. Accordingly, the appearances of the phrase " in one embodiment "or" in an embodiment "in various places throughout the specification are not necessarily all referring to the same embodiment.

For example, in the case of "A and / or B" and "at least one of A and B", the use of the terms "at least one" and / or " (B), or both of the options (A and B). As a further example, in the case of "A, B and / or C" and "At least one of A, B and C", these phrases may be selected only in the first listed option (A) , Only the third listed option (C), or only the first and second listed options (A and B), only the first and third listed options (A and C), the second and third listed options (B and C), and the selection of all three options (A and B and C). This can be extended because many listed items can also be easily understood by those skilled in the art.

In addition, while one or more embodiments of the present principles are described herein with reference to the MPEG-4 AVC standard, the present principles are not limited to this standard, and thus, while retaining the spirit of the present principles, -4 AVC standard extensions, in particular video coding standards, recommendations, and extensions thereof.

Further, although one or more embodiments of the present principles are described herein for a multi-view video coding extension of the MPEG-4 AVC standard, it is to be appreciated that the principles are not limited to this standard, Coding standards, recommendations, and extensions related to multi-view coding. Multi-view video coding (MVC) is a compression framework for the encoding of multi-view sequences. A multi-view video coding (MVC) sequence is a set of two or more video sequences that capture the same scene from different viewpoints.

Further, although one or more embodiments of these principles using depth information for video content are described herein, the principles are not limited to these embodiments, and thus, while maintaining the spirit of these principles, It is to be understood that other embodiments that do not utilize the information may be implemented.

Also, as used herein, a "high level syntax" refers to a syntax that exists in a bitstream that is hierarchically above a microblock layer. For example, the high-level syntax used herein includes a slice header level, a SEI (Supplemental Enhancement Information) level, a PPS (Picture Parameter Set) level, a SPS (Sequence Parameter Set) level, a VPS (But not limited to) the syntax of a Layer Header Level.

Part 10 AVC (Advanced Video Coding) standard / ITU-T (International Telecommunication Union, Telecommunication Sector) H.264 Recommendation (ISO / IEC) MPEG-4 Moving Picture Experts Group- In an implementation of multi-video coding (MVC) based on the MPEG-4 AVC standard, the reference software encodes each view into one encoder and considers cross-view references Thereby achieving multi-view prediction. Each view is coded as a separate bitstream by the encoder at its original resolution, and all bitstreams are combined to form a single bitstream that is later decoded. Each view produces a separate YUV decoded output.

Another method for multi-view prediction involves grouping a set of views into pseudo-views. In one example of this method, it is possible to tile pictures from all N views of a total of M views (sampled concurrently) on a large frame or super frame with possible downsampling or other operations. Referring to FIG. 1, an example of four views tiled on a single frame is generally indicated by reference numeral 100. All four views are in their normal orientation.

Referring to FIG. 2, an example of four views flipped and tiled on a single frame is indicated generally by the reference numeral 200. The top-left view is in its normal direction. The top-right view is flipped horizontally. The bottom-left view is vertically flipped. The bottom-right view is flipped horizontally and vertically. Thus, if there are four views, the image from each view is arranged in a super-frame like a tile. This results in a single non-coded input sequence with large resolution.

Alternatively, we can downsample the image to produce a smaller resolution. Thus, we can generate multiple sequences each containing different views that are tiled together. Each such sequence then forms a pseudo view, and each pseudo view includes N different tiled views. Figure 1 shows one pseudo-view, and Figure 2 shows another pseudo-view. These pseudo-views can then be encoded using existing video coding standards such as the ISO / IEC MPEG-2 standard and the MPEG-4 AVC standard.

Another method for multi-view prediction involves simply encoding the different views independently using the new standard and tiling the views needed by the player after decoding.

Also, in other methods, views may also be tiled in pixel-wise fashion. For example, in a superview consisting of four views, a pixel (x, y) can come from view 0, a pixel (x + 1, y) ) Can come from view 2, and pixel (x + 1, y + 1) can come from view 3.

Many display manufacturers use this framework to arrange or tile different views on a single frame and then extract and render views from their respective locations. In these cases, there is no standard way of determining whether a bitstream has this property. Thus, when the system uses a method of tiling images of different views in a large frame, the method of extracting the different views is proprietary.

However, there is no standard way of determining whether a bitstream has this property. We propose a high-level syntax to make it easier for the renderer or player to extract this information to aid display or other post-processing. It is also possible that the sub-pictures have different resolutions and some upsampling may ultimately be needed to render the view. The user may want the upsampling method to be also displayed in the high level syntax. Additionally, parameters may also be transmitted to change the depth focus.

In one embodiment, we propose a new Supplemental Enhancement Information (SEI) message that signals multi-view information in an MPEG-4 AVC standard compliant bitstream, wherein each picture includes sub-pictures belonging to different views. Embodiments are intended for easy and convenient display of multi-view video streams on, for example, three-dimensional (3D) monitors that can utilize such a framework. The concept can be extended to other video coding standards and recommendations for signaling this information using high level syntax.

Further, in one embodiment, we propose a signaling method that arranges views before being sent to a multi-view video encoder and / or decoder. Advantageously, this embodiment can lead to a simplified implementation of multi-view coding and can benefit coding efficiency. Certain views may be grouped together and may form a pseudo-view or a super-view, and then the tiled superviews may be combined in accordance with the current MPEG-4 AVC standard, for example based on the implementation of multi- View video encoder and / or decoder. A new flag in the sequence parameter set (SPS) of multi-view video coding is proposed to signal the use of the description of pseudo views. One embodiment is intended for easy and convenient display of a multi-view video stream on 3D monitors that can utilize such a framework.

Encoding / decoding using single-view video encoding / decoding standards / recommendations

Part 10 AVC (Advanced Video Coding) standard / ITU-T (International Telecommunication Union, Telecommunication Sector) H.264 Recommendation (ISO / IEC) MPEG-4 Moving Picture Experts Group- In the current implementation of multi-video coding (MVC) based on the MPEG-4 AVC standard, the reference software encodes each view into one encoder, . Each view is coded as a separate bitstream by the encoder at the original resolution, and all bitstreams are combined to form a single bitstream that is later decoded. Each view produces a separate YUV decoded output.

Another method for multi-view prediction involves tiling images from each view (sampled concurrently) on a large frame or superframe with possible downstream operations. Referring to FIG. 1, an example of four views tiled on a single frame is generally indicated by reference numeral 100. Referring to FIG. 2, an example of four views flipped and tiled on a single frame is generally indicated by reference numeral 200. As shown in FIG. Thus, if there are four views, the images from each view are arranged in a super-frame, such as a tile. This results in a single non-coded input sequence with large resolution. This signal can then be encoded using existing video coding standards such as the ISO / IEC MPEG-2 standard and the MPEG-4 AVC standard.

Another method for multi-view prediction involves simply encoding the different views independently using the new standard and tiling the views needed by the player after decoding.

Many display manufacturers use this framework to arrange or tile different views on a single frame and then extract and render views from their respective locations. In these cases, there is no standard way of determining whether a bitstream has this property. Thus, if the system uses a method of tiling images of different views in a large frame, the method of extracting the different views is expedient.

Referring to FIG. 3, a video encoder capable of performing video encoding in accordance with the MPEG-4 AVC standard is generally indicated by reference numeral 300.

The video encoder 300 includes a frame ordering buffer 310 having an output coupled to communicate a signal with the non-inverting input of the combiner 385. The output of the combiner 385 is coupled to communicate a signal with a first input of a transducer and a quantizer 325. The output of the transducer and quantizer 325 is connected to communicate a signal with a first input of the entropy coder 345 and a first input of the inverse transformer and dequantizer 350. The output of entropy coder 345 is coupled to communicate a signal with a first non-inverting input of combiner 390. The output of combiner 390 is coupled to communicate a signal with a first input of output buffer 335.

The first output of the encoder controller 305 is coupled to a second input of a frame ordering buffer 310, a second input of an inverse transformer and dequantizer 350, an input of a picture-type determining module 315, (MB-type) decision block 320, a second input of an intra prediction module 360, a second input of a deblocking filter 365, a first input of a motion compensator 370, a motion estimator 375, a second input of the reference picture buffer 380, and a signal.

The second input of the encoder controller 305 is coupled to a first input of a SEI (Supplemental Enhancement Information) inserter 330, a second input of a transducer and a quantizer 325, a second input of an entropy coder 345, (SPS) and the picture parameter set (PPS) inserter 340 of the first input 335 of the second set of instructions, the sequence parameter set (SPS) and the picture parameter set (PPS)

The first input of the picture-type determination module 315 is coupled to communicate a signal with a third input of the frame ordering buffer 310. [ The second output of the picture-type determination module 315 is coupled to communicate a signal with a second input of the macroblock-type determination module 320. [

The output of the sequence parameter set (SPS) and picture parameter set (PPS) inserter 340 is coupled to communicate a signal with a third non-inverting input of the combiner 390. The output of the SEI inserter 330 is connected to the second non-inverting input of the combiner 390 to communicate signals.

The outputs of the inverse quantizer and inverse transformer 350 are connected to communicate signals with the first non-inverting input of the combiner 319. The output of combiner 319 is coupled to communicate a signal with a first input of intra prediction module 360 and a first input of deblocking filter 365. The output of deblocking filter 365 is coupled to communicate a signal with a first input of reference image buffer 380. The output of the reference picture buffer 380 is coupled to communicate a signal with a second input of the motion estimator 375 and a first input of the motion compensator 370. A first output of the motion estimator 375 is coupled to communicate a signal with a second input of the motion compensator 370. A second output of the motion estimator 375 is coupled to communicate a signal with a third input of the entropy coder 345.

The output of the motion compensator 370 is coupled to communicate a signal with the first input of the switch 397. The output of intra prediction module 360 is coupled to communicate a signal with a second input of switch 397. The output of the macroblock-type determination module 320 is coupled to communicate a signal with a third input of the switch 397 to provide a control signal to the switch 397. The third input of switch 397 determines whether the "data" input of the switch (compared to the control input, i.e. the third input), is provided by motion compensator 370 or intra prediction module 360. The output of the switch 397 is connected to the second non-inverting input of the combiner 319 and the inverting input of the combiner 385 to communicate signals.

The inputs of the frame ordering buffer 310 and the encoder controller 105 are available as inputs to an encoder 300 that receives an input image 301. [ In addition, the input of the Supplemental Enhancement Information (SEI) inserter 330 is available as an input to the encoder 300 that receives the metadata. The output of the output buffer 335 is available as an output of the encoder 300 that outputs the bit stream.

Referring to FIG. 4, a video decoder capable of performing video decoding in accordance with the MPEG-4 AVC standard is generally indicated by reference numeral 400.

The video decoder 400 includes an input buffer 410 having an output coupled to communicate a signal with a first input of an entropy decoder 445. A first output of the entropy decoder 445 is coupled to communicate a signal with a first input of the inverse transformer and dequantizer 450. The output of the inverse transformer and dequantizer 450 is coupled to communicate a signal with a second non-inverting input of the combiner 425. The output of combiner 425 is coupled to communicate a signal with a second input of deblocking filter 465 and a first input of intra prediction module 460. A second output of deblocking filter 465 is coupled to communicate a signal with a first input of reference image buffer 480. The output of the reference picture buffer 480 is connected to communicate a signal with a second input of the motion compensator 470.

A second input of the entropy decoder 445 is coupled to communicate a signal with a third input of the motion compensator 470 and a first input of the deblocking filter 465. A third output of the entropy decoder 445 is coupled to communicate a signal with the input of the decoder controller 405. The first output of decoder controller 405 is coupled to communicate a signal with a second input of entropy decoder 445. A second output of the decoder controller 405 is coupled to communicate a signal with a second input of the inverse transformer and the inverse quantizer 450. A third input of controller 405 is coupled to communicate a signal with a third input of deblocking filter 465. The fourth output of decoder controller 405 is coupled to communicate a signal with a second input of intra prediction module 460, a first input of motion compensator 470 and a second input of reference picture buffer 480.

The output of the motion compensator 470 is coupled to communicate a signal with a first input of the switch 497. The output of intraprediction module 460 is coupled to communicate a signal with a second input of switch 497. The output of the switch 497 is connected to communicate a signal with the first non-inverting input of the combiner 425.

The input of the input buffer 410 is available as an input to a decoder 400 that receives the input stream. The first output of deblocking filter 465 is available as an output of decoder 400 that outputs an output image.

Referring to FIGS. 5A and 5B, an exemplary method of encoding images for multiple views using the MPEG-4 AVC standard is generally indicated by reference numeral 500.

The method 500 includes a start block 502 that passes control to a function block 504. The function block 504 arranges each view of a particular time instance as a sub-picture of the tile format and passes control to a function block 506. [ The function block 506 sets a syntax element num_coded_view_minus1 and passes control to a function block 508. [ The function block 508 sets the syntax elements org_pic_width_in_mbs_minus1 and org_pic_height_in_mbs_minus1 and passes control to a function block 510. The function block 510 sets the variable i to zero and passes control to a decision block 512. Decision block 512 determines if variable i is less than the number of views. If so, control passes to a function block 514. Otherwise, control passes to a function block 524.

The function block 514 sets the syntax element view_id [i] and passes control to a function block 516. The function block 516 sets the syntax element num_parts [view_id [i]] and passes control to a function block 518. [ The function block 518 sets the variable j to zero and passes control to the decision block 520. [ The decision block 520 determines if the current value of the variable j is less than the current value of the syntax element num_parts [view_id [i]]. If so, control passes to a function block 522. Otherwise, control is passed to a function block 528.

The function block 522 sets the following syntax elements, increases the variable j, and then returns control to the decision block 520:

Figure 112009067731434-pct00001

The function block 528 sets the syntax element upsample_view_flag [view_id [i]] and passes control to a decision block 530. The decision block 530 determines whether the value of the syntax element upsample_view_flag [view_id [i]] is one. If so, control passes to a function block 532. Otherwise, control passes to decision block 534.

The function block 532 sets the syntax element upsample_filter [view_id [i]] and passes control to a decision block 534.

The decision block 534 determines whether the current value of the syntax element upsample_filter [view_id [i]] is three. If so, control passes to a function block 536. Otherwise, control passes to a function block 540.

The function block 536 includes the following syntax elements: vert_dim [view_id [i]]; hor_dim [view_id [i]]; And the quantizer [view_id [i]], and passes control to a function block 538.

 The function block 538 sets the filter coefficients for each YUV component and passes control to the control block 540.

The function block 540 increments the variable i and returns control to decision block 512.

The function block 524 records these syntax elements in at least one of a Sequence Parameter Set (SPS), a Picture Parameter Set (PPS), a Supplemental Enhancement Information (SEI) message, a Network Abstraction Layer , And passes control to a function block 526. The function block 526 encodes each picture using the MPEG-4 AVC standard or another single view codec, and passes control to the end block 599.

Referring to FIGS. 6A and 6B, an exemplary method for decoding an image for a plurality of views using the MPEG-4 AVC standard is generally indicated by reference numeral 600.

The method 600 includes a start block 602 that passes control to a function block 604. [ The function block 604 parses the following syntax elements from at least one of Sequence Parameter Set (SPS), Picture Parameter Set (PPS), Supplemental Enhancement Information (SEI) message, Network Abstraction Layer (NAL) parsing, and passes control to a function block 606. The function block 606 parses the syntax element num_coded_views_minus1 and passes control to a function block 608. The function block 608 parses the syntax elements org_pic_width_in_mbs_minus1 and org_pic_height_in_mbs_minus1 and passes control to a function block 610. The function block 610 sets the variable i to zero and passes control to a decision block 612. The decision block 612 determines if the variable i is less than the number of views. If so, control passes to a function block 614. Otherwise, control passes to a function block 624.

The function block 614 parses the syntax element view_id [i] and passes control to a function block 616. The function block 616 parses the syntax element num_parts_minus1 [view_id [i]] and passes control to a function block 618. The function block 618 sets the variable j to zero and passes control to a decision block 620. [ The decision block 620 determines if the current value of the variable j is less than the value of the syntax element num_parts [view_id [i]]. If so, control is passed to a function block 622. Otherwise, control is passed to a function block 628.

The function block 622 parses the following syntax elements, increments the variable j, and then returns control to decision block 620:

Figure 112009067731434-pct00002

The function block 628 parses the syntax element upsample_view_flag [view_id [i]] and passes control to a decision block 630. The decision block 630 determines whether the current value of the syntax element upsample_view_flag [view_id [i]] is one. If so, control is passed to a function block 632. Otherwise control is passed to decision block 634.

The function block 632 parses the syntax element upsample_filter [view_id [i]] and passes control to a decision block 634.

The decision block 634 determines whether the current value of the syntax element upsample_filter [view_id [i]] is three. If so, control is passed to a function block 636. Otherwise control is passed to a function block 640.

The function block 636 parses the following syntax elements: vert_dim [view_id [i]], hor_dim [view_id [i]] and quantizer [view_id [i]] and passes control to a function block 638 .

The function block 638 parses the filter coefficients for each YUV component and passes control to a function block 640.

The function block 640 increments the variable i and returns control to decision block 612.

The function block 624 decodes each picture using the MPEG-4 AVC standard or other single view codec and passes control to a function block 626. [ The function block 626 separates each view from the image using a high level syntax and passes control to an end block 699.

7A and 7B, an exemplary method of encoding an image for a plurality of views and depths using the MPEG-4 AVC standard is generally indicated by reference numeral 700.

The method 700 includes a start block 702 that passes control to a function block 704. The function block 704 arranges each view and corresponding depth of a particular time instance as a sub-picture of the file format, and passes control to a function block 706. [ The function block 706 sets the syntax element num_coded_view_minus1 and passes control to a function block 708. The function block 708 sets the syntax elements org_pic_width_in_mbs_minus1 and org_pic_height_in_mbs_minus1 and passes control to a function block 710. The function block 710 sets the variable i to zero and passes control to a decision block 712. The decision block 712 determines if the variable i is less than the number of views. If so, control is passed to a function block 714. If not, control is passed to a function block 724.

The function block 714 sets the syntax element view_id [i] and passes control to a function block 716. The function block 716 sets the syntax element num_parts [view_id [i]] and passes control to a function block 718. The function block 718 sets the variable j to zero and passes control to a decision block 720. The decision block 720 determines if the current value of the variable j is less than the current value of the syntax element num_parts [view_id [i]]. If so, control is passed to a function block 722. Otherwise, control passes to a function block 728.

The function block 722 sets the following syntax elements, increases the variable j, and then returns control to decision block 720:

Figure 112009067731434-pct00003

The function block 728 sets the syntax element upsample_view_flag [view_id [i]] and passes control to a decision block 730. The decision block 730 determines whether the value of the syntax element upsample_view_flag [view_id [i]] is one. If so, control passes to a function block 732. Otherwise, control passes to decision block 734.

The function block 732 sets the syntax element upsample_filter [view_id [i]] and passes control to a decision block 734.

The decision block 734 determines whether the current value of the syntax element upsample_filter [view_id [i]] is three. If so, control passes to a function block 736. Otherwise, control is passed to a function block 740.

Function block 736 includes the following syntax elements: vert_dim [view_id [i]]; hor_dim [view_id [i]]; And quantizer [view_id [i]], and passes control to a function block 738. [

 The function block 738 sets the filter coefficients for each YUV component and passes control to the control block 740.

The function block 740 increments the variable i and returns control to decision block 712.

The function block 724 records these syntax elements in at least one of a Sequence Parameter Set (SPS), a Picture Parameter Set (PPS), a Supplemental Enhancement Information (SEI) message, a Network Abstraction Layer , And passes control to a function block 726. The function block 726 encodes each picture using the MPEG-4 AVC standard or other single view codec, and passes control to the end block 799.

8A and 8B, an exemplary method for decoding an image for a plurality of views and depths using the MPEG-4 AVC standard is generally indicated by reference numeral 800.

The method 800 includes a start block 802 that passes control to a function block 804. The function block 804 parses the following syntax elements from at least one of a Sequence Parameter Set (SPS), a Picture Parameter Set (PPS), a Supplemental Enhancement Information (SEI) message, a Network Abstraction Layer , And passes control to a function block 806. The function block 806 parses the syntax element num_coded_views_minus1 and passes control to a function block 808. The function block 808 parses the syntax elements org_pic_width_in_mbs_minus1 and org_pic_height_in_mbs_minus1 and passes control to a function block 810. The function block 810 sets the variable i to zero and passes control to the decision block 812. Decision block 812 determines if variable i is less than the number of views. If so, control is passed to a function block 814. Otherwise, control is passed to a function block 824.

The function block 814 parses the syntax element view_id [i] and passes control to a function block 816. The function block 816 parses the syntax element num_parts_minus1 [view_id [i]] and passes control to a function block 818. The function block 818 sets the variable j to zero and passes control to a decision block 820. The decision block 820 determines if the current value of the variable j is less than the value of the syntax element num_parts [view_id [i]]. If so, control is passed to a function block 822. Otherwise, control is passed to a function block 828.

The function block 822 parses the following syntax elements, increments the variable j, and then returns control to decision block 820:

Figure 112009067731434-pct00004

The function block 828 parses the syntax element upsample_view_flag [view_id [i]] and passes control to a decision block 830. The decision block 830 determines whether the current value of the syntax element upsample_view_flag [view_id [i]] is one. If so, control is passed to a function block 832. Otherwise control is passed to decision block 834.

The function block 832 parses the syntax element upsample_filter [view_id [i]] and passes control to a decision block 834.

The decision block 834 determines whether the current value of the syntax element upsample_filter [view_id [i]] is three. If so, control is passed to a function block 836. Otherwise control is passed to a function block 840.

The function block 836 parses the following syntax elements: vert_dim [view_id [i]], hor_dim [view_id [i]] and quantizer [view_id [i]] and passes control to a function block 838 .

The function block 838 parses the filter coefficients for the YUV component and passes control to a function block 840.

The function block 840 increments the variable i and returns control to the decision block 812.

The function block 824 decodes each picture using the MPEG-4 AVC standard or other single view codec and passes control to a function block 826. [ The function block 826 separates each view and corresponding depth from the image using the high level syntax and passes control to a function block 827. [ The function block 827 potentially performs view parsing using the extracted view and depth signals and passes it to the end block 899. [

For the depths used in Figures 7 and 8, Figure 9 shows an example of a depth signal 900 in which the depth is provided as a pixel value for the position of each corresponding image (not shown). Figure 10 also shows an example of two depth signals included in the tile 1000. [ The upper-right portion of the tile 1000 is a depth signal having depth values corresponding to the image on the upper-left side of the tile 1000. The lower-right portion of the tile 1000 is a depth signal having depth values corresponding to the image on the lower-left side of the tile 1000.

Referring to Fig. 11, an example of five views tiled on a single frame is generally indicated by reference numeral 1100. [ The top four views are in the normal direction. The fifth view is also in the normal direction, but is divided into two parts along the bottom of the tile 1100. The left-hand portion of the fifth view shows the "top" of the fifth view, and the right-hand portion of the fifth view shows the "bottom"

Encoding / decoding using multi-view video encoding / decoding standards / recommendations

Referring to FIG. 12, an exemplary multi-view video coding (MVC) encoder is generally indicated by reference numeral 1200. Encoder 1200 includes a combiner 1205 having an output coupled to communicate a signal with an input of a converter 1210. The output of the converter 1210 is coupled to communicate the signal with the input of the quantizer 1215. The output of quantizer 1215 is connected to the input of entropy coder 1220 and to the input of dequantizer 1225 to communicate signals. The output of the dequantizer 1225 is connected to communicate the signal with the input of the inverse transformer 1230. The output of the inverse transformer 1230 is coupled to communicate a signal with the first non-inverting input of the combiner 1235. The output of the combiner 1235 is connected to the input of the intra predictor 1245 and to the input of the deblocking filter 1250 to communicate signals. The output of deblocking filter 1250 is coupled to communicate a signal with an input of reference image store 1255 (for view i). The output of the reference image store 1255 is coupled to communicate a signal with a first input of the motion compensator 1275 and a first input of the motion estimator 1280. The output of motion estimator 1280 is coupled to communicate a signal with a second input of motion compensator 1275.

The output of the reference image store 1260 (for other views) is coupled to communicate a signal with a first input of a disparity estimator 1270 and a first input of a disparity compensator 1265. The output of disparity estimator 1270 is coupled to communicate a signal with a second input of disparity compensator 1265.

The output of the entropy decoder 1220 is available as an output of the encoder 1200. The non-inverting input of combiner 1205 is available as an input to encoder 1200 and is coupled to communicate a signal with a second input of disparity estimator 1270 and a second input of motion estimator 1280. The output of the switch 1285 is connected to the second non-inverting input of the combiner 1235 and the inverting input of the combiner 1205 to communicate signals. The switch 1285 has a first input coupled to communicate a signal with the output of the motion compensator 1275, a second input coupled to communicate a signal with the output of the disparity compensator 1265, and an output of the intra predictor 1245 and a signal And a third input coupled to communicate.

Mode determination module 1240 has an output coupled to switch 1285 to control which input is selected by switch 1285. [

Referring to FIG. 13, an exemplary multi-view video coding (MVC) decoder is generally indicated by reference numeral 1300. Decoder 1300 includes an entropy decoder 1305 having an output coupled to communicate a signal with an input of dequantizer 1310. [ The output of the inverse quantizer is coupled to communicate the signal with the input of the inverse transformer 1315. [ The output of the inverse converter 1315 is coupled to communicate a signal with the first non-inverting input of the combiner 1320. The output of the combiner 1320 is coupled to communicate the signal to the input of the de-blocking filter 1325 and the input of the intra predictor 1330. The output of deblocking filter 1325 is coupled to communicate a signal with an input of reference image store 1340 (for view i). The output of reference image store 1340 is coupled to communicate a signal with a first input of motion compensator 1335.

The output of reference image store 1345 (for other views) is connected to communicate a signal with a first input of disparity compensator 1350.

The input of the entropy coder 1305 is available as an input to the decoder 1300 to receive the residual bitstream. The input of the mode module 1360 is also available as an input to the decoder 1300 to receive the control syntax to control which input was selected by the switch 1355. [ A second input of motion compensator 1335 is also available as an input to decoder 1300 to receive motion vectors. The second input of disparity compensator 1350 is also available as an input to decoder 1300 to receive disparity vectors.

The output of switch 1355 is coupled to communicate a signal with a second non-inverting input of combiner 1320. The first input of the switch 1355 is connected to communicate the signal with the output of the disparity compensator 1350. A second input of the switch 1355 is connected to communicate the signal with the output of the motion compensator 1335. A third input of the switch 1355 is connected to communicate the signal with the output of the intra predictor 1330. The output of mode module 1360 is connected to communicate a signal with switch 1355 to control which input is selected by switch 1355. [ The output of deblocking filter 1325 is available as an output of decoder 1300.

14, an exemplary method of processing images for a plurality of views against encoding of images using a multi-view video coding (MVC) extension of the MPEG-4 AVC standard is described generally by reference numeral 1400 It is displayed in outline.

The method 1400 includes a start block 1405 that passes control to a function block 1410. The function block 1410 arranges all N views between the total M views at a particular time instance as a super-picture of the tile format and passes control to a function block 1415. [ The function block 1415 sets the syntax element num_coded_views_minus1, and passes control to a function block 1420. The function block 1420 sets the syntax element view_id [i] for all (num_coded_views_minus1 + 1) views and passes control to a function block 1425. The function block 1425 sets up inter-view reference dependency information for the anchor pictures and passes control to a function block 1430. The function block 1430 sets the inter-view reference dependency information for the non-anchor pictures and passes control to a function block 1435. The function block 1435 sets the syntax element pseudo_view_present_flag and passes control to a decision block 1440. Decision block 1440 determines if the current value of the syntax element pseudo_view_present_flag is true. If so, control is passed to a function block 1445. Otherwise, control is passed to an end block 1499.

Function block 1445 includes the following syntax elements: tiling_mode; org_pic_width_in_mbs_minus1; And org_pic_height_in_mbs_minus1, and passes control to a function block 1450. The function block 1450 calls the syntax element pseudo_view_info (view_id) for each coded view and passes control to the end block 1499.

15A and 15B, an exemplary method for encoding images for multiple views using a multi-view video coding (MVC) extension of the MPEG-4 AVC standard is generally indicated by reference numeral 1500 .

The method 1500 includes a start block 1502 that has an input parameter pseudo_view_id and passes control to a function block 1504. The function block 1504 sets the syntax element num_sub_views_minus 1 and passes control to a function block 1506. The function block 1506 sets the variable i to zero and passes control to a decision block 1508. [ Decision block 1508 determines if variable i is less than the number of sub_views. If so, control is passed to a function block 1510. If not, control is passed to a function block 1520.

The function block 1510 sets the syntax element sub_view_id [i] and passes control to a function block 1512. The function block 1512 sets the syntax element num_parts_minus1 [sub_view_id [i]] and passes control to a function block 1514. The function block 1514 sets the variable j to zero and passes control to a decision block 1516. [ Decision block 1516 determines if the variable j is less than the syntax element num_parts_minus1 [sub_view_id [i]]. If so, control passes to a function block 1518. Otherwise, control passes to decision block 1522.

The function block 1518 sets the following syntax elements, increments the variable j, and returns control to decision block 1516:

Figure 112009067731434-pct00005

The function block 1520 encodes the current picture for the current view using multi-view video coding (MVC) and passes control to an end block 1599.

The decision block 1522 determines if the syntax element tiling_mode is zero. If so, control passes to a function block 1524. Otherwise, control passes to a function block 1538.

The function block 1524 sets the syntax element flip_dir [sub_view_id [i]] and the syntax element upsample_view_flag [sub_view_id [i]] and passes control to decision block 1526. The decision block 1526 determines whether the syntax element upsample_view_flag [sub_view_id [i]] is zero. If so, control passes to a function block 1528. Otherwise, control passes to decision block 1530.

The function block 1528 sets the syntax element upsample_filter [sub_view_id [i]] and passes control to decision block 1530. The decision block 1530 determines whether the syntax element upsample_filter [sub_view_id [i]] is three. If so, control passes to a function block 1532. Otherwise, control passes to a function block 1536.

The function block 1532 includes the following syntax elements: vert_dim [sub_view_id [i]]; hor_dim [sub_view_id [i]]; And a quantizer [sub_view_id [i]], and passes control to a function block 1534. The function block 1534 sets filter coefficients for each YUV component and passes control to a function block 1536. [

The function block 1536 increments the variable i and returns control to decision block 1508.

The function block 1538 sets the syntax element pixel_dist_x [sub_view_id [i]] and the syntax element flip_dist_y [sub_view_id [i]] and passes control to a function block 1540. The function block 1540 sets the variable j to 0 and passes control to a decision block 1542. [ Decision block 1542 determines if the current value of variable j is less than the current value of the syntax element num_parts [sub_view_id [i]]. If so, control passes to a function block 1544. Otherwise, control passes to a function block 1536.

The function block 1544 sets the syntax element num_pixel_tiling_filter_coeffs_minus1 [sub_view_id [i]] and passes control to a function block 1546. The function block 1546 sets the coefficients for all pixel tiling filters and passes control to a function block 1536.

16, an exemplary method for processing images for a plurality of views against decoding of images using a multi-view video coding (MVC) extension of the MPEG-4 AVC standard is described generally by reference numeral 1600 Is displayed.

The method 1600 includes a start block 1605 that passes control to a function block 1615. [ The function block 1615 parses the syntax element num_coded_views_minus1 and passes control to a function block 1620. The function block 1620 parses the syntax element view_id [i] for all (num_coded_views_minus1 + 1) views and passes control to a function block 1625. The function block 1625 parses the inter-view reference dependency information for the anchor pictures and passes control to a function block 1630. [ The function block 1630 parses the inter-view reference dependency information for the non-anchor pictures and passes control to a function block 1635. The function block 1635 parses the syntax element pseudo_view_present_flag and passes control to a decision block 1640. Decision block 1640 determines if the current value of the syntax element pseudo_view_present_flag is true. If so, control is passed to a function block 1645. Otherwise, control is passed to end block 1699.

Function block 1645 includes the following syntax elements: tiling_mode; org_pic_width_in_mbs_minus1; And org_pic_height_in_mbs_minus1, and passes control to a function block 1650. The function block 1650 calls the syntax element pseudo_view_info (view_id) for each coded view and passes control to the end block 1699.

17A and 17B, an exemplary method for decoding pictures for a plurality of views using a multi-view video coding (MVC) extension of the MPEG-4 AVC standard is generally indicated by reference numeral 1700 .

The method 1700 includes a start block 1702 that starts with an input parameter pseudo_view_id and passes control to a function block 1704. The function block 1704 parses the syntax element num_sub_views_minus1 and passes control to a function block 1706. The function block 1706 sets the variable i to zero and passes control to a decision block 1708. Decision block 1708 determines if variable i is less than the number of sub_views. If so, control passes to a function block 1710. If not, control is passed to a function block 1720.

The function block 1710 parses the syntax element sub_view_id [i] and passes control to a function block 1712. The function block 1712 parses the syntax element num_parts_minus1 [sub_view_id [i]] and passes control to a function block 1714. The function block 1714 sets the variable j to zero and passes control to a decision block 1716. Decision block 1716 determines if the variable j is less than the syntax element num_parts_minus1 [sub_view_id [i]]. If so, control passes to a function block 1718. Otherwise, control passes to decision block 1722.

The function block 1718 sets the following syntax elements, increments the variable j, and returns control to decision block 1716:

Figure 112009067731434-pct00006

The function block 1720 decodes the current picture for the current view using multi-view video coding (MVC) and passes control to a function block 1721. The function block 1721 uses the high level syntax to separate each view from the image and passes control to the end block 1799. [

The separation of each view from the decoded picture is done using the high level syntax displayed in the bitstream. This high-level syntax can display the correct position and the possible directions (and possible corresponding depths) of views present in the image.

Decision block 1722 determines if the syntax element tiling_mode is zero. If so, control is passed to a function block 1724. Otherwise, control passes to a function block 1738.

The function block 1724 parses the syntax element flip_dir [sub_view_id [i]] and the syntax element upsample_view_flag [sub_view_id [i]] and passes control to decision block 1726. The decision block 1726 determines if the syntax element upsample_view_flag [sub_view_id [i]] is zero. If so, control passes to a function block 1728. Otherwise, control passes to decision block 1730.

The function block 1728 parses the syntax element upsample_filter [sub_view_id [i]] and passes control to decision block 1730. The decision block 1730 determines whether the syntax element upsample_filter [sub_view_id [i]] is three. If so, control passes to a function block 1732. Otherwise, control passes to a function block 1736.

Function block 1732 includes the following syntax elements: vert_dim [sub_view_id [i]]; hor_dim [sub_view_id [i]]; And quantizer [sub_view_id [i]], and passes control to a function block 1734. The function block 1734 parses the filter coefficients for each YUV component and passes control to a function block 1736.

The function block 1736 increments the variable i and returns control to decision block 1708.

The function block 1738 parses the syntax element pixel_dist_x [sub_view_id [i]] and the syntax element flip_dist_y [sub_view_id [i]] and passes control to a function block 1740. The function block 1740 sets the variable j to 0 and passes control to a decision block 1742. Decision block 1742 determines if the current value of the variable j is less than the current value of the syntax element num_parts [sub_view_id [i]]. If so, control passes to a function block 1744. Otherwise, control passes to a function block 1736.

The function block 1744 parses the syntax element num_pixel_tiling_filter_coeffs_minus1 [sub_view_id [i]] and passes control to a function block 1746. The function block 1746 parses the coefficients for all pixel tiling filters and passes control to a function block 1736.

18, an exemplary method for processing images for multiple views and depths in preparation for encoding images using a multi-view video coding (MVC) extension of the MPEG-4 AVC standard is described generally in reference numeral 1800 ).

The method 1800 includes a start block 1805 that passes control to a function block 1810. The function block 1810 arranges all of the N views and depth maps in the total M views and depth maps at a particular time instance as a super-picture of the tile format and passes control to a function block 1815. The function block 1815 sets the syntax element num_coded_views_minus1 and passes control to a function block 1820. The function block 1820 sets the syntax element view_id [i] for all (num_coded_views_minus1 + 1) depths corresponding to view_id [i] and passes control to a function block 1825. The function block 1825 sets the inter-view reference dependency information for the anchor depth images and passes control to a function block 1830. The function block 1830 sets the inter-view reference dependency information for the non-anchor depth images and passes control to a function block 1835. The function block 1835 sets the syntax element pseudo_view_present_flag and passes control to a decision block 1840. Decision block 1840 determines if the current value of the syntax element pseudo_view_present_flag is true. If so, control passes to a function block 1845. Otherwise, control passes to end block 1899.

Function block 1845 includes the following syntax elements: tiling_mode; org_pic_width_in_mbs_minus1; And org_pic_height_in_mbs_minus1 and passes control to a function block 1850. Function block 1850 calls pseudo_view_info (view_id) for each coded view and passes control to end block 1899.

19A and 19B, an exemplary method for encoding images for multiple views and depths using a multi-view video coding (MVC) extension of the MPEG-4 AVC standard is described generally in reference numeral 1900 Lt; / RTI >

The method 1900 includes a start block 1902 that passes control to a function block 1904. The function block 1904 sets the syntax element num_sub_views_minus1 and passes control to a function block 1906. The function block 1906 sets the variable i to zero and passes control to a decision block 1908. Decision block 1908 determines if variable i is less than the number of sub_views. If so, control is passed to a function block 1910. If not, control is passed to a function block 1920.

The function block 1910 sets the syntax element sub_view_id [i] and passes control to a function block 1912. The function block 1912 sets the syntax element num_parts_minus1 [sub_view_id [i]] and passes control to a function block 1914. The function block 1914 sets the variable j to zero and passes control to a decision block 1916. The decision block 1916 determines if the variable j is less than the syntax element num_parts_minus1 [sub_view_id [i]]. If so, control passes to a function block 1918. Otherwise, control passes to decision block 1922.

The function block 1918 sets the following syntax elements, increments the variable j, and returns control to decision block 1916:

Figure 112009067731434-pct00007

The function block 1920 encodes the current depth for the current view using multi-view video coding (MVC) and passes control to the end block 1999. The depth signal may be encoded similar to the manner in which its corresponding video signal is encoded. For example, a depth signal for a view may be included only on different depth signals, or only on video signals, or on a tile including both depth and video signals. The tile (pseudo-view) is then treated as a single view for the MVC, and there are also other tiles, possibly treated as different views for the MVC.

The decision block 1922 determines if the syntax element tiling_mode is zero. If so, control is passed to a function block 1924. Otherwise, control passes to a function block 1938.

The function block 1924 sets the syntax element flip_dir [sub_view_id [i]] and the syntax element upsample_view_flag [sub_view_id [i]] and passes control to a decision block 1926. The decision block 1926 determines whether the current value of the syntax element upsample_view_flag [sub_view_id [i]] is one. If so, control is passed to a function block 1928. Otherwise control is passed to decision block 1930.

The function block 1928 sets the syntax element upsample_filter [sub_view_id [i]] and passes control to a decision block 1930. The decision block 1930 determines whether the value of the syntax element upsample_filter [sub_view_id [i]] is three. If so, control is passed to a function block 1932. Otherwise, control is passed to a function block 1936.

Function block 1932 includes the following syntax elements: vert_dim [sub_view_id [i]]; hor_dim [sub_view_id [i]]; And quantizer [sub_view_id [i]], and passes control to a function block 1934. A function block 1934 sets filter coefficients for each YUV component and passes control to a function block 1936.

The function block 1936 increments the variable i and returns control to decision block 1908.

The function block 1938 sets the syntax element pixel_dist_x [sub_view_id [i]] and the syntax element flip_dist_y [sub_view_id [i]] and passes control to a function block 1940. The function block 1940 sets the variable j to zero and passes control to a decision block 1942. Decision block 1942 determines if the current value of the variable j is less than the current value of the syntax element num_parts [sub_view_id [i]]. If so, control is passed to a function block 1944. Otherwise, control is passed to a function block 1936.

The function block 1944 sets the syntax element num_pixel_tiling_filter_coeffs_minus1 [sub_view_id [i]] and passes control to a function block 1946. The function block 1946 sets the coefficients for all pixel tiling filters and passes control to a function block 1936.

Referring to FIG. 20, an exemplary method for processing images for multiple views and depths in preparation for decoding images using a multi-view video coding (MVC) extension of the MPEG-4 AVC standard is described generally in reference numeral 2000 ).

The method 2000 includes a start block 2005 that passes control to a function block 2015. [ The function block 2015 parses the syntax element num_coded_views_minus1 and passes control to a function block 2020. The function block 2020 parses the syntax element view_id [i] for all (num_coded_views_minus1 + 1) depths corresponding to view_id [i] and passes control to a function block 2025. The function block 2025 parses the inter-view reference dependency information for the anchor depth images and passes control to a function block 2030. [ The function block 2030 parses the inter-view reference dependency information for non-anchor depth images and passes control to a function block 2035. The function block 2035 parses the syntax element pseudo_view_present_flag and passes control to a decision block 2040. The decision block 2040 determines if the current value of the syntax element pseudo_view_present_flag is true. If so, control is passed to a function block 2045. Otherwise, control is passed to end block 2099.

Function block 2045 includes the following syntax elements: tiling_mode; org_pic_width_in_mbs_minus1; And org_pic_height_in_mbs_minus1, and passes control to a function block 2050. The function block 2050 calls pseudo_view_info (view_id) for each coded view and passes control to the end block 2099.

21A and 21B, an exemplary method for decoding pictures for multiple views and depths using a multi-view video coding (MVC) extension of the MPEG-4 AVC standard is described generally in reference numeral 2100 Lt; / RTI >

The method 2100 includes a start block 2102 that starts with an input parameter pseudo_view_id and passes control to a function block 2104. The function block 2104 parses the syntax element num_sub_views_minus1 and passes control to a function block 2106. The function block 2106 sets the variable i to zero and passes control to a decision block 2108. [ Decision block 2108 determines if variable i is less than the number of sub_views. If so, control is passed to a function block 2110. If not, control is passed to a function block 2120.

The function block 2110 parses the syntax element sub_view_id [i] and passes control to a function block 2112. The function block 2112 parses the syntax element num_parts_minus1 [sub_view_id [i]] and passes control to a function block 2114. The function block 2114 sets the variable j to zero and passes control to a decision block 2116. [ Decision block 2116 determines if the variable j is less than the syntax element num_parts_minus1 [sub_view_id [i]]. If so, control is passed to a function block 2118. Otherwise, control passes to decision block 2122.

The function block 2118 sets the following syntax elements, increments the variable j, and returns control to decision block 2116:

Figure 112009067731434-pct00008

The function block 2120 decodes the current picture using multi-view video coding (MVC) and passes control to a function block 2121. The function block 2121 separates each view from the image using the high level syntax and passes control to the end block 2199. [ The separation of each view from the image using the high level syntax is as described above.

The decision block 2122 determines if the syntax element tiling_mode is zero. If so, control is passed to a function block 2124. Otherwise, control is passed to a function block 2138.

The function block 2124 parses the syntax element flip_dir [sub_view_id [i]] and the syntax element upsample_view_flag [sub_view_id [i]] and passes control to decision block 2126. The decision block 2126 determines whether the current value of the syntax element upsample_view_flag [sub_view_id [i]] is one. If so, control passes to a function block 2128. Otherwise control is passed to decision block 2130.

The function block 2128 parses the syntax element upsample_filter [sub_view_id [i]] and passes control to decision block 2130. The decision block 2130 determines whether the value of the syntax element upsample_filter [sub_view_id [i]] is three. If so, control passes to a function block 2132. Otherwise, control is passed to a function block 2136.

Function block 2132 includes the following syntax elements: vert_dim [sub_view_id [i]]; hor_dim [sub_view_id [i]]; And quantizer [sub_view_id [i]], and passes control to a function block 2134. The function block 2134 parses the filter coefficients for each YUV component and passes control to a function block 2136.

The function block 2136 increments the variable i and returns control to decision block 2108.

The function block 2138 parses the syntax element pixel_dist_x [sub_view_id [i]] and the syntax element flip_dist_y [sub_view_id [i]] and passes control to a function block 2140. The function block 2140 sets the variable j to 0 and passes control to a decision block 2142. Decision block 2142 determines if the current value of the variable j is less than the current value of the syntax element num_parts [sub_view_id [i]]. If so, control is passed to a function block 2144. Otherwise, control is passed to a function block 2136.

The function block 2144 parses the syntax element num_pixel_tiling_filter_coeffs_minus1 [sub_view_id [i]] and passes control to a function block 2146. The function block 2146 parses the coefficients for all pixel tiling filters and passes control to a function block 2136.

Referring to FIG. 22, examples of tiling at the pixel level are generally indicated by reference numeral 2200. 22 is further described below.

MPEG -4 AVC  or MVC View using Tiling

The application of multi-view video coding is FTV (free view point TV). This application requires the user to be able to move freely between two or more views. To achieve this, "virtual" views between the two views need to be interpolated or synthesized. There are several methods for performing view interpolation. One of these methods uses depth for view interpolation / synthesis.

Each view can have an associated depth signal. Thus, the depth can be considered another type of video signal. FIG. 9 shows an example of a depth signal 900. To enable applications such as FTV, the depth signal is transmitted with the video signal. In the proposed tiling framework, the depth signal can also be added as one of the tiles. Figure 10 shows an example of a depth signal added as a tile. The depth signals / tiles are shown on the right side of FIG.

If the depth is encoded as a tile of the entire frame, the high level syntax should indicate which tile is the depth signal so that the renderer can use the depth signal properly.

If the input sequence (e.g., shown in FIG. 1) is encoded using an MPEC-4 AVC standard encoder (or an encoder corresponding to a different video coding standard and / or recommendation), the proposed high- For example, in a sequence parameter set (SPS), a picture parameter set (PPS), a slice header, and / or a supplemental enhancement information (SEI) message. An embodiment of the proposed method is shown in Table 1 in which the syntax is present in the SEI message.

1) of the MPEC-4 AVC standard encoder (or an encoder corresponding to a multi-view video coding standard for other video coding standards and / or recommendations) In the case of being encoded using a video coding (MVC) extension, the proposed high level syntax may be in an SPS, PPS, slice header, SEI message or a specific profile. Examples of the proposed method are shown in Table 1. Table 1 shows the syntax elements present in the SPS structure including the proposed syntax elements according to embodiments of the present principles.

Figure 112009067731434-pct00009

Table 1

Table 2 shows the syntax elements for the pseudo_view_info syntax element of Table 1 according to an embodiment of the present principles.

Figure 112009067731434-pct00010

Table 2

Table 1 and Table 2 Syntax  Semantics of elements

If pseudo_view_present_flag is true, some views are superviews of multiple subviews.

If tiling_mode is 0, it indicates that the sub-views are tiled at the picture level. A value of 1 indicates that tiling is done at the pixel level.

The new SEI message may use the values for the SEI payload type not used in the extension of the MPEC-4 AVC standard or the MPEC-4 AVC standard. The new SEI message contains some syntax elements with the following semantics.

num_coded_views_minus1 + 1 represents the number of coded views supported by the bitstream. The value of num_coded_views_minus1 is in the range of 0 to 1023.

org_pic_width_in_mbs_minus1 + 1 specifies the width of the picture in each view as units of macroblocks.

The parameters for the picture width as units of macroblocks are derived as follows:

PicWidthInMbs = org_pic_width_in_mbs_minus1 + 1

The parameters of the image width for the luma component are derived as follows:

PicWidthInSamplesL = PicWidthInMbs * 16

The parameters of the image width for the chroma components are derived as follows:

PicWidthInSamplesC = PicWidthInMbs * MbWidthC

org_pic_height_in_mbs_minus1 + 1 specifies the height of the image in each view as a unit of blocks.

The parameters for the image height as units of macroblocks are derived as follows:

PicHeightInMbs = org_pic_height_in_mbs_minus1 + 1

The variable of the image height for the luma component is derived as follows:

PicHeightInSamplesL = PicHeightInMbs * 16

The parameters of the image height for the chroma components are derived as follows:

PicHeightInSamplesC = PicHeightInMbs * MbHeightC

num_sub_views_minus1 + 1 represents the number of coded sub-views included in the current view. The value of num_coded_views_minus1 is in the range of 0 to 1023.

sub_view_id [i] specifies the sub_view_id of the sub-view in the decoding order denoted by i.

num_parts [sub_view_id [i]] specifies the number of parts in which the image of sub_view_id [i] is divided.

loc_left_offset [sub_view_id [i]] [j] and loc_top_offset [sub_view_id [i]] [j] respectively designate the positions of the offsets of the left and upper pixels, where the current portion j has the same sub_view_id as sub_view_id [i] Lt; RTI ID = 0.0 > reconstruction < / RTI >

view_id [i] is the view_id of the view with the coding order denoted by i.

frame_crop_left_offset [view_id [i]] [j], frame_crop_right_offset [view_id [i]] [j], frame_crop_top_offset [view_id [i]] [j], and frame_crop_bottom_offset [view_id [i]] [j] Specifies samples of pictures in the coded video sequence that are part of num_part (j) and view_id (i) into the designated rectangular area.

The variables CropUnitX and CropUnitY are derived as follows:

- If chroma_format_idc is 0, then CropUnitX and CropUnitY are derived as follows:

CropUnitX = 1

CropUnitY = 2 - frame_mbs_only_flag

- Otherwise (chroma_format_idc is 1, 2 or 3), CropUnitX and CropUnitY are derived as follows:

CropUnitX = SubWidthC

CropUnitY = SubHeightC * (2 - frame_mbs_only_flag)

A frame cropping rectangle is a luma sample with vertical frame coordinates from CropUnitX * frame_crop_left_offset to PicWidthInSamplesL- (CropUnitX * frame_crop_right_offset + 1) and vertical frame coordinates from CropUnitY * frame_crop_top_offset to (16 * FrameHeightInMbs) - (CropUnitY * frame_crop_bottom_offset + 1) . the value of frame_crop_left_offset is in the range of 0 to (PicWidthInSamplesL / CropUnitX) - (frame_crop_right_offset + 1), and the value of frame_crop_top_offset is in the range of 0 to (16 * FrameHeightInMbs / CropUnitY) - (frame_crop_bottom_offset + 1).

If chroma_format_idc is not zero, the corresponding designated samples of the two chroma arrays are samples with frame coordinates (x / SubWidthC, y / SubHeightC), where (x, y) are frame coordinates of the designated luma samples.

For decoded fields, the specified samples of the decoded field are the samples in the rectangle specified in the frame coordinates.

  num_parts [view_id [i]] specifies the number of parts in which the image of view_id [i] is divided.

depth_flag [view_id [i]] specifies whether the current portion is a depth signal. If depth_flag is 0, the current portion is not a depth signal. If depth_flag is 1, then the current portion is the depth signal associated with the view identified by view_id [i].

flip_dir [sub_view_id [i]] [j] specifies the flipping direction for the current portion. If flip_dir is 0, it means that there is no flipping. If flip_dir is 1, fliping in the horizontal direction is indicated. If flip_dir is 2, flipping in the vertical direction is indicated.

flip_dir [view_id [i]] [j] specifies the flipping direction for the current portion. If flip_dir is 0, it means that there is no flipping. If flip_dir is 1, fliping in the horizontal direction is indicated. If flip_dir is 2, flipping in the vertical direction is indicated.

loc_left_offset [view_id [i]] [j], loc_top_offset [view_id [i]] [j] specifies the location of the pixel offsets, where the current portion j is the end of the view with view_id [ Is located in the reconstructed image.

upsample_view_flag [view_id [i]] indicates whether the image belonging to the view specified by view_id [i] needs to be upsampled. If upsample_view_flag [view_id [i]] is 0, it specifies that an image with view_id equal to view_id [i] will not be upsampled. If upsample_view_flag [view_id [i]] is 1, it specifies that an image with view_id equal to view_id [i] is to be upsampled.

 upsample_filter [view_id [i]] returns the type of filter used for upsampling. tab SVC filter should be used if upsample_filter [view_id [i]] is 0 and upsample_filter [view_id [i]] is 1 if upsample_filter [ ] Indicates that a dual linear filter should be used, and upsample_filter [view_id [i]] is 3, indicating that custom filter coefficients are to be transmitted. If upsample_filter [view_id [i]] does not exist, it is set to zero. In this embodiment, we use a 2D custom filter. This can be easily extended to 1D filters, any other nonlinear filter.

vert_dim [view_id [i]] specifies the vertical dimension of the custom 2D filter.

hor_dim [view_id [i]] specifies the horizontal dimension of the custom 2D filter.

The quantizer [view_id [i]] specifies quantities and factors for each filter coefficient.

filter_coeffs [view_id [i]] [yuv] [y] [x] specifies the quantized filter coefficients. yuv signals the component to which the filter coefficients are to be applied. If yuv is 0, Y component is specified. If yuv is 1, U component is specified. If yuv is 2, V component is assigned.

pixel_dist_x [sub_view_id [i]] and pixel_dist_y [sub_view_id [i]] specify the distance between the horizontal and vertical directions of the last reconstruction pseudo-view between adjacent pixels of the view with a sub_view_id such as sub_view_id [i], respectively.

num_pixel_tiling_filter_coeffs_minus1 [sub_view_id [i] [j]] + 1 represents the number of filter coefficients when the tiling board is set to 1.

pixel_tiling_filter_coeffs [sub_view_id [i] [j]] signals the filter coefficients needed to represent a filter that can be used to filter the tiled image.

At the pixel level Tiling  Examples

22, two examples illustrating the synthesis of a pseudo-view by tiling pixels from four views are shown generally by reference numerals 2210 and 2220, respectively. The four views are collectively indicated by reference numeral 2250. The syntax values for the first example of FIG. 22 are provided in Table 3 below.

Figure 112009067731434-pct00011

Table 3

The syntax values for the second example of FIG. 22 are all the same except for the two syntax elements: loc_left_offset [3] [0] = 5 and loc_top_offset [3] [0] = 3.

The offset indicates that the pixels corresponding to the view should start at a particular offset position. This is shown in Figure 22 (2220). This can be done, for example, when two views create images in which a common object is seen to be moved from one view to another. For example, if the first and second cameras (representing the first and second views) have taken an image of the object, the object may appear to be shifted by 5 pixels to the right in the second view relative to the first view have. This means that the pixel (i-5, j) of the first view corresponds to the pixel (i, j) of the second view. If the pixels of the two views are simply tiled on a pixel by pixel basis, there may not be much correlation between adjacent pixels of the tile and the spatial coding gains may be small. Conversely, by moving the tiling so that the pixel (i-5, j) from one view is located next to the pixel (i, j) from the second view, the spatial correlation can be increased and the spatial coding gain is also increased can do. This is because, for example, the corresponding pixels for the objects of the first and second views are tiled next to each other.

Thus, the presence of loc_left_offset and loc_top_offset may be advantageous for coding efficiency. The offset information can be obtained by an external means. For example, location information of cameras or global disparity vectors between views may be used to determine such offset information.

As a result of the offset, some pixels of the pseudo-view are not assigned pixel values from any view. Continuing with the example above, for values of i = 0 ... 4, if the tiling pixel (i-5, j) from one view is side by side with the pixel (i, j) from the second view, There is no pixel (i-5, j) from one view, so these pixels are empty in the tile. For these pixels of the pseudo-view (tile) where pixel values are not assigned from any view, at least one implementation uses an interpolation procedure similar to the sub-pixel interpolation procedure of motion compensation of the AVC. That is, the empty tile pixel can be interpolated from adjacent pixels. This interpolation can result in greater spatial correlation in the tile and greater coding gain for the tile.

In video coding, we choose different coding types for each picture, such as I, P, and B pictures. In multi-view video coding, we also define anchor and non-anchor images. In one embodiment, we propose that the determination of grouping is made based on the picture type. This grouping information is signaled in a high-level syntax.

Referring to FIG. 11, an example of five views that are tiled on a single frame is generally indicated by reference numeral 1100. In particular, a ballroom sequence is shown with five views tiled on a single frame. Additionally, it can be seen that the fifth view can be divided into two parts and arranged on a square frame. Here, each view is QVGA in size, and thus the total frame dimension is 640 * 600. Since 600 is not a multiple of 16, it should be extended to 608.

In this example, a possible SEI message may be as shown in Table 4. < tb > < TABLE >

Figure 112009067731434-pct00012

Figure 112009067731434-pct00013

Table 4

Table 5 shows a general syntax structure for transmitting the multi-view information for the example shown in Table 4. < tb > < TABLE >

Figure 112009067731434-pct00014

Table 5

23, a video processing device 2300 is shown. Video processing device 2300 may be, for example, a set-top box or other device that receives encoded video and provides decoded video to a user, for example, for storage or for display. Thus, device 2300 may provide its output to a television, computer monitor or computer or other processing device.

The device 2300 includes a decoder 2310 that receives a data signal 2320. The data signal 2320 may include, for example, an AVC or MVC compatible stream. Decoder 2310 decodes some or all of received signal 2320 and provides decoded video signal 2330 and tiling information 2340 as an output. The decoded video 2330 and tiling information 2340 are provided to a selector 2350. Device 2300 also includes a user interface 2360 that receives user input 2370. User interface 2360 provides image selection signal 2380 to selector 2350 based on user input 2370. Image selection signal 2380 and user input 2370 indicate which of multiple pictures the user wants to display. Selector 2350 provides the selected image (s) as output 2390. [ The selector 2350 uses the image selection information 2380 to select any of the images of the decoded video 2330 to provide as output 2390. [ The selector 2350 uses the tiling information 2340 to find the selected image (s) of the decoded video 2330.

In various implementations, the selector 2350 includes a user interface 2360, and in other implementations, no user interface 2360 is required because the selector 2350 does not perform a separate interface function And directly receives user input 2370. The selector 2350 may be implemented in software or as an integrated circuit, for example. The selector 2350 may also include a decoder 2310.

More generally, decoders of various implementations described in this application may provide a decoded output that includes the entirety of the tiles. Additionally or alternatively, the decoders may provide a decoded output that includes only one or more selected pictures (e.g., images or depth signals) from the tile.

As noted above, high level syntax can be used to perform signaling in accordance with one or more embodiments of the principles of the present invention. The high level syntax may include, for example, the number of coded views present in a large frame; The original width and height of all views; For each coded view, a view identifier corresponding to the view; For each coded view, the number of parts where the frame of the view is divided; For each portion of the view, the flip direction (e.g., can be no flip, only horizontal flip, only vertical flip, or horizontal and vertical flip); For each part of the view, the left position of the number of macroblocks or pixels to which the current part belongs in the last frame to the view; For each portion of the view, the upper position of the portion of the plurality of macroblocks or pixels to which the current portion belongs in the last frame to the view; The left position in the current large decoded / encoded frame of the cropping window of multiple macroblocks or pixels; For each part of the view, the right position in the current large decoded / encoded frame of the cropping window of multiple macroblocks or pixels; For each portion of the view, the upper position in the current large decoded / encoded frame of the cropping window of multiple macroblocks or pixels; And a bottom position in the current large decoded / encoded frame of the cropping window of the plurality of macroblocks or pixels; For each coded view, whether or not the view needs to be upsampled before output (if upsampling needs to be performed, the high level syntax can be upsampled (AVC 6-tap filter, SVC 4- (Which may be used to represent, for example, including, but not limited to, dual linear filters or custom 1D, 2D linear or non-linear filters).

It should be noted that the terms " encoder "and" decoder "refer to general structures and are not limited to any particular function or feature. For example, a decoder may receive a modulated carrier comprising an encoded bit stream, demodulate the encoded bit stream, and decode the bit stream.

Various methods have been described. Most of these methods have been described in detail to provide sufficient disclosure. However, it should be noted that transforms that can transform most of the specific features or features described for these methods may be contemplated. In addition, many of the cited features are well known in the art and, therefore, have not been described in much detail.

Also, in some implementations, references have been made to use high-level syntax to transmit specific information. Note, however, that other implementations may use low-level syntax or other mechanisms (e.g., to transmit information as part of the encoded data) to provide the same information (or a transform of this information) do.

Various implementations provide tiling and appropriate signaling to allow multiple views to be tiled into a single image, encoded as a single image, and transmitted as a single image. The signaling information may cause the post-processor to separate views / pictures. Also, the tiled multiple images may be views, and at least one of the images may be depth information. These implementations may provide one or more advantages. For example, users may want to display multiple views in a tiled manner, and these various implementations provide an efficient way to encode, transmit, or store these views by tiling prior to encoding and transmitting / storing in a tiled manner do.

Implementations that tile multiple views in the context of AVC and / or MVC also provide additional benefits. AVC is used for only a single view on the surface and no additional views are expected. However, because, for example, tiled views are arranged to recognize that tiled pictures belong to different views (e.g., the upper left picture of the pseudo-view belongs to view 1, the upper right picture belongs to view 2, etc.) These AVC-based implementations can provide multiple views in an AVC environment.

Additionally, the MVC already includes multiple views, so multiple views are not expected to be included in a single pseudo-view. In addition, MVC has a limitation on the number of views that can be supported and allows the number of views that can be supported (such as in AVC-based implementations) by allowing these MVC-based implementations to be tiled Effectively. For example, each pseudo-view may correspond to one of the supported views of the MVC, and the decoder may recognize that each "supported view" actually includes four views in a pre-arranged tiling order. Thus, in this implementation, the number of possible views is four times the number of "supported views ".

The implementations described herein may be implemented by, for example, methods or processes, devices, or software programs. Although discussed in the context of one type of implementation (e.g., discussed only as a method), implementations of the discussed features may also be implemented in other types (e.g., a device or a program). The device may be implemented, for example, with suitable hardware, software, and firmware. The methods may generally be implemented in devices such as, for example, a processor, which refers to processing devices including, for example, a computer, microprocessor, integrated circuit or programmable logic device. The processing devices also include communication devices such as, for example, computers, cell phones, portable / personal digital assistants ("PDAs"), and other devices that facilitate communication of information between end users do.

Implementations of the various processes and features described herein may be embodied in a wide variety of different devices or applications, such as, for example, equipment or applications associated with data encoding and decoding. Examples of devices include video coders, video decoders, video codecs, web servers, set-top boxes, laptops, personal computers, cell phones, PDAs and other communication devices. For clarity, the equipment can be moved and also installed on a moving vehicle.

Additionally, the methods may be implemented by instructions executed by a processor, which may be, for example, an integrated circuit, a software carrier or a hard disk, a compact disk, a random access memory (" Readable medium, such as a read-only memory ("ROM"). The instructions may form an application program that is embodied on a processor-readable medium. For clarity, a processor may include, for example, a processor-readable medium having instructions for executing the process. These application programs may be uploaded to or executed by a machine containing any suitable architecture. Preferably, the machine is implemented on a computer platform having hardware such as a central processing unit ("CPU"), random access memory ("RAM") and input / output ("I / O" The computer platform may also include an operating system and microinstruction code. The various process functions described herein may be portions of the microinstruction code, portions of the application program, or any combination thereof that may be executed by the CPU. In addition, various other peripheral units may be connected to a computer platform such as an additional data storage unit and a printing unit.

As will be apparent to those skilled in the art, implementations may also generate signals formatted to convey information that may be, for example, stored or transmitted. The information may include, for example, instructions for carrying out the method or data generated by one of the described implementations. Such a signal may be formatted, for example, as an electromagnetic wave (e.g., using a radio frequency portion of the spectrum) or as a baseband signal. The formatting may include, for example, encoding a data stream, generating a syntax and modulating the carrier into an encoded data stream and syntax. The information that the signal carries may be, for example, analog or digital information. The signal may be transmitted over a variety of different wired or wireless links as is known.

Because some of the constituent system components and methods shown in the accompanying drawings may be implemented in software, the actual connections or process functional blocks between the system components may differ depending on how these principles are programmed It should also be understood. The teachings provided herein may be considered by those skilled in the art to the above and similar implementations or configurations of these principles.

A number of implementations have been described. Nevertheless, it will be appreciated that a variety of conversions can be made. For example, elements of different implementations may be combined, supplemented, transformed, or eliminated to produce different implementations. Additionally, those skilled in the art will appreciate that other structures or processes may replace what is disclosed, and that the resulting implementations be, at least substantially, in substantially the same manner (s) to achieve at least substantially the same result (S) < / RTI > In particular, while exemplary implementations have been described herein with reference to the accompanying drawings, it is to be understood that these principles are not limited to such precise embodiments, and that various changes and modifications may be effected by one of ordinary skill in the art without departing from the scope or spirit of these principles Can be understood. Accordingly, these and other implementations are contemplated by the present disclosure and are within the scope of the following claims.

Claims (43)

  1. A method of processing a video image,
    Accessing the video picture comprising multiple pictures combined into a single picture, the multiple pictures comprising a first picture of a first view of multi-view video and a second picture of a second view of multi-view video, A common object or region in the multiple images appears to have been moved from one view to another, the video picture being part of a received video stream;
    Accessing information indicating how multiple images of the accessed video image are combined, the accessed information indicating whether each of the multiple images is individually flipped in at least one of a horizontal direction or a vertical direction The accessed information being part of the received video stream;
    Decoding the video picture to provide a decoded representation of at least one of the multiple pictures; And
    Providing at least one of the decoded representation and the accessed information as an output
    / RTI >
  2. The method according to claim 1,
    Wherein the accessed information is indicative of a combination of the multiple pictures in a single picture in a manner that provides improved spatial correlation between corresponding portions of the multiple pictures to allow coding gains in coding the single picture. Lt; / RTI >
  3. 2. The method of claim 1, further comprising flipping the decoded representation if the accessed information indicates that at least one of the multiple pictures is being flipped.
  4. delete
  5. The method according to claim 1,
    Accessing the video image, accessing the information, and decoding the video picture are performed in a decoder.
  6. delete
  7. delete
  8. delete
  9. delete
  10. delete
  11. delete
  12. delete
  13. A device configured to perform the method according to any one of claims 1 to 3 and 5.
  14. A computer-readable medium having computer-readable program code stored thereon,
    The computer readable program code is configured to perform the method according to any one of claims 1 to 3 and 5.
  15. delete
  16. delete
  17. delete
  18. delete
  19. delete
  20. delete
  21. delete
  22. delete
  23. delete
  24. delete
  25. delete
  26. delete
  27. delete
  28. delete
  29. delete
  30. delete
  31. delete
  32. delete
  33. delete
  34. delete
  35. delete
  36. delete
  37. delete
  38. delete
  39. delete
  40. delete
  41. delete
  42. delete
  43. delete
KR1020097023043A 2007-04-12 2008-04-11 Tiling in video encoding and decoding KR101467601B1 (en)

Priority Applications (4)

Application Number Priority Date Filing Date Title
US92301407P true 2007-04-12 2007-04-12
US60/923,014 2007-04-12
US92540007P true 2007-04-20 2007-04-20
US60/925,400 2007-04-20

Publications (2)

Publication Number Publication Date
KR20100016212A KR20100016212A (en) 2010-02-12
KR101467601B1 true KR101467601B1 (en) 2014-12-02

Family

ID=39810226

Family Applications (8)

Application Number Title Priority Date Filing Date
KR1020097023043A KR101467601B1 (en) 2007-04-12 2008-04-11 Tiling in video encoding and decoding
KR1020197009228A KR102044130B1 (en) 2007-04-12 2008-04-11 Tiling in video encoding and decoding
KR1020147017631A KR101646089B1 (en) 2007-04-12 2008-04-11 Tiling in video encoding and decoding
KR20157009602A KR20150046385A (en) 2007-04-12 2008-04-11 Tiling in video encoding and decoding
KR1020187022161A KR101965781B1 (en) 2007-04-12 2008-04-11 Tiling in video encoding and decoding
KR1020197032816A KR20190127999A (en) 2007-04-12 2008-04-11 Tiling in video encoding and decoding
KR1020177021734A KR101885790B1 (en) 2007-04-12 2008-04-11 Tiling in video encoding and decoding
KR1020167028058A KR101766479B1 (en) 2007-04-12 2008-04-11 Tiling in video encoding and decoding

Family Applications After (7)

Application Number Title Priority Date Filing Date
KR1020197009228A KR102044130B1 (en) 2007-04-12 2008-04-11 Tiling in video encoding and decoding
KR1020147017631A KR101646089B1 (en) 2007-04-12 2008-04-11 Tiling in video encoding and decoding
KR20157009602A KR20150046385A (en) 2007-04-12 2008-04-11 Tiling in video encoding and decoding
KR1020187022161A KR101965781B1 (en) 2007-04-12 2008-04-11 Tiling in video encoding and decoding
KR1020197032816A KR20190127999A (en) 2007-04-12 2008-04-11 Tiling in video encoding and decoding
KR1020177021734A KR101885790B1 (en) 2007-04-12 2008-04-11 Tiling in video encoding and decoding
KR1020167028058A KR101766479B1 (en) 2007-04-12 2008-04-11 Tiling in video encoding and decoding

Country Status (19)

Country Link
US (13) US8780998B2 (en)
EP (5) EP2887671B1 (en)
JP (6) JP5324563B2 (en)
KR (8) KR101467601B1 (en)
CN (1) CN101658037B (en)
AU (1) AU2008239653C1 (en)
BR (5) BRPI0823512A2 (en)
DK (2) DK2512136T3 (en)
ES (2) ES2675164T3 (en)
HU (2) HUE029776T2 (en)
LT (1) LT2887671T (en)
MX (1) MX2009010973A (en)
PL (2) PL2887671T3 (en)
PT (2) PT2512136T (en)
RU (3) RU2521618C2 (en)
SI (1) SI2887671T1 (en)
TR (1) TR201809177T4 (en)
WO (1) WO2008127676A2 (en)
ZA (2) ZA201006649B (en)

Families Citing this family (51)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8692695B2 (en) 2000-10-03 2014-04-08 Realtime Data, Llc Methods for encoding and decoding data
BRPI0823512A2 (en) 2007-04-12 2013-11-26 Thomson Licensing Video encoding and decoding
CN101682785B (en) 2007-05-16 2017-03-08 汤姆森特许公司 Using the method and apparatus of chip set in encoding multi-view video coding information
CN105721877A (en) 2008-09-23 2016-06-29 杜比实验室特许公司 Method For Decoding Interleaved Image Data And Device Comprising Decoder
EP2338281A4 (en) 2008-10-17 2012-08-15 Nokia Corp Sharing of motion vector in 3d video coding
RU2543954C2 (en) * 2009-01-26 2015-03-10 Томсон Лайсенсинг Frame packing for video coding
CN105376549B (en) 2009-01-29 2017-08-11 杜比实验室特许公司 The method of method for video coding and decoding video signal
CA2749583C (en) * 2009-02-01 2016-01-12 Lg Electronics Inc. Broadcast receiver and 3d video data processing method
US9066075B2 (en) * 2009-02-13 2015-06-23 Thomson Licensing Depth map coding to reduce rendered distortion
EP2422522A1 (en) 2009-04-20 2012-02-29 Dolby Laboratories Licensing Corporation Directed interpolation and data post-processing
EP2425626A2 (en) * 2009-05-01 2012-03-07 Thomson Licensing Inter-layer dependency information for 3dv
US9124874B2 (en) 2009-06-05 2015-09-01 Qualcomm Incorporated Encoding of three-dimensional conversion information with two-dimensional video sequence
WO2011005624A1 (en) 2009-07-04 2011-01-13 Dolby Laboratories Licensing Corporation Encoding and decoding architectures for format compatible 3d video delivery
JP2011109397A (en) * 2009-11-17 2011-06-02 Sony Corp Image transmission method, image reception method, image transmission device, image reception device, and image transmission system
BR112012018976A2 (en) 2010-01-29 2018-03-27 Thomson Licensing block-based collation
US8555324B2 (en) 2010-02-02 2013-10-08 Microsoft Corporation Video download mechanism for transferring large data
IT1399417B1 (en) * 2010-04-12 2013-04-16 Sisvel Technology Srl A method for generating and rebuilding a compatible stereoscopic video stream and associated encoding and decoding devices.
US9225961B2 (en) * 2010-05-13 2015-12-29 Qualcomm Incorporated Frame packing for asymmetric stereo video
IT1401731B1 (en) 2010-06-28 2013-08-02 Sisvel Technology Srl Method for 2D-compatible decoding of stereoscopic video streams
US9596447B2 (en) 2010-07-21 2017-03-14 Qualcomm Incorporated Providing frame packing type information for video coding
US8619852B2 (en) 2010-07-21 2013-12-31 Dolby Laboratories Licensing Corporation Systems and methods for multi-layered frame-compatible video delivery
EP2602999A1 (en) * 2010-08-06 2013-06-12 Panasonic Corporation Encoding method, display device, and decoding method
JP5889899B2 (en) * 2010-08-09 2016-03-22 コーニンクレッカ フィリップス エヌ ヴェKoninklijke Philips N.V. Method of encoding a pair of images corresponding to two fields of view of a multi-field signal, method of decoding, encoder, decoder, computer program and software tool
CN103299619A (en) 2010-09-14 2013-09-11 汤姆逊许可公司 Compression methods and apparatus for occlusion data
WO2012057564A2 (en) * 2010-10-28 2012-05-03 엘지전자 주식회사 Receiver apparatus and method for receiving a three-dimensional broadcast signal in a mobile environment
JP2012114575A (en) * 2010-11-22 2012-06-14 Sony Corp Image data transmission device, image data transmission method, image data reception device, and image data reception method
US9215473B2 (en) 2011-01-26 2015-12-15 Qualcomm Incorporated Sub-slices in video coding
CN102972024A (en) * 2011-05-11 2013-03-13 松下电器产业株式会社 Video transmission device and video transmission method
US9521418B2 (en) * 2011-07-22 2016-12-13 Qualcomm Incorporated Slice header three-dimensional video extension for slice header prediction
US9077998B2 (en) * 2011-11-04 2015-07-07 Qualcomm Incorporated Padding of segments in coded slice NAL units
US9124895B2 (en) 2011-11-04 2015-09-01 Qualcomm Incorporated Video coding with network abstraction layer units that include multiple encoded picture partitions
US9485503B2 (en) 2011-11-18 2016-11-01 Qualcomm Incorporated Inside view motion prediction among texture and depth view components
US9749661B2 (en) 2012-01-18 2017-08-29 Qualcomm Incorporated Sub-streams for wavefront parallel processing in video coding
TWI594616B (en) 2012-06-14 2017-08-01 杜比實驗室特許公司 Depth map delivery formats for stereoscopic and auto-stereoscopic displays
US9584803B2 (en) * 2012-07-08 2017-02-28 Cisco Technology, Inc. Picture output management in video applications with fixed picture rate
US9426462B2 (en) 2012-09-21 2016-08-23 Qualcomm Incorporated Indication and activation of parameter sets for video coding
US9979960B2 (en) 2012-10-01 2018-05-22 Microsoft Technology Licensing, Llc Frame packing and unpacking between frames of chroma sampling formats with different chroma resolutions
US9661340B2 (en) 2012-10-22 2017-05-23 Microsoft Technology Licensing, Llc Band separation filtering / inverse filtering for frame packing / unpacking higher resolution chroma sampling formats
KR101861979B1 (en) * 2013-01-18 2018-05-28 캐논 가부시끼가이샤 Generation method and device of media file comprising media data region and metadata region
TWI558166B (en) 2013-04-04 2016-11-11 杜比國際公司 Depth map delivery formats for multi-view auto-stereoscopic displays
US9762868B2 (en) * 2013-06-28 2017-09-12 Thomson Licensing Highlighting an object displayed by a pico projector
RU2652099C2 (en) * 2013-07-05 2018-04-25 Сони Корпорейшн Transmission device, transmission method, reception device and reception method
US9749646B2 (en) 2015-01-16 2017-08-29 Microsoft Technology Licensing, Llc Encoding/decoding of high chroma resolution details
US9854201B2 (en) 2015-01-16 2017-12-26 Microsoft Technology Licensing, Llc Dynamically updating quality to higher chroma sampling rate
CN104869457B (en) * 2015-05-21 2019-04-26 京东方科技集团股份有限公司 Display, display device, player, playing device and broadcasting display system
US10170154B2 (en) 2015-11-16 2019-01-01 Virtual Dreamware, LLC Symmetrically mirrored video segment
CN109155854A (en) * 2016-05-27 2019-01-04 松下电器(美国)知识产权公司 Code device, decoding apparatus, coding method and coding/decoding method
JP6185635B2 (en) * 2016-09-02 2017-08-23 株式会社ユニバーサルエンターテインメント Game machine
US10334277B2 (en) * 2016-09-02 2019-06-25 Qualcomm Incorporated Signaling target display parameters of color remapping information supplemental enhancement information messages
US10368080B2 (en) 2016-10-21 2019-07-30 Microsoft Technology Licensing, Llc Selective upsampling or refresh of chroma sample values
US10021976B1 (en) * 2016-11-01 2018-07-17 Scott Allen Furniture system

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5915091A (en) * 1993-10-01 1999-06-22 Collaboration Properties, Inc. Synchronization in video conferencing
EP1581003A1 (en) * 2004-03-23 2005-09-28 British Broadcasting Corporation Monitoring system

Family Cites Families (84)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5193000A (en) * 1991-08-28 1993-03-09 Stereographics Corporation Multiplexing technique for stereoscopic video system
JPH0870475A (en) * 1994-06-23 1996-03-12 Sanyo Electric Co Ltd Method and device for encoding and decoding stereoscopic animation
US5748786A (en) 1994-09-21 1998-05-05 Ricoh Company, Ltd. Apparatus for compression using reversible embedded wavelets
US6055012A (en) 1995-12-29 2000-04-25 Lucent Technologies Inc. Digital multi-view video compression with complexity and compatibility constraints
WO1997043883A1 (en) 1996-05-10 1997-11-20 Melcher Ag Device for mounting electrical components on printed-circuit boards
DE19619598A1 (en) * 1996-05-15 1997-11-20 Deutsche Telekom Ag A process for the storage or transmission of stereoscopic video signals
US6075905A (en) 1996-07-17 2000-06-13 Sarnoff Corporation Method and apparatus for mosaic image construction
JP3263807B2 (en) * 1996-09-09 2002-03-11 ソニー株式会社 The image coding apparatus and image coding method
US6173087B1 (en) 1996-11-13 2001-01-09 Sarnoff Corporation Multi-view image registration with application to mosaicing and lens distortion correction
AU5720698A (en) 1996-12-27 1998-07-31 Chequemate International Inc. System and method for synthesizing three-dimensional video from a two-dimensio nal video source
CN1223772A (en) 1997-06-09 1999-07-21 松下电器产业株式会社 Method of encoding image important, its encoder, its decoding/synthesizing method, its decoder/synthesizer and recording medium on which those methods as recorded
US6390980B1 (en) 1998-12-07 2002-05-21 Atl Ultrasound, Inc. Spatial compounding with ultrasonic doppler signal information
US6223183B1 (en) 1999-01-29 2001-04-24 International Business Machines Corporation System and method for describing views in space, time, frequency, and resolution
US6157396A (en) 1999-02-16 2000-12-05 Pixonics Llc System and method for using bitstream information to process images for use in digital display systems
US7254265B2 (en) 2000-04-01 2007-08-07 Newsight Corporation Methods and systems for 2D/3D image conversion and optimization
DE10016074B4 (en) 2000-04-01 2004-09-30 Tdv Technologies Corp. Method and device for generating 3D images
GB0013273D0 (en) 2000-06-01 2000-07-26 Philips Electronics Nv Video signal encoding and buffer management
KR100535147B1 (en) * 2000-09-18 2005-12-07 인터내셔널 비지네스 머신즈 코포레이션 Method, system and computer program for managing views at a computer display
US20030004835A1 (en) * 2001-05-31 2003-01-02 International Business Machines Corporation Method, system, and product service for locating items satisfying a general requirement and ranking items according to utility
US7277121B2 (en) * 2001-08-29 2007-10-02 Sanyo Electric Co., Ltd. Stereoscopic image processing and display system
EP1451775A1 (en) 2001-11-24 2004-09-01 TDV Technologies Corp. Generation of a stereo image sequence from a 2d image sequence
US7263240B2 (en) 2002-01-14 2007-08-28 Eastman Kodak Company Method, system, and software for improving signal quality using pyramidal decomposition
CA2380105A1 (en) * 2002-04-09 2003-10-09 Nicholas Routhier Process and system for encoding and playback of stereoscopic video sequences
JP4134027B2 (en) * 2002-04-25 2008-08-13 シャープ株式会社 Image encoding device, image decoding device, and image recording device
JP4154569B2 (en) 2002-07-10 2008-09-24 日本電気株式会社 Image compression / decompression device
JP2004120169A (en) * 2002-09-25 2004-04-15 Sharp Corp Electronic apparatus
JP4251864B2 (en) * 2002-12-13 2009-04-08 シャープ株式会社 Image data creating apparatus and image data reproducing apparatus for reproducing the data
US20040260827A1 (en) 2003-06-19 2004-12-23 Nokia Corporation Stream switching based on gradual decoder refresh
US7496234B2 (en) 2003-06-20 2009-02-24 Microsoft Corporation System and method for seamless multiplexing of embedded bitstreams
CN1571508B (en) 2003-07-19 2010-05-12 华为技术有限公司 A method for implementing multi-frame
JP2005049665A (en) 2003-07-30 2005-02-24 Nec Plasma Display Corp Video signal processing circuit, display device, and image signal processing method therefor
MX2007000369A (en) 2004-07-29 2007-03-12 Microsoft Corp Image processing using linear light values and other image processing improvements.
JP4355914B2 (en) * 2003-10-01 2009-11-04 日本電気株式会社 Multi-view image transmission system and method, multi-view image compression device and method, multi-view image decompression device and method, and program
KR100519776B1 (en) 2003-11-24 2005-10-07 삼성전자주식회사 Method and apparatus for converting resolution of video signal
KR100587952B1 (en) 2003-12-05 2006-06-08 한국전자통신연구원 Video encoding/decoding apparatus and method using compensation for the asymmetric decimated left/right-view images
US8139142B2 (en) * 2006-06-01 2012-03-20 Microsoft Corporation Video manipulation of red, green, blue, distance (RGB-Z) data including segmentation, up-sampling, and background substitution techniques
US20050259729A1 (en) * 2004-05-21 2005-11-24 Shijun Sun Video coding with quality scalability
KR100679740B1 (en) 2004-06-25 2007-02-07 학교법인연세대학교 Method for Coding/Decoding for Multiview Sequence where View Selection is Possible
US20060007200A1 (en) * 2004-07-08 2006-01-12 David Young Method and system for displaying a sequence of image frames
WO2006041261A1 (en) 2004-10-16 2006-04-20 Electronics And Telecommunications Research Institute Method and system for encoding/decoding multi-view video based on layered-depth image
EP1667448A1 (en) 2004-12-02 2006-06-07 Deutsche Thomson-Brandt Gmbh Method and apparatus for encoding and for decoding a main video signal and one or more auxiliary video signals
US7903737B2 (en) 2005-11-30 2011-03-08 Mitsubishi Electric Research Laboratories, Inc. Method and system for randomly accessing multiview videos with known prediction dependency
US7489342B2 (en) * 2004-12-17 2009-02-10 Mitsubishi Electric Research Laboratories, Inc. Method and system for managing reference pictures in multiview videos
US7728878B2 (en) * 2004-12-17 2010-06-01 Mitsubishi Electric Research Labortories, Inc. Method and system for processing multiview videos for view synthesis using side information
JP2006191357A (en) * 2005-01-06 2006-07-20 Victor Co Of Japan Ltd Reproduction device and reproduction program
US20060176318A1 (en) 2005-02-09 2006-08-10 Martin Virginia L Method for manipulating artwork to form decorative pieces
JP4542447B2 (en) 2005-02-18 2010-09-15 株式会社日立製作所 Image encoding / decoding device, encoding / decoding program, and encoding / decoding method
US8228994B2 (en) 2005-05-20 2012-07-24 Microsoft Corporation Multi-view video coding based on temporal and view decomposition
KR100716999B1 (en) 2005-06-03 2007-05-10 삼성전자주식회사 Method for intra prediction using the symmetry of video, method and apparatus for encoding and decoding video using the same
ES2557476T3 (en) * 2005-06-23 2016-01-26 Koninklijke Philips N.V. Combined exchange of image data and associated depth data
JP4637672B2 (en) * 2005-07-22 2011-02-23 株式会社リコー Encoding processing apparatus and method
US7668366B2 (en) 2005-08-09 2010-02-23 Seiko Epson Corporation Mosaic image data processing
US20100158133A1 (en) 2005-10-12 2010-06-24 Peng Yin Method and Apparatus for Using High-Level Syntax in Scalable Video Encoding and Decoding
CN101292538B (en) 2005-10-19 2012-11-28 汤姆森特许公司 Multi-view video coding using scalable video coding
US8457219B2 (en) 2005-12-30 2013-06-04 Ikanos Communications, Inc. Self-protection against non-stationary disturbances
ZA200805337B (en) 2006-01-09 2009-11-25 Thomson Licensing Method and apparatus for providing reduced resolution update mode for multiview video coding
US20070177674A1 (en) 2006-01-12 2007-08-02 Lg Electronics Inc. Processing multiview video
US20070205367A1 (en) 2006-03-01 2007-09-06 General Electric Company Apparatus and method for hybrid computed tomography imaging
KR101245251B1 (en) 2006-03-09 2013-03-19 삼성전자주식회사 Method and apparatus for encoding and decoding multi-view video to provide uniform video quality
JP2008034892A (en) 2006-03-28 2008-02-14 Victor Co Of Japan Ltd Multi-viewpoint image encoder
EP1999966A2 (en) 2006-03-29 2008-12-10 Thomson Licensing Multi-view video coding method and device
US7609906B2 (en) 2006-04-04 2009-10-27 Mitsubishi Electric Research Laboratories, Inc. Method and system for acquiring and displaying 3D light fields
WO2008024345A1 (en) 2006-08-24 2008-02-28 Thomson Licensing Adaptive region-based flipping video coding
EP2082280A2 (en) 2006-11-07 2009-07-29 Philips Electronics N.V. Method and device for speckle reduction
PT2103136T (en) 2006-12-21 2017-12-12 Thomson Licensing Methods and apparatus for improved signaling using high level syntax for multi-view video coding and decoding
US8515194B2 (en) 2007-02-21 2013-08-20 Microsoft Corporation Signaling and uses of windowing information for images
BRPI0823512A2 (en) 2007-04-12 2013-11-26 Thomson Licensing Video encoding and decoding
WO2008140190A1 (en) 2007-05-14 2008-11-20 Samsung Electronics Co, . Ltd. Method and apparatus for encoding and decoding multi-view image
JP4418827B2 (en) 2007-05-16 2010-02-24 三菱電機株式会社 Image display apparatus and method, and image generation apparatus and method
KR100962696B1 (en) 2007-06-07 2010-06-11 경희대학교 산학협력단 Format for encoded stereoscopic image data file
US8373744B2 (en) 2007-06-07 2013-02-12 Reald Inc. Stereoplexing for video and film applications
CN101803394A (en) 2007-06-19 2010-08-11 韩国电子通信研究院 Metadata structure for storing and playing stereoscopic data, and method for storing stereoscopic content file using this metadata
US8471893B2 (en) 2007-06-26 2013-06-25 Samsung Electronics Co., Ltd. Method and apparatus for generating stereoscopic image bitstream using block interleaved method
MY162861A (en) 2007-09-24 2017-07-31 Koninl Philips Electronics Nv Method and system for encoding a video data signal, encoded video data signal, method and system for decoding a video data signal
US8218855B2 (en) 2007-10-04 2012-07-10 Samsung Electronics Co., Ltd. Method and apparatus for receiving multiview camera parameters for stereoscopic image, and method and apparatus for transmitting multiview camera parameters for stereoscopic image
KR100918862B1 (en) 2007-10-19 2009-09-28 광주과학기술원 Method and device for generating depth image using reference image, and method for encoding or decoding the said depth image, and encoder or decoder for the same, and the recording media storing the image generating the said method
EP3007440A1 (en) 2007-12-20 2016-04-13 Koninklijke Philips N.V. Image encoding method for stereoscopic rendering
KR101506217B1 (en) 2008-01-31 2015-03-26 삼성전자주식회사 Method and appratus for generating stereoscopic image data stream for temporally partial three dimensional data, and method and apparatus for displaying temporally partial three dimensional data of stereoscopic image
US20090219985A1 (en) 2008-02-28 2009-09-03 Vasanth Swaminathan Systems and Methods for Processing Multiple Projections of Video Data in a Single Video File
US8878836B2 (en) 2008-02-29 2014-11-04 Samsung Electronics Co., Ltd. Method and apparatus for encoding datastream including additional information on multiview image and method and apparatus for decoding datastream by using the same
KR101506219B1 (en) 2008-03-25 2015-03-27 삼성전자주식회사 Method and apparatus for providing and reproducing 3 dimensional video content, and computer readable medium thereof
CN102100070A (en) 2008-07-20 2011-06-15 杜比实验室特许公司 Encoder optimization of stereoscopic video delivery systems
US8106924B2 (en) 2008-07-31 2012-01-31 Stmicroelectronics S.R.L. Method and system for video rendering, computer program product therefor
EP2197217A1 (en) 2008-12-15 2010-06-16 Philips Electronics N.V. Image based 3D video format

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5915091A (en) * 1993-10-01 1999-06-22 Collaboration Properties, Inc. Synchronization in video conferencing
EP1581003A1 (en) * 2004-03-23 2005-09-28 British Broadcasting Corporation Monitoring system

Also Published As

Publication number Publication date
US10129557B2 (en) 2018-11-13
BRPI0823512A2 (en) 2013-11-26
RU2684184C1 (en) 2019-04-04
US20180213246A1 (en) 2018-07-26
US20170257638A1 (en) 2017-09-07
US10298948B2 (en) 2019-05-21
WO2008127676A3 (en) 2008-12-18
PT2887671T (en) 2018-07-09
BRPI0809510A2 (en) 2012-12-25
KR101885790B1 (en) 2018-08-06
US20150341665A1 (en) 2015-11-26
KR101766479B1 (en) 2017-10-23
AU2008239653C1 (en) 2015-05-07
ZA201201942B (en) 2012-11-28
US20100046635A1 (en) 2010-02-25
DK2512136T3 (en) 2016-08-22
HUE029776T2 (en) 2017-04-28
US20190253727A1 (en) 2019-08-15
ZA201006649B (en) 2012-09-26
WO2008127676A2 (en) 2008-10-23
KR20140098825A (en) 2014-08-08
US9706217B2 (en) 2017-07-11
US20140301479A1 (en) 2014-10-09
KR20190127999A (en) 2019-11-13
CN101658037B (en) 2013-07-10
US9445116B2 (en) 2016-09-13
KR20170106987A (en) 2017-09-22
US20180152719A1 (en) 2018-05-31
US9185384B2 (en) 2015-11-10
BR122018004903B1 (en) 2019-10-29
DK2887671T3 (en) 2018-07-23
JP2010524398A (en) 2010-07-15
MX2009010973A (en) 2009-10-30
US20190379897A1 (en) 2019-12-12
US20160360218A1 (en) 2016-12-08
US20190037230A1 (en) 2019-01-31
US10432958B2 (en) 2019-10-01
JP2019201435A (en) 2019-11-21
KR102044130B1 (en) 2019-11-12
AU2008239653B2 (en) 2012-10-04
JP5674873B2 (en) 2015-02-25
EP2137975A2 (en) 2009-12-30
WO2008127676A9 (en) 2009-02-12
EP2512135B1 (en) 2015-03-18
JP2017135756A (en) 2017-08-03
JP5324563B2 (en) 2013-10-23
US9232235B2 (en) 2016-01-05
US9973771B2 (en) 2018-05-15
KR20180089560A (en) 2018-08-08
JP2019201436A (en) 2019-11-21
EP2887671A1 (en) 2015-06-24
AU2008239653A1 (en) 2008-10-23
PL2512136T3 (en) 2016-11-30
BR122018004904B1 (en) 2019-11-05
RU2651227C2 (en) 2018-04-18
US9219923B2 (en) 2015-12-22
PL2887671T3 (en) 2018-09-28
US20150281736A1 (en) 2015-10-01
EP2512136A1 (en) 2012-10-17
RU2009141712A (en) 2011-05-20
EP2512135A1 (en) 2012-10-17
JP2015092715A (en) 2015-05-14
RU2521618C2 (en) 2014-07-10
BRPI0809510B1 (en) 2019-10-08
RU2014116612A (en) 2015-10-27
KR101646089B1 (en) 2016-08-05
PT2512136T (en) 2016-08-16
BR122018004906B1 (en) 2019-11-12
US20180048904A1 (en) 2018-02-15
EP2887671B1 (en) 2018-05-16
US8780998B2 (en) 2014-07-15
HUE038192T2 (en) 2018-10-29
KR20190038680A (en) 2019-04-08
JP2013258716A (en) 2013-12-26
CN101658037A (en) 2010-02-24
KR20150046385A (en) 2015-04-29
ES2586406T3 (en) 2016-10-14
US9838705B2 (en) 2017-12-05
LT2887671T (en) 2018-07-25
KR101965781B1 (en) 2019-04-05
US9986254B1 (en) 2018-05-29
KR20100016212A (en) 2010-02-12
EP3399756A1 (en) 2018-11-07
KR20160121604A (en) 2016-10-19
TR201809177T4 (en) 2018-07-23
ES2675164T3 (en) 2018-07-09
EP2512136B1 (en) 2016-06-01
SI2887671T1 (en) 2018-10-30
US20160080757A1 (en) 2016-03-17

Similar Documents

Publication Publication Date Title
US9973739B2 (en) Sharing of motion vector in 3D video coding
US9179153B2 (en) Refined depth map
KR101691572B1 (en) Inter-layer dependency information for 3dv
KR101649207B1 (en) Multiview video coding and decoding
US10038916B2 (en) Encoding and decoding architectures for format compatible 3D video delivery
JP5587552B2 (en) Multi-view video coding using scalable video coding
JP5698391B2 (en) Multi-view video plus depth content coding
US9467689B2 (en) Systems and methods for multi-layered image and video delivery using reference processing signals
Vetro et al. 3D-TV content storage and transmission
US8532410B2 (en) Multi-view video coding with disparity estimation based on depth information
CN104702960B (en) Device for video decoding
JP5866364B2 (en) Stereo video data encoding
US9998726B2 (en) Apparatus, a method and a computer program for video coding and decoding
JP2015535405A (en) Method and apparatus for video coding
US20110310982A1 (en) Video signal processing method and apparatus using depth information
US20110038418A1 (en) Code of depth signal
US9172942B2 (en) Method and apparatus for generating header information of stereoscopic image data
US20130287093A1 (en) Method and apparatus for video coding
KR20130030270A (en) Frame packing for asymmetric stereo video
Vetro et al. Overview of the stereo and multiview video coding extensions of the H. 264/MPEG-4 AVC standard
KR101972962B1 (en) 3d video formats
KR101697598B1 (en) Virtual view image synthesis method and apparatus
US10110903B2 (en) Multi-view signal codec with reusing coding parameters
RU2684184C1 (en) Mosaic arrangement at video encoding and decoding
ES2561078T3 (en) Hybrid video encoding that supports intermediate view synthesis

Legal Events

Date Code Title Description
A201 Request for examination
E902 Notification of reason for refusal
A107 Divisional application of patent
E701 Decision to grant or registration of patent right
GRNT Written decision to grant
FPAY Annual fee payment

Payment date: 20171109

Year of fee payment: 4

FPAY Annual fee payment

Payment date: 20181108

Year of fee payment: 5