Classifications

• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
  • H04N19/597—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding specially adapted for multi-view video sequence encoding
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
  • H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
  • H04N13/106—Processing image signals
  • H04N13/156—Mixing image signals
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
  • H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
  • H04N13/106—Processing image signals
  • H04N13/161—Encoding, multiplexing or demultiplexing different image signal components

Definitions

• the present invention relates generally to a method used to combine dual streams of video into a standard single stream of video. More particularly, the present invention relates to a method of combining a dual stream of standard video, to occupy a single stream of standard video, providing a means to enhance a viewers experience in several ways.
• This invention describes a method of combining a dual stream of standard video, to occupy a single stream of standard video, providing a means to enhance a viewers experience in several ways.
• This invention provides a method of combining two standard video streams, into one standard video stream, by tiling two lower resolution images frames into one higher resolution image frame, without loss of pixel data.
• This tiling method which is done by mapping pixel data from two lower resolution frames into new pixel positions of a single higher resolution frame. This is done by tiling the higher resolution frame, with segments of the two lower resolution frames.
• this tiling will ensure in most cases, that when there is camera movement from one camera, the other camera will have movement in the same vector direction.
• This tiling will ensure in most cases, that when there is no camera movement from one camera, the other camera will have no movement as well.
• This tiling method is therefore advantageous for the compression of the tiled frame sequence, by compression algorithms such as MPEG-2, MPEG- 4, and WM-9, which rely on temporal redundancy to encode more efficiently.
• This display device may have one or more capabilities to present to the viewer, several modes which are possible, and described in this invention as “2D Mode”, “Dual-View” mode, “Pan-and-Scan Mode”, and “Stereoscopic 3D Mode”
• FIG.l shows the first video source, with a frame resolution of
• FIG.l is labeled "Left-Eye" to distinguish it from the second video source, by example.
• FIG.2 shows the second video source, with a frame resolution of
• FIG.2 is labeled "Right-Eye" to distinguish it from the first video source, by example.
• FIG.3 shows the combined pair of video frames of [FIG.l]
• FIG.2 as a "tiled" frame having a resolution of 1920x1080, which could constitute the Stereoscopic image pair, for example.
• This resolution is an ATSC and SMPTE video standard.
• Fig.3 is considered the encoded "tiled" frame. It is a typical layout for the tiling, but is not limited to this arrangement of tiled segments.
• FIG.3 which occupies l/9th of the area of the frame, or 640x360 pixels, may be used to insert additional imagery, such as a thumbnail sub-frame, or areas of the imagery adjacent to the stitched areas of the tiling, if this improves the compression efficiency.
• each video stream [FIG.1,2] is first digitized to an associated memory buffer.
• the memory buffers are updated for each incoming video stream, on a pixel-by-pixel sequential basis.
• the memory buffers can be in a dual-ported FIFO configuration, or single-ported SRAM or VRAM configuration, as long as the bus bandwidth for writing and reading the memory is sufficient to satisfy a simultaneous read and write cycle, and read/write address contention is avoided by hardware, or bank- switched (toggled) to ensure no contention.
• FIG 1,2] are linearly addressed, and the read cycles have an address generator which transposes the address to match the sequence required to tile the output frame [FIG.3].
• the memory buffer needs to have the capacity to hold two input video frames, or four input frames if the contention avoidance is created by bank switching.
• FIG1,2] are addressed by an address generator, which transposes the write address, such that the output read cycles for the output tiled frame [FIG.3] will be linearly addressed.
• the memory buffer needs to have the capacity to hold a single output tiled frame, or two output frames if the contention avoidance is created by bank switching.
• the input source frames [FIG1 ,2] are typically gen-locked together to ensure this memory model works.
• the above method describes a hardware method of combining two sources frames [FIG1,2] to an output tiled frame [FIG.3]. This operation may also be done by rendering the frames in software to render the same output frame [FIG.3] from the two source frames [FIG1,2] stored in a computer's memory, or on a disk. [0029] There are various HDTV standards that will accommodate this tiling method, which is done by mapping pixel data from two lower resolution frames into new pixel positions of a single higher resolution tiled frame, without loss of pixel data.
• two frames of 1280x720 can be tiled into a frame of 1920 x 1080. It is similarly possible to tile two frames of 640x480 into a frame of 1280x720.
• pixel data is not lost, but it is also possible to reduce the size of the input frames to match the tiling requirements of the output tiled frame, in which case pixel interpolation will be required, and some pixel data will be lost in this conversion.
• FIG.3 will ensure in most cases, that when there is no camera movement from one camera [FIG.1] , the other camera [FIG.2] will normally have no movement as well.
• This tiling method is therefore advantageous for the compression of the tiled frame sequence, by video compression algorithms such as MPEG-2,
• the tiled video after it is decompressed into a single stream of tiled video [FIG.3] , needs to be decoded back into dual streams of video [FIG.1 ,2] just prior to viewing on a display device, such as a TV, projector, or computer monitor.
• a display device such as a TV, projector, or computer monitor.
• This display device may have one or more capabilities to present to the viewer, several modes which are possible, and described in this invention as “2D Mode”, “Dual-View” mode, “Pan-and-Scan Mode”, and “Stereoscopic 3D Mode”
• “2D Mode” is a mode that displays a single stream of decoded video. Either [FIG.1] or [FIG.2] just like regular 2D Video. The decoder presents to the display just one fixed source of video.
• Double- View Mode is a mode that allows the viewer to select one of the two sources from the decoder, just like an A/B switch selecting a source of either [FIG.1] or [FIG.2] .
• the input to the display can multiplex from one source to the other.
• the viewer can manually select, from two camera views that have been encoded, for example.
• Pan-and-Scan Mode is a mode in which the source material of the encoded tiled frame contains video imagery that has been "stitched” together either horizontally or vertically, to create a panoramic view. This can be done by capturing from two adjacent video cameras, with each having a field of view with a common side, such that when "stitched” together would create a panoramic view either horizontally or vertically.
• the viewer can adjust a sliding "window” to view any portion of the panorama in full screen.
• This windowing needs to be performed by the decoder, by shifting the pixel column or row starting address of the memory being read, and displayed on the display device.
• Steposcopic 3D Mode is a mode that displays the two video sources [FIG.1,2] and normally requires the tiled video stream [FIG.3] to contain
• the display device will display
• the display device can support, such as anaglyph, polarized, or field interleaved.
• the viewer also has the choice to view the Stereoscopic video content in 2D, by selecting "Dual- View Mode” and manually choosing "left-eye” view [FIG.l], or "right-eye” view [FIG.2]
• the display if it has the capability to convert dual streams to anaglyph 3D, by the standard mathematical process, in prior art, the viewer will be capable to view anaglyph 3D, using colorized glasses.
• the source material for each eye may also be encoded such that it is already in anaglyph format, in which case the TV will display the summation of the colorized "left-eye” view [FIG.l] and "right-eye” view [FIG.2].
• the viewer will be capable to view anaglyph 3D, using colorized glasses.
• the source material for each eye may also be encoded such that it is already in anaglyph format, in which case the TV will display the summation of the uncolorized 2D normal view [FIG.1] and the combined colorized "right-eye” and "left-eye” views [FIG.2].
• the viewer will be capable of watching the content in a 2D mode without glasses, or to view anaglyph 3D, using colorized glasses.
• the TV is capable of generating polarized Stereoscopic 3D, from a dual stream of video, then the viewer will be capable of viewing Stereoscopic 3D using polarized glasses.
• the TV is capable of generating field-interleaved Stereoscopic
• the capabilities enabled by having a source of dual streams of video presented to the display device creates an enhanced viewing experience.

Landscapes

• Engineering & Computer Science (AREA)
• Multimedia (AREA)
• Signal Processing (AREA)
• Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)
• Compression Or Coding Systems Of Tv Signals (AREA)

Priority Applications (1)

Applications Claiming Priority (2)

Publications (2)

Family

ID=35394842

Family Applications (1)

Country Status (4)

Cited By (5)

Families Citing this family (79)

Family Cites Families (12)

• 2004
  • 2004-05-07 US US10/840,592 patent/US20050041736A1/en not_active Abandoned
• 2005
  • 2005-05-06 CN CNA2005800186368A patent/CN1981522A/zh active Pending
  • 2005-05-06 JP JP2007511598A patent/JP2007536825A/ja active Pending
  • 2005-05-06 WO PCT/US2005/015678 patent/WO2005112448A2/en not_active Ceased

Also Published As

Similar Documents

Legal Events