Classifications

• • G—PHYSICS
  • G11—INFORMATION STORAGE
  • G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
  • G11B27/00—Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
  • G11B27/02—Editing, e.g. varying the order of information signals recorded on, or reproduced from, record carriers
  • G11B27/031—Electronic editing of digitised analogue information signals, e.g. audio or video signals
• • G—PHYSICS
  • G11—INFORMATION STORAGE
  • G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
  • G11B27/00—Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
  • G11B27/10—Indexing; Addressing; Timing or synchronising; Measuring tape travel
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
  • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
  • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
  • H04N21/238—Interfacing the downstream path of the transmission network, e.g. adapting the transmission rate of a video stream to network bandwidth; Processing of multiplex streams
  • H04N21/2389—Multiplex stream processing, e.g. multiplex stream encrypting
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
  • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
  • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
  • H04N21/4302—Content synchronisation processes, e.g. decoder synchronisation
  • H04N21/4307—Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen
  • H04N21/43072—Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen of multiple content streams on the same device
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
  • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
  • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
  • H04N21/432—Content retrieval operation from a local storage medium, e.g. hard-disk
  • H04N21/4325—Content retrieval operation from a local storage medium, e.g. hard-disk by playing back content from the storage medium
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
  • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
  • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
  • H04N21/438—Interfacing the downstream path of the transmission network originating from a server, e.g. retrieving encoded video stream packets from an IP network
  • H04N21/4385—Multiplex stream processing, e.g. multiplex stream decrypting
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
  • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
  • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
  • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
  • H04N21/44016—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving splicing one content stream with another content stream, e.g. for substituting a video clip

Definitions

• the invention relates generally to video and audio signal encoding and decoding and manipulation, and in particular, to a method and apparatus for flexibly encoding and decoding multiple MPEG digital video and audio streams.
• the television industry requires the availability of high quality, full motion audio and video signals, typically emanating from different sources.
• United States Patent Number 5,862,312, filed October 24, 1995, issued January 19, 1999, and entitled LOOSELY COUPLED MASS STORAGE COMPUTER CLUSTER (the disclosure of which is incorporated herein, in its entirety, by reference) , there is disclosed a system and method for reliably and continuously delivering multiple media streams even if one of the nodes of the described cluster were to fail.
• These streams are typically stored as MPEG encoded streams and/or compressed audio streams. It is important to be able to switch between the various streams in a seamless, continuous and reliable manner and to provide high quality audio and video signals having on-air broadcast quality.
• the invention relates to a method for playing the decoded MPEG signals in a flexible and structured manner.
• the invention relates to receiving, at a high data rate, plural MPEG data encoded streams from at least one source, decoding the received streams, and synchronizing the multiple decoded streams to a reference clock.
• the multiple output streams can have varying formats and can include both video and audio streams .
• the invention also provides the ability to play any type of MPEG file immediately followed by another file and to do so seamlessly.
• the invention further provides the ability to skip B frames to obtain a desired reference frame quickly. Further, the invention enables the user to edit MPEG streams regardless of the GOP size. The editing can be accomplished for GOP sizes of two or greater.
• Figure 1 illustrates an overall functional block diagram of the apparatus of the invention
• FIG. 2 shows a more detailed schematic diagram of a controller in accordance with the invention
• Figure 3 illustrates a more detailed schematic of the controller of Figure 2 ;
• FIG. 4 illustrates a schematic diagram of an interface switch in accordance with the invention
• FIG. 5 illustrates a more detailed schematic of the interface switch of Figure 4;
• Figure 6 illustrates the interface switch FPGA;
• Figure 7 illustrates a GENLOCK circuitry in accordance with the invention;
• Figure 8 illustrates schematically the frequency phase detector circuitry in accordance with the invention.
• Figure 9 illustrates operation of the GENLOCK method .
• a decoding system 8 has a controller 10, incorporating a central processing unit (CPU) 12, a random access memory 14, a PCI interface 16, and MPEG decoders 20, and an interface switch 24, having digital and analog outputs 26, a GENLOCK input 28, audio and video switching 30, a decoder to decoder interface 32 and a Movie2 bus interface 34.
• CPU central processing unit
• RAM random access memory
• PCI random access memory
• MPEG decoders 20 MPEG decoders
• an interface switch 24 having digital and analog outputs 26, a GENLOCK input 28, audio and video switching 30, a decoder to decoder interface 32 and a Movie2 bus interface 34.
• the decoding system produces two independent switchable channels of analog or digital video and four independent switchable stereo channels of analog or digital audio (available over lines 36) from compressed MPEG data delivered over the PCI bus.
• the PCI bus can operate with a 33 Mhz, 32-bit data bus architecture that shares bus bandwidth between a PCI controller 42 (Fig. 2) through bus master arbitration.
• Each channel has an MPEG decoder 20 and, with an optional second decoding system, four (4) MPEG decoders 20 can be used, each providing two stereo channels of audio and one channel of video decompression.
• These decompressed audio/video streams are switched using an "on-the-fly" re-programmable digital cross-point switch 30 to connect to any analog or digital output 26.
• two or more decoding systems can be connected together to add additional channels and more robust switching capabilities.
• a "master- slave” configuration is used, and a total of four analog/digital (a/d) video and eight a/d audio stereo channels can be "on-the-fly” switched to any outputs.
• This configuration of re-programmable switching is very useful in "Program/Preview” applications typically used in a broadcast environment. This switching capability can also be used in a variety of editing applications.
• the decoding system 8 provides an interface to signals from off-the-shelf video effects units. This is accomplished using a Movie2 bus 70 (see Fig. 1) .
• a Movie2 bus is a board-to-board interconnect standard that provides a 27 Mhz time-multiplexed uncompressed multi-channel video exchange between connected units. Based on CCIR 601, SMPTE126M and EBU technical specifications, a Movie2 bus allows professional audio/video products to interconnect and exchange full -resolution studio quality video without burdening local CPU busses.
• the Movie2 bus supports a sustained aggregate data rate well in excess of 300 MBps (MegaBytes per second) .
• Movie2 bus allows the decoding system, in accordance with the illustrated embodiment, to route video to, from and/or through other units that support the Movie2 bus.
• Movie2 signals are switched using the cross-point switch 30, of switch 24. This allows decoded MPEG outputs to be routed to a Movie2 bus and signals from a Movie2 bus to be routed to the system's digital and analog outputs 26.
• two decoding systems would be connected, as noted above, through a master-slave bus while a special feature unit would be connected to the master decoding system through a Movie2 bus connection 70.
• decoded video from multiple channels can be routed through Movie2 bus 70 and connected to special effects units (such as character generators, still stores, 2D and 3D effects/mixers, etc.)
• special effects units such as character generators, still stores, 2D and 3D effects/mixers, etc.
• the result can be displayed before or after passing through switch 30, and can be routed back through an MPEG encoder to encode any resulting edited video.
• the controller 10 consists of the following major functional elements: the CPU 12 and the PCI Control 42 for the PCI bus 111 through PCI interface 16, RAM 14, an FPGA (field programmable gate array) Control 84, FPGA unit 86, two MPEG Audio/Video Decoders 20, Audio/Video Switch 87, Audio/Video Outputs 36, Peripherals and a System Sync. These functional areas combine to produce two independent switchable channels over lines 36, outputting analog/digital audio and video from MPEG data delivered through PCI bus 111.
• the embedded CPU processor 12 for example, an Intel 80960HA controls high speed data transfers from the RAM to the audio/video decoders 20.
• a local Bus/PCI bus master interface chip acting as PCI controller 42 (for example, PCL Technology PCI9080) performs high speed data transfers from the PCI bus to the RAM, controlled by logic in the FPGA 84.
• the embedded processor also performs audio/video initialization and assists with an MPEG video decoding for each audio/video decoder channel over outputs 36.
• the controller contains two serial ports 100, 102, and status LED's over lines 104 for external status/monitoring.
• An external system sync (external video input) at 106 provides optional synchronization for all channels of a system.
• CPU and PCI elements include the PCI bus interface 16, embedded CPU 12, a CPU Boot PLX 121 and the controller FPGA logic 84.
• The, system receives MPEG data through the PCI bus interface 42.
• a PCI interface chip 110 controls the interface function between the PCI bus 111 and an embedded CPU's local bus 112.
• the PCI interface 16 requests the CPU bus 112 (becomes local bus master) in order to transfer data in a burst mode, into and out of local RAM 14.
• the embedded CPU processor 12 is used to control MPEG data movement to the two audio/video decoders 20, initialization of the audio/video decoders and initialization/management of the system.
• the CPU 12 shares a bus with a PCI bus master interface chip 110 (of interface 16) and a DMA bus master control 120 located in a controller FPGA 84.
• RAM control is performed by the controller FPGA 84 and direct memory access (DMA) and I/O control are processed by the controller FPGA 84.
• Chip selects, CPU start-up logic and FPGA programming control are performed in the CPU boot PLD 121.
• a boot sectored flash memory 122 (for example, 512K x 8) provides initialization code for CPU boot operations.
• Application code is downloaded to RAM as part of an initialization process.
• Mainline CPU code is processed from RAM.
• Serial ports 100, 102 or the PCI interface 16 can be used to download new code to the flash memory 122.
• FPGA configuration (s) video quantization tables and other non-volatile configuration data is stored in flash memory as well.
• the decoding system decompresses two channels of video and four channels of stereo audio.
• Video can be displayed at SIF, half D-l or D-l resolutions in NTSC or PAL formats in accordance with the invention.
• Audio outputs can be mono, dual mono, stereo or joint stereo.
• the MPEG decoder for example, an IBM MPEGCD21 decodes compressed source material having one of the formats listed below:
• the MPEG decoder 20 extracts MPEG DTS/PTS time stamps from packetized data streams and associates these time stamps to synchronize the audio/video outputs.
• Digital video outputs are transferred through an audio/video decoder switch to programmed analog and digital video/audio outputs 36.
• Video is converted to analog using a digital video encoder 126.
• Digital audio outputs are likewise converted using a digital-to-analog audio converter 128. Audio/video analog outputs are buffered and amplified prior to final output.
• the decoding system provides CPU controlled audio level adjustments and fine tuning adjustments through the interface switch 24 (Fig. 1) .
• Video can also be converted to serial Dl, component digital video, through the interface switch.
• a digital audio bit stream can be converted and output as AES/EBU type digital audio.
• MPEG compressed data is input through the FPGA' s 84, 86, which create a serial input/output on bus 130, a sixteen bit data bus, which is shared, for asynchronous reads and writes, to an internal processor of an MPEG decoder chip.
• Bit and display buffers are dynamically sizable and are located in the MPEG decoder's dedicated RAM 133. RAM can be configured for 4:2:0, Dl, for 4:2:2, Dl, or for 4:2:2, full CCIR 601 resolution.
• the audio/video decoder 20 has a 27 MHz operational clock implemented using a VCXO/system sync circuit located on the interface switch 24. Audio and video specific clocks are produced by an internal PLL circuitry of the MPEG decoder. Audio PCM clocks and a 256fs clock (256 times the sampling frequency) are output to an A/V cross-point switch 30 of the interface switch 24.
• the VCXO/system sync function also produces vertical and horizontal sync signals to all MPEG decoders 20.
• Audio levels are controlled by software programming of the MPEG decoder's attenuation registers and/or programming audio level adjust ICs located on the interface switch 24.
• a more detailed schematic block diagram of the control 10 is illustrated in Fig. 3.
• interface switch 24 receives decoded digital audio/video data from the controller 10 as well as optionally from another decoding system 10a and/or the Movie2 bus interface 34. Digital audio and video are switched using the re-programmable cross-point switch array 30.
• the audio/video signals can be switched to: (1) on-board output circuitry, (2) another decoding system, or (3) a Movie2 bus interface - from: (1) onboard decoders, (2) another decoding system, or (3) a Movie2 bus interface .
• the output can be broadcast quality analog audio, AES/EBU digital audio, broadcast quality analog video and SMPTE serial Dl (component digital) video.
• the interface switch 24 also provides line locked GENLOCK for all video channels and color locked GENLOCK for analog video channels.
• the re-programmable cross-point switch 30 is used to reconfigure connections of decoder 20 audio/video outputs 36.
• Each MPEG decoder audio and video output over lines 36 can be connected to one of analog and/or digital outputs 26 of the interface switch.
• This switching arrangement provides program and preview functions which are used throughout the broadcast industry. It also provides an - 9 - ability to route signals to and from a daughter card, and to and from a Movie2 bus 70, providing a wide variety of an on-air, on-line and off-line editing and effects capabilities .
• the apparatus of the invention provides the ability to play any type of MPEG file immediately followed by another file.
• the transition between files is seamless and is controlled by the embedded CPU 12 of the controller in conjunction with the interface switch 24.
• the first MPEG stream is stored in the CPU memory 14 and is decoded using one of the decoders 20.
• the second stream is stored in the CPU memory 14 and is available for decoding by the other decoder 20 of the controller 10.
• the second stream begins playing by the action of the cross-point switching array 30. For example, that array switches the first input stream over lines 36 to the second input stream over those lines in a seamless and continuous manner so that the output over the analog/digital audio/video outputs 26 is provided with a seamless transition from the first stream to the second stream.
• the decoders 20 are flexible enough to handle varying formats . Not only can they handle different input formats, but they can also provide multiple output formats as well. As an example of the flexibility of the system, both decoders can be operating simultaneously on different input streams and the outputs, over the lines 36, from each switch to the cross-point switching array 30 can then be connected to the correct output lines 26. Accordingly, the invention provides for the output of multiple streams, simultaneously, one stream, and for a transition between two streams. - 10 -
• the control circuitry 10 also enables the system to skip a plurality of so-called B frames in order to get to the desired reference frames more quickly.
• the skipped frames need not be played and can, accordingly, be discarded.
• all of the input streams can be synchronized to the same reference clock so that, using the GENLOCK signal and circuitry, they can be synchronized to each other.
• the structure of the invention can be used to enable one stream to be previewed while the other stream is being played as an output, that is, on-air.
• the previewed stream can be directed to either the Movie2 bus directly or to a separate output directed to a display monitor at, for example, the studio itself. In this manner, different video streams can be reviewed while a known stream is being output for broadcast.
• a second decoding system is employed, yet greater flexibility can be obtained with the control elements preferably operating in the master- slave relationship.
• the Movie2 bus 70 provides a standard passive back plane interface 34 to add special features to an output of a decoder. Mixing, 2D video effects and linear and alpha keying are a few special features that can be added.
• the Movie2 bus is located on the interface switch on which I/O is processed using the cross-point switch 30 and programmable logic located in an interface FPGA 241.
• An Interface EPLD 243 provides two channels of YUV to SMPTE-125 parallel Dl conversion (for data sent to Dl digital video converters), video GENLOCK PLL control, - 11 - video Hsync, Vsync and field timing, SMPTE-125 embedded audio packaging, LITC data intput and output formatting, and miscellaneous hardware control. Miscellaneous hardware control communicates to the control 10 through a serial input/output (SIO) bus 270.
• SIO serial input/output
• FIG. 5 A more detailed schematic description of the interface switch is provided in Fig. 5 and Fig. 6 (interface switch FPGA) .
• Video GENLOCK PLL Control Referring to Fig. 7, an FPGA is used to provide video GENLOCK control between an on board VCXO 293 and an external reference 284.
• the internal VCXO is locked in frequency and phase to the external video hsync. If an external clock source is not present, a control voltage will be set to a mid-range frequency of the VCXO.
• System GENLOCK uses the VCXO clock source 293, the FPGA and a 12 bit digital to analog (D/A) converter 290. All of the phase-locking intelligence is located in the FPGA.
• the D/A converter 290 is updated every horizontal sync period.
• the D/A converter responds, in approximately 16-32 usec, by changing its output analog control voltage over line 292 to the VCXO 293.
• the FPGA creates an outgoing horizontal sync (H out ) (as well all other video timing) based on the VCXO clock frequency.
• the outgoing horizontal sync (H out ) locks to incoming horizontal sync (H ⁇ n ) through a two step process. Each step uses digital logic to produce the desired response.
• a frequency detector 296 measures the H ⁇ n period in numbers of 27Mhz clock samples.
• the VCXO 293 is then adjusted until the number of clocks per line is correct (1716 for NTSC, and 1728 for PAL) .
• the second step of the phase lock process further adjusts frequency and aligns the phases of H out and H in .
• step 506 This is achieved by first synchronizing H out with H in (step 508) , then measuring and adjusting the D/A 290 for each clock sample of phase difference detected, (step 510)
• a "snap" algorithm is applied to achieve minimum alignment time. This algorithm is based upon a continuous or analog model of phase locking. As the two signals "slide” in phase with respect to one another, the phase lock loop attempts to "pull” or adjust the VCXO. When the VCXO frequency passes through that state in which H out and H in are equal, the direction of the "slide” will change.
• the frequency/phase detector 296 is illustrated in greater detail in Fig. 8.
• a continuous system is approximated by adjusting the VCXO in equal frequency increments every line time. In this way, the rate of change of the frequency is linear.
• the desired frequency is achieved.
• the lock frequency is the midpoint of the frequency difference between the threshold frequencies. (This is the point where the slope is zero.) This frequency is loaded into the D/A converter the next time the phase difference is zero, causing a "snapping" effect to the desired frequency.

Landscapes

• Engineering & Computer Science (AREA)
• Multimedia (AREA)
• Signal Processing (AREA)
• Databases & Information Systems (AREA)
• Television Signal Processing For Recording (AREA)
• Studio Circuits (AREA)

Applications Claiming Priority (3)

Publications (1)

Family

ID=22160481

Family Applications (1)

Country Status (6)

Families Citing this family (7)

Family Cites Families (8)

• 1999
  • 1999-04-06 EP EP99917346A patent/EP1078522A1/en not_active Withdrawn
  • 1999-04-06 CA CA002327620A patent/CA2327620A1/en not_active Abandoned
  • 1999-04-06 WO PCT/US1999/007507 patent/WO1999052283A1/en not_active Application Discontinuation
  • 1999-04-06 JP JP2000542917A patent/JP2003514404A/ja active Pending
  • 1999-04-06 IL IL13887099A patent/IL138870A/xx not_active IP Right Cessation
  • 1999-04-06 CN CNB998049077A patent/CN1184813C/zh not_active Expired - Fee Related

Non-Patent Citations (1)

Also Published As

Similar Documents

Legal Events