WO2012171156A1 - Wireless video streaming using usb connectivity of hd displays - Google Patents

Wireless video streaming using usb connectivity of hd displays Download PDF

Info

Publication number
WO2012171156A1
WO2012171156A1 PCT/CN2011/075641 CN2011075641W WO2012171156A1 WO 2012171156 A1 WO2012171156 A1 WO 2012171156A1 CN 2011075641 W CN2011075641 W CN 2011075641W WO 2012171156 A1 WO2012171156 A1 WO 2012171156A1
Authority
WO
WIPO (PCT)
Prior art keywords
video
file
data
video data
stream
Prior art date
Application number
PCT/CN2011/075641
Other languages
French (fr)
Inventor
Junfeng Wang
Original Assignee
Panovel Technology Corporation
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Panovel Technology Corporation filed Critical Panovel Technology Corporation
Priority to PCT/CN2011/075641 priority Critical patent/WO2012171156A1/en
Publication of WO2012171156A1 publication Critical patent/WO2012171156A1/en

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41Structure of client; Structure of client peripherals
    • H04N21/4104Peripherals receiving signals from specially adapted client devices
    • H04N21/4122Peripherals receiving signals from specially adapted client devices additional display device, e.g. video projector
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/436Interfacing a local distribution network, e.g. communicating with another STB or one or more peripheral devices inside the home
    • H04N21/4363Adapting the video or multiplex stream to a specific local network, e.g. a IEEE 1394 or Bluetooth® network
    • H04N21/43637Adapting the video or multiplex stream to a specific local network, e.g. a IEEE 1394 or Bluetooth® network involving a wireless protocol, e.g. Bluetooth, RF or wireless LAN [IEEE 802.11]
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00Details of television systems
    • H04N5/76Television signal recording
    • H04N5/765Interface circuits between an apparatus for recording and another apparatus
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00Details of television systems
    • H04N5/76Television signal recording
    • H04N5/765Interface circuits between an apparatus for recording and another apparatus
    • H04N5/775Interface circuits between an apparatus for recording and another apparatus between a recording apparatus and a television receiver

Definitions

  • the present invention relates to wireless video data streaming technology.
  • Video refers to a technology for electronically capturing, storing, and playing a sequence of still images that represent scenes in motion.
  • High-definition video e.g., "HD” video
  • HD video displays may have display resolutions of 1,280 by 720 pixels (720p), 1,920 by 1,080 pixels (1080i/1080p), or other resolutions.
  • Televisions and other display devices are being manufactured with high definition resolutions in increasing numbers. As such, techniques for providing high definition video data to high definition displays are being developed, including wireless techniques.
  • Wireless video streaming is desired between various types of video source devices, such as set top boxes, desktop computers, and laptops, and various types of display devices, including HD displays
  • Push mode and pull mode video streaming techniques have been used to transport video between electronic devices and displays.
  • a portable electronic device initiates and controls the transfer of the video to the display device.
  • push mode video streaming is the broadcast video service in cable or satellite networks.
  • the video source device e.g., a satellite or cable service.
  • pull mode streaming the display initiates and controls the transfer of the video from the portable electronic device.
  • HDMI High-Definition Multimedia Interface
  • WiHD and WHDI interface solutions provide a replacement for HDMI, while some solutions based on WiFi and UWB use an external video decoder at the display device to provide HDMI connectivity.
  • existing replacement technologies either wirelessly stream raw HD video at very high data rates (e.g., in the order of gigabits-per-second), such as in WiHD, or use video codecs (coder-decoders) to reduce the wireless streaming data rate, such as in WHDI. Both of these approaches have cost problems that make them prohibitive for mass production. Cost issues also exist for solutions based on WiFi (e.g., WiDi) or UWB with HDMI connectivity due to the external HD decoder used at the display device.
  • WiFi e.g., WiDi
  • UWB with HDMI connectivity due to the external HD decoder used at the display device.
  • Video source devices are configured to stream video data
  • display devices are configured to receive the streamed video data, using standard communication interface technologies, such as USB (universal serial bus) interfaces.
  • standard communication interface technologies such as USB (universal serial bus) interfaces.
  • a video source device includes a USB connector, a storage device, and a driver.
  • the driver receives a request through the USB connector for a video file that contains video data.
  • the request is wirelessly received at the video source device from a display device.
  • the request indicates a virtual file storage location at the video source device for the video file.
  • the virtual file storage location is a virtual location in a virtual USB mass storage device.
  • the driver determines a mapping for the requested video file from the virtual file storage location to a storage location in the storage device.
  • the driver retrieves the requested video file from the storage location in the storage device, and transmits the video data of the video file through the USB connector to be wirelessly transmitted in a video data stream in response to the request.
  • Video source devices implementations are also provided that wirelessly stream video data received by the video source device in real time.
  • a display device in another implementation, includes a USB connector and a decoder.
  • the decoder includes a USB interface and a decoding engine.
  • the USB interface receives video data through the USB connector that is wirelessly received at the display device from a video source device.
  • the decoding engine decodes the received video data for display.
  • the decoder further includes a data diverter, a stream data path processing logic, and a file data path processing logic.
  • the stream data path processing logic is configured to process stream data to generate first processed video data that is received by the decoding engine.
  • the file data path processing logic is configured to process file data to generate second processed video data that is received by the decoding engine. If the video data received through the USB connector is stream data, the data diverter routes the received video data to the stream data path processing logic. If the video data received through the USB connector is received in the form of file data, the received video data is provided to the file data path processing logic.
  • Computer program products are also described herein that enable streaming of video data, and that enable further embodiments as described herein.
  • Devices in which embodiments may be implemented may include storage, such as storage drives, memory devices, and further types of computer-readable storage media described herein.
  • Such computer-readable storage media may store program modules that include computer program logic for performing the systems, processes, and further embodiments described herein.
  • FIG. 1 shows a block diagram of a video streaming system that includes a video source device and a display device, according to an example embodiment.
  • FIGS. 2-4 show respective block diagrams of example video streaming systems.
  • FIG. 5 shows a block diagram of a video streaming system, according to an example embodiment.
  • FIG. 6 shows a flowchart for wirelessly streaming video data from a video source device, according to an example embodiment.
  • FIG. 7 shows a flowchart for receiving streamed video at a display device, according to an example embodiment.
  • FIG. 8 shows a block diagram of a video streaming system, according to an example embodiment.
  • FIG. 9 shows a block diagram of a video source device configured to store video data in a virtual storage to be accessible over a USB (universal serial bus) link as a USB file, according to an example embodiment.
  • a USB universal serial bus
  • FIG. 10 shows a block diagram of the video source device of FIG. 9 providing the video data over the USB link, according to an example embodiment.
  • FIG. 11 shows a flowchart providing a process for streaming video data from virtual storage over a USB link, according to an example embodiment.
  • FIGS. 12A and 12B show block diagrams of a decoder, according to example embodiments.
  • FIG. 13 shows a flowchart providing a process in a decoder for rerouting a video data stream received at a USB interface of the decoder, according to an example embodiment.
  • FIG. 14 shows a block diagram of a video streaming system, according to an example embodiment.
  • FIG. 15 shows a process in a video source device for streaming a received video data stream, according to an example embodiment.
  • FIG. 16 shows a block diagram of a video streaming system, according to an example embodiment.
  • FIG. 17 shows a flowchart providing a process in a video source device for streaming a received video data stream from virtual storage, according to an example embodiment.
  • references in the specification to "one embodiment,” “an embodiment,” “an example embodiment,” etc., indicate that the embodiment described may include a particular feature, structure, or characteristic, but every embodiment may not necessarily include the particular feature, structure, or characteristic. Moreover, such phrases are not necessarily referring to the same embodiment. Further, when a particular feature, structure, or characteristic is described in connection with an embodiment, it is submitted that it is within the knowledge of one skilled in the art to effect such feature, structure, or characteristic in connection with other embodiments whether or not explicitly described.
  • Embodiments of the present invention enable the wireless streaming of video, such as high definition (HD) video, from source devices.
  • source devices may store the video (e.g., in the form of video data) locally (e.g., in local storage) or may stream the video from the source device simultaneously with the video being received in a stream from another source device.
  • a display device may receive the streamed video, and may display the streamed video on a display screen.
  • FIG. 1 shows a block diagram of a video streaming system 100.
  • system 100 includes a video source device 102 and a display device 104.
  • video source device 102 includes an antenna 106 and video information 1 10.
  • Video source device 102 may optionally include a display screen for display of video, a user interface, and/or other features. When present, the display screen may be any type of display screen, including an LCD display screen, an LED display screen, etc.
  • Video source device 102 may store video information 110 in storage in the form of video files, may receive video information 110 from one or more sources external to video source device 102, and/or may have access to video information 1 10 in other ways. As shown in FIG.
  • video source device 102 generates a video data stream 1 16 based on video information 110.
  • Video data stream 116 is streamed from antenna 106 as a stream of video data to display device 104.
  • display device 104 includes an antenna 108 and a display screen 1 12.
  • Antenna 108 of display device 104 receives video data stream 1 16, and display device 104 displays video 1 14 on display screen 112 based on video data stream 112.
  • video 114 may be simultaneously displayed on display screen 1 12 of display device 104 and on a display screen of video source device 102.
  • display screen 110 may display video 114 imposed on background graphics shown on the display screen of video source 102.
  • Such background graphics may be visible when video 114 is displayed by the display screen of video source device 102 in less than full screen, and may include one or more of a background color and/or pattern, one or more icons, one or more other applications being used (e.g., an Internet browser viewing a web page, a document open in a word processor, an email tool, etc.), etc., displayed on the display screen of video source device 102.
  • video 114 may be displayed by display screen 112 of display device 104 without being simultaneously displayed at video source device 102, and/or without background graphics from the display screen of video source device 102.
  • HDMI HDMI is a compact audio/video interface for transmitting uncompressed digital data.
  • HDMI is becoming a standard for transmitting HD video data for HDTV, enabling the transmission of numerous HD formats, such as 720p, 1080i, 1080p, etc. Because uncompressed data is transmitted according to HDMI, the use of HDMI leads to very high data transmission rates, including data rates in the order of gigabits-per-second (Gbps). Examples of wireless video streaming according to WiFi, UWB, WiHD, and WHDI are described as follows with respect to FIGS. 2-4.
  • FIG. 2 shows a block diagram of a video streaming system 200.
  • System 200 is an example of system 100 of FIG. 1. As shown in FIG. 2, system 200 includes a video source device 202, a display device 204, an HDMI receiver 208, a WiHD transmitter 210, an HDMI transmitter 214, a WiHD receiver 216, an antenna 218, and an antenna 220.
  • Video source device 202 is an example of video source device 102
  • display device 204 is an example of display device 104.
  • HDMI receiver 208, WiHD transmitter 210, and antenna 218 form a transmitter for video source device 202 for raw video data.
  • video source device 202 has an HDMI connector 206.
  • HDMI receiver 208 includes an HDMI connector (not shown in FIG. 2) that mates with HDMI connector 206.
  • Video source device 202 provides video information 1 10 through HDMI connector 206 as raw video data to HDMI receiver 208.
  • HDMI receiver 208 receives the raw video data, and WiHD transmitter 210 transmits the raw video data from antenna 218 as wireless raw video data stream 222 according to the WiHD communication protocol.
  • WiHD receiver 216 forms a receiver for display device 204 for raw video data.
  • display device 204 has an HDMI connector 212.
  • HDMI transmitter 214 includes an HDMI connector (not shown in FIG. 2) that mates with HDMI connector 212.
  • WiHD receiver 216 receives raw video data stream 222 according to the WiHD communication protocol.
  • HDMI transmitter 214 transmits the raw video data of raw video data stream 222 received by WiHD receiver 216 to display device 204 through HDMI connector 212.
  • Display device 204 displays the raw video data as video 114.
  • FIG. 3 shows a block diagram of a video streaming system 300.
  • System 300 is an example of system 100 of FIG. 1. As shown in FIG.
  • system 300 includes video source device 202, display device 204, HDMI receiver 208, HDMI transmitter 214, antenna 218, antenna 220, a first WHDI communication module 302, and a second WHDI communication module 304. These elements of FIG. 3 are described as follows.
  • First WHDI communication module 302 is a transmitter for video source device
  • first WHDI communication module 302 includes an encoder 306 and a WHDI transmitter 308.
  • Video source device 202 has HDMI connector 206, and as described above, HDMI receiver 208 mates with HDMI connector 206.
  • Video source device 202 provides video information 1 10 through HDMI connector 206 as raw video data to HDMI receiver 208.
  • HDMI receiver 208 receives the raw video data, and encoder 306 compresses the raw video data to generate compressed video data.
  • encoder 306 may compress the raw video data according to MPEG-2, H.264, or other video file standard.
  • WHDI transmitter 308 transmits the compressed video data from antenna 218 as compressed video data stream 314 according to the WHDI communication protocol.
  • Second WHDI communication module 304 is a receiver for display device 204 for compressed video data.
  • second WHDI communication module 304 includes a WHDI receiver 312 and a decoder 310.
  • display device 204 has HDMI connector 212, and as described above, HDMI transmitter 214 mates with HDMI connector 212.
  • WHDI receiver 312 receives compressed video data stream 314 according to the WHDI communication protocol.
  • Decoder 310 decompresses the compressed video data of compressed video data stream 314 to raw video data, and HDMI transmitter 214 transmits the raw video data to display device 204 through HDMI connector 212.
  • Display device 204 displays the raw video data as video 1 14.
  • compressed video data stream 314 can be transmitted at a lower data rate compared to raw video data stream 222 of FIG. 2.
  • encoder 306 is present to compress the raw video data of video information 1 10 (typically in the form of a hardware-based encoder)
  • decoder 310 is present to decompress the compressed video data of compressed video data stream 314 (typically in the form of a hardware-based decoder), resulting in additional costly hardware to be present.
  • system 300 may be implemented less expensively than system 200 of FIG. 2, but the costs are still high enough to discourage mass production.
  • FIG. 4 shows a block diagram of a video streaming system 400.
  • System 400 is an example of system 100 of FIG. 1. As shown in FIG. 4, system 400 includes a video source device 402, display device 204, a first transceiver 406, a decoder 408, a second transceiver 412, antenna 218, and antenna 220. Furthermore, video source device 402 includes an encoder 414 and a driver 416. These elements of FIG. 4 are described as follows.
  • Transceiver 406 is a transmitter for video source device 402 for compressed video data.
  • transceiver 406 may be a WiFi or UWB transceiver.
  • encoder 414 compresses the raw video data of video information 1 10 to generate compressed video data.
  • Video source device 202 has a USB connector 404.
  • Transceiver 406 has a USB connector (not shown in FIG. 4) that mates with USB connector 404.
  • Encoder 414 provides the compressed video data through USB connector 404 as compressed video data to transceiver 406.
  • Transceiver 406 transmits the compressed video data from antenna 218 as compressed video data stream 420 according to the corresponding communication protocol (e.g., either WiFi or UWB).
  • the corresponding communication protocol e.g., either WiFi or UWB
  • Transceiver 412 is a receiver for display device 204 for compressed video data.
  • Transceiver 412 has a USB connector 410 that mates with a USB connector (not shown in FIG. 4) of decoder 408.
  • Display device 204 has HDMI connector 212.
  • Decoder 408 has an ITDMI connector (not shown in FIG. 4) that mates with ITDMI connector 212.
  • Transceiver 412 receives compressed video data stream 420, and the received compressed video data of compressed video data stream 420 is transmitted through USB connector 410 to decoder 408.
  • Decoder 408 decompresses the compressed video data to raw video data, and transmits the raw video data to display device 204 through ITDMI connector 212.
  • Display device 204 displays the raw video data as video 114.
  • System 400 of FIG. 4 uses software encoding by encoder 414 to capture a full display screen of video source device 402 (e.g., including video 114 and background graphics), for display at display device 204. Furthermore, hardware decoding by decoder 408 is performed at the display device end in FIG. 4 to enable HDMI connectivity between video source device 402 and display device 204. As such, system 400 may be implemented less expensively than system 200 of FIG. 2 and system 300 of FIG. 3, but the costs are still high enough to discourage mass production (e.g., due to the hardware- based decoder 408).
  • the WiDi type of WiFi is an example of the communication technique used in system 400.
  • some UWB techniques may be used that use software encoding at the video source device and use hardware decoding at the display device to achieve HDMI connectivity. These two techniques result in similarly expensive cost structures.
  • HDMI connectivity is a contributing factor toward wireless video streaming techniques having high costs.
  • Systems 200 and 300 both have HDMI connectivity at both ends, while system 400 has HDMI connectivity at the display device end.
  • Embodiments are described in the following sections for wireless video streaming that do not use HDMI connectivity at the video source device and at the display device, resulting in systems that are low enough in cost to enable mass adoption.
  • FIG. 5 shows a block diagram of a video streaming system 500, according to an example embodiment.
  • system 500 includes a video source device 502, a display device 504, a first communication module 508, a second communication module 512, a first antenna 514, and a second antenna 530.
  • Video source device 502 includes a driver 516
  • display device 504 includes a decoder 520 and a display screen 522.
  • video source device 502 is a video source device enabled to stream video, such as HD video, according to the pull mode (where display device 504 requests the video content from video source device 502).
  • Display device 504 is an example video sink device configured to display the video streamed by video source device 502.
  • FIG. 6 shows a flowchart 600 for streaming video from a video source device, according to an example embodiment.
  • video source device 502 of FIG. 5 may operate according to flowchart 600.
  • FIG. 7 shows a flowchart 700 for receiving streamed video at a display device, according to an example embodiment.
  • display device 504 of FIG 5 may operate according to flowchart 700.
  • step 602 video data is received from local storage or a remote source.
  • driver 516 may receive video data 518.
  • Video data 518 may be compressed video data that is stored in compressed form, or that may be compressed in real time as provided to driver 516.
  • Video data 518 may be formatted according to one or more different video compression formats mentioned elsewhere herein or otherwise known.
  • Video data 518 may be received by driver 516 from various sources, such as from storage located in or associated with video source device 502, or from a real time source such as a video camera included in video source device 502 (e.g., mounted to a housing of video source device 502, attached to video source device 502, etc.) or a remote source (e.g., that streams compressed video data 302 to video source device 502 in a wired and/or wireless manner) such as the Internet, a cable head-end, or a wireless network of a wireless operator.
  • Table 1 indicates examples of video compression standards for video data 518 and corresponding examples of storage media of storage of video source device 502 in which video files containing video data 518 may be stored:
  • step 604 the video data is transmitted through a USB connector to be wirelessly transmitted from the video source device by a wireless communication module connected to the video source device at the USB connector.
  • driver 516 generates a compressed video data stream 524.
  • Compressed video data stream 524 includes video data 518 converted into the form of a video data stream.
  • video source device 502 includes a USB connector 506.
  • USB connector 506 is a connector configured according to the USB specification.
  • Video source device 502 is capable of transmitting USB signals (e.g., serial USB signals over D+ and D- pins) from video source device 502 to devices coupled to USB connector 506.
  • compressed video data stream 524 may be transmitted through USB connector 506 to be received by communication module 508.
  • Communication module 508 has a USB connector (not shown in FIG. 5) that mates with USB connector 506 so that compressed video data stream 524 may be provided to communication module 508.
  • Communication module 508 is configured to wirelessly transmit compressed video data stream 524 from video source device 502 as wireless video data stream 528.
  • step 702 video data is received through a USB connector, the video data having been wirelessly received at the display device from a video source device.
  • communication module 512 is a receiver (and optionally a transmitter) for display device 504.
  • Communication module 512 has a USB connector (not shown in FIG. 5) that mates with a USB connector 510 of display device 504.
  • Communication module 512 receives wireless video data stream 528, and the received compressed video data of wireless video data stream 528 is transmitted through USB connector 510 to decoder 520.
  • step 704 the received video data is decoded for display.
  • decoder 520 decompresses the compressed video data received in wireless video data stream 528 to decoded video data 532, and transmits decoded video data 532 to display screen 522.
  • Display screen 522 generates a display of video 114 based on the video data of decoded video data 532.
  • video data is wirelessly streamed from video source device 502 in a compressed form without using any UDMI connectivity.
  • wireless video data stream 528 contains compressed data that can be transmitted at a lower data rate, and does not use an external hardware decoder or encoder.
  • decoder 520 which is internal to display device 504, may be used to decode/decompress the received compressed video data.
  • Decoder 520 may be a type of decoder that is integrated in many commercially available display devices, typically implemented in software, relieving the need for an additional external decoder. Decoder 520 may be configured/modified to perform according to the embodiments described herein
  • Display device 504 may be any type of display device or device with a display, including a high definition (HD) display device that includes display screen 522 as an LCD (liquid crystal display) display or monitor, a plasma display or monitor, an LED (light emitting diode) display or monitor, an HD projector, or further type of display screen.
  • a high definition (HD) display device that includes display screen 522 as an LCD (liquid crystal display) display or monitor, a plasma display or monitor, an LED (light emitting diode) display or monitor, an HD projector, or further type of display screen.
  • Video source device 502 may be any type of stationary or mobile video source device, including a desktop computer, a set top box, a digital video disc (DVD) player, a video game console, a smart phone, a handheld computing device (e.g., a personal digital assistant (PDA), a BLACKBERRY device, a PALM device, etc.), a laptop computer, a tablet computer (e.g., an APPLE IP AD), or further type of video source device.
  • a desktop computer e.g., a set top box, a digital video disc (DVD) player, a video game console, a smart phone, a handheld computing device (e.g., a personal digital assistant (PDA), a BLACKBERRY device, a PALM device, etc.), a laptop computer, a tablet computer (e.g., an APPLE IP AD), or further type of video source device.
  • PDA personal digital assistant
  • PALM PALM device
  • APPLE IP AD APPLE IP AD
  • Video source device 502 may include a camera used to capture images, such as still images and video images that are streamed from the video source device in compressed form.
  • Video source device 502 may include storage that stores compressed video data.
  • video source devices may receive video data from external sources, such as from a cable television provider, a satellite television provider, an Internet, a wireless network of a wireless operator, or other network-based web site, or other remote source.
  • First and second communication modules 508 and 512 may be configured to enable wireless communications for video source device 502 and display device 504, respectively.
  • communication modules 508 and 512 may each include a frequency up-converter (transmitter) and down-converter (receiver).
  • Communication modules 508 and 512 may each enable communication using one or more proprietary or commercially available wireless communication protocols/technologies, including wireless USB, WiFi, UWB, and/or other communication protocols/technologies.
  • Communication modules 508 and 512 each include a USB connector.
  • First communication module 508 receives video data in USB file form over a USB port that interfaces with USB connector 506 of video source device 502, and wirelessly transmits a video data stream over a second port (according to the applicable wireless communication protocol/technology).
  • Second communication module 512 wirelessly receives the video data stream over a second port, and outputs video data in USB file form over a USB port that interfaces with USB connector 516 of display device 504.
  • wireless video data stream 528 may be generated and transmitted by communication module 508 in any video data stream form, proprietary or commercially available.
  • wireless video data stream 528 may be generated in elementary stream (ES) format, defined by an MPEG communication protocol.
  • ES elementary stream
  • An elementary stream is often referred to as “elementary”, “data”, “audio”, or “video” bitstreams or streams.
  • the format of the elementary stream depends upon the codec or data carried in the stream, but will often carry a common header when packetized into a packetized elementary stream.
  • Packetized elementary stream is a specification in the MPEG-2 Part 1 (Systems) (ISO IEC 13818-1) and ITU-T H.222.0 that defines carrying of elementary streams in packets within an MPEG program stream and MPEG transport stream.
  • the elementary stream is packetized by encapsulating sequential data bytes from the elementary stream inside PES packet headers.
  • wireless video data stream 528 may be generated in transport stream (TS) format, a standard format for transmission and storage of audio, video, and data.
  • Transport stream format is used in broadcast systems such as DVB (digital video broadcasting) and ATSC (Advanced Television Systems Committee).
  • Transport stream specifies a container format encapsulating packetized elementary streams, with error correction and stream synchronization features for maintaining transmission integrity when the signal is degraded.
  • wireless video data stream 528 may be generated according to other video data stream format.
  • System 500 including source video device 502 and display device 504, may be implemented in various ways to wirelessly stream video data.
  • system 500 may be implemented to wirelessly stream video data according to the pull mode, where display device 504 generates a video content request 526 that is transmitted to video source device 502, and defines the video content to be provided to display device 504.
  • display device 504 controls the transmitting of video content to display device 504.
  • system 500 may be implemented to wirelessly stream video data according to the push mode, where video source device 502 transmits (e.g., broadcasts) the video content without display device 504 being involved in controlling the transmitting of the video content.
  • Video source device 502 and display device 504 of system 500 may be configured in various ways to enable the streaming of video data according to the pull mode, where display device 504 controls the streaming of video data by video source device 502.
  • FIG. 8 shows a block diagram of a video streaming system 800, according to an example embodiment.
  • Video streaming system 800 is an example of video streaming system 500 of FIG. 5.
  • system 800 includes video source device 502, display device 504, first communication module 508, second communication module 512, first antenna 514, and second antenna 530.
  • Video source device 502 includes driver 516 and a virtual disk 802, and display device 504 includes decoder 520 and display screen 522.
  • the features of system 800 are described as follows.
  • video data 518 is stored in the form of a file in storage of video source device 502 that is accessible in a virtual disk 802 maintained at video source device 502.
  • Virtual disk 802 is configured to mimic the behavior of a USB mass storage device (e.g., a "U-disk", etc.) that may be accessed by display device 504.
  • communication modules 508 and 512 may relay a command or control message (e.g., request 526) from display device 504 to video source device 502 to initiate delivery of video data 518 from virtual disk 802 to display device 504.
  • Driver 516 interprets the command and controls the delivery of video data 518 to display device 504 through communication modules 508 and 512 in response to the command.
  • video decoder 520 reads video data 518 as a video file from virtual disk 802 through the wireless link provided by communication modules 508 and 512.
  • video decoder 520 is a video decoder that includes USB connectivity and functionality, and therefore can support this reading of the video file from virtual disk 802 emulating a USB mass storage device. Because display device 504 (e.g., decoder 520) controls this video transfer process, the video data streaming of FIG. 8 is performed according to the pull mode.
  • Video source device 502 may be configured in various ways to enable video content to be retrieved from virtual disk 802 according to the pull mode.
  • FIG. 9 shows a block diagram of a video source device 900, according to an example embodiment.
  • Video source device 900 is an example of video source device 502 of FIG. 8.
  • video source device 900 includes an application 902, driver 516, virtual disk 802, and a storage device 904.
  • Application 902 includes a graphical user interface 906 and an application storage manager 908.
  • Driver 516 includes a USB mass storage module 910 and a driver storage manager 912.
  • Virtual disk 802 includes a file allocation table (FAT) 914 and a mapping table 916.
  • Video source device 900 is configured to store video data in a virtual storage to be accessible over a USB (universal serial bus) link as a USB file stored a USB mass storage device.
  • Video source device 900 is described as follows.
  • application 902 is an application supported by video source device 900.
  • Application 902 may be implemented in computer code that executes in one or more processors, or in any other form or combination of hardware, software, and/or firmware.
  • Application 902 is configured to enable a user of device 900 to select one or more files stored in storage device 904 to be mapped into virtual disk 802.
  • application storage manager 908 of application 902 may access storage device 904 to determine the one or more files stored in storage device 904.
  • application 902 may generate GUI 906, which may enable the user to select the one or more files determined to be stored in storage device 904 (e.g., by displaying the files in as list, a file/directory navigator, or in other manner).
  • GUI 906 may then enable the user to submit the selected file(s), such as video file 918, to be mapped to virtual disk 802.
  • GUI 906 may have any form, including one or more windows and user interface elements (e.g., displayed menus, buttons, text entry blanks, etc.) that may be interacted with by a user interface of device 900 (e.g., a keyboard, a mouse, a touch screen, a roller ball, a thumb wheel, etc.).
  • Application storage manager 908 of application 902 may read physical storage information (e.g., file name, file size, file location in storage device 904, etc.) associated with the selected files, and may transmit the physical storage information to driver storage manager 912 of driver 516.
  • Driver storage manager 912 generates FAT 914 to include at least some of the received physical storage information.
  • FAT 914 is a dummy file allocation table that simulates a real file allocation table, and may contain one or more dummy entries associated with dummy data clusters indicating virtual storage locations for the selected files. For instance, driver storage manager 912 may generate FAT 914 to indicate the virtual locations in virtual disk 802 that the selected files, including video file 918, are stored.
  • FAT 914 may at least identify each of the selected files and their locations in virtual disk 802.
  • Driver storage manager 912 may additionally generate mapping table 916 to indicate a mapping of the physical storage location in storage device 904 to the virtual storage location in virtual disk 802 for each of the selected files.
  • mapping table 916 may indicate the physical storage location for video file 918 in storage device 904 and the virtual storage location for video file 918 in virtual disk 802.
  • FAT 14 and mapping table 916 a physical storage location in storage device 904 for video file 918 can be determined when the virtual location of video file 918 in virtual disk 802 is provided during a virtual disk operation.
  • Storage device 904 may include any suitable type of storage in the form of one or more storage devices, including one or more primary or main memory devices (e.g., random access memory (RAM)), one or more secondary storage devices (e.g., a hard disk drive, a removable storage device or drive, a memory card, a memory stick, a floppy disk drive, a magnetic tape drive, a compact disc (CD) drive for CDs, a digital video disc (DVD) drive for DVDs, an optical storage device), and/or other types of storage.
  • FIG. 10 shows a block diagram of video source device 900 of FIG. 9 streaming video data of video file 904 to a requesting display device, according to an example embodiment. As shown in FIG.
  • communication module 508 connected to video source device 900 may transmit stored file information 1002 to display device 504 of FIG. 8.
  • Stored file information 1002 may be transmitted to display device 504 in response to a request by display device 504 (e.g., a request by a user to determine available video content), may be transmitted automatically, or in other manner.
  • Stored file information 1002 may include FAT 914, which indicates virtual storage locations in virtual disk 802 for available video files for display.
  • Display device 504 may enable a user to select a video file from FAT 914 for display.
  • display device 504 of FIG. 8 may provide a user interface, such as a graphical user interface (e.g., similar to GUI 906 in FIG.
  • Display device 504 may transmit video content request 526 to request the selected video file.
  • decoder 520 of display device 504 may have a USB interface that enables display device 504 to generate request 526, and to transmit request 526 for the video file as if a USB mass storage device connected to USB connector 510 is being accessed for the video file.
  • communication module 512 which is connected to USB connector 510, transmits request 526 from display device 504.
  • Video source device 900 may receive video content request 526 transmitted from display device 504 of FIG. 8, which indicates the selected video file to be provided to display device 504.
  • request 526 may include the virtual file location from FAT 914 for the selected video file, such as the virtual storage location for video file 918.
  • communication module 508 may receive video content request 526 at antenna 526, and may provide file request information 1004 to USB mass storage module 910 of driver 516 (through USB connector 506).
  • File request information 1004 indicates the virtual storage location for video file 918 in virtual disk 802 provided in request 526.
  • USB mass storage module 910 is configured to respond to USB transactions with the USB interface of decoder 520 of display device 504.
  • USB interface of decoder 520 may provide request 526 in the form of a USB CBW (command block wrapper), as would be known to persons skilled in the relevant art(s).
  • USB mass storage module 910 provides the virtual storage location to driver storage manager 912.
  • Driver storage manager 912 accesses mapping table 916 to map the received virtual storage location to the physical storage location in storage device 904 for video file 918.
  • driver storage manager 912 accesses storage device 904 for video file 918 at the mapped physical storage location.
  • USB mass storage module 910 transmits video file 918 to communication module 508 (through USB connector 506) as requested video file 1006, and communication module 508 transmits video file 918 as wireless video data stream 528.
  • wireless video data stream 528 is received at communication module 512, and video file 918 of wireless video data stream 528 is transmitted through USB connector 510 to decoder 520.
  • Decoder 520 decompresses the compressed video data received in wireless video data stream 528 to decoded video data 532, and transmits decoded video data 532 to display screen 522.
  • Display screen 522 generates a display of video 114 based on the video data of decoded video data 532.
  • FIG. 11 shows a flowchart 1100 for pull mode streaming of video from a video source device to a display device, according to an example embodiment.
  • video source device 900 of FIGS. 9 and 10 may operate according to flowchart 1 100. Further structural and operational embodiments will be apparent to persons skilled in the relevant art(s) based on the following description of flowchart 1 100.
  • Flowchart 1100 begins with step 1102.
  • a file allocation table (FAT) is provided that is wirelessly transmitted to a display device, the FAT indicating the virtual storage location for the USB file in the virtual USB mass storage device.
  • FAT 914 may be generated to map virtual storage locations in virtual disk 802 to physical storage locations in storage 904 for video files such as video file 918.
  • USB mass storage module 910 may provide FAT 914 to communication module 508 (through USB connector 506) connected to video source device 900. As shown in FIG. 10, communication module 508 may transmit stored file information 1002, which contains FAT 914, to display device 504 of FIG. 8.
  • a request for a USB file is received that contains video data, the request being wirelessly received at the video source device.
  • video content request 526 may be wirelessly received by communication module 508 from display device 504 (e.g., from a USB interface of decoder 520), and may be handled by USB mass storage module 910.
  • Request 526 indicates a request for a video file, such as video file 918.
  • Video file 918 is referred to as a USB file because USB mass storage module 910 emulates a USB mass storage device connected to display device 504 at USB connector 510 on which video file 918 is stored.
  • step 1106 a mapping for the requested USB file from the virtual file storage location to a storage location in the storage device is determined.
  • driver storage manager 912 may access mapping table 916, which maps the virtual file storage location in virtual disk 802 for video file 918 to the physical storage location in storage device 904.
  • step 1108 the requested video data is retrieved from the storage location in the storage device. For instance, as described above, video file 918 is retrieved from storage device 904 at the physical storage location indicated by the mapping.
  • step 11 10 the requested video data is provided to be wirelessly transmitted in response to the request.
  • USB mass storage module 910 provides video file 918 to communication module 508, and communication module 508 wirelessly transmits video file 918 in wireless video data stream 528 in response to request 526.
  • Video source device 502 and display device 504 of system 500 may be configured in various ways to enable the streaming of video data according to the push mode.
  • a video stream transferred to a display device is not provided as a result of file reading, such as in the pull mode described above.
  • a video transfer occurs similarly to a broadcast video program.
  • the video could be transmitted from real time compression of graphical/audio output of a video source device (e.g., a laptop computer), a real time Internet video download, or a stream formed by the real time parsing of a stored file.
  • This type of video data streaming may be referred to as push mode, because the video source device is controlling the video transfer, and the video is provided in the form of a stream.
  • the decoder of the display device may actually decode a broadcast type of video program.
  • Video decoders in display devices typically have at least two data paths (or
  • a first data path is referred to as a "stream data path” that is received through one or more transport stream (TS) ports of the decoder
  • a second data path is referred to as a "file data path” that is received through one or more USB ports of the decoder.
  • a push video stream is received through the transport stream (TS) ports instead of the USB ports of the decoder, due to the nature of the video stream being a stream of data packets formed according to the MPEG TS standard format or other TS format (or ES - elementary stream - format, which is typically the output of a video decoder).
  • most wireless communication modules e.g., WiFi, etc.
  • USB ports are widely integrated with wireless communication modules.
  • a video decoder is formed that is configured to enable
  • USB connectivity for received video data push stream.
  • the firmware of a video decoder may be modified, so that a video data stream delivered to the USB port of the video decoder can be redirected to the stream data path of the decoder.
  • FIG. 12A shows a block diagram of a video decoder 1200, according to an example embodiment.
  • Video decoder 1200 is an example of decoder 520 of FIG. 5.
  • decoder 1200 includes a stream interface (or port) 1202, a stream data path processing logic 1204, a USB interface (or port) 1206, a file data path processing logic 1208, a decoding engine 1210, and a data diverter 1212.
  • Decoder 1200 is described as follows.
  • stream interface 1202 of decoder 1200 receives a first video data stream 1214 in TS (or ES) format.
  • First video data stream 1214 may be received from a remote source or from storage associated with display device 504.
  • Stream interface 1202 provides first video data stream 1214 as first video data stream 1218 to stream data path processing logic 1204.
  • Stream data path processing logic 1204 is configured to process the stream data of first video data stream 1218 to generate first processed video data 1222.
  • Decoding engine 1210 receives and decodes first processed video data 1222 for display, generating decoded video data 1226.
  • decoded video data 1226 may be provided by decoding engine 1210 to display screen 522 as decoded video data 532, as shown in FIG. 5.
  • USB interface 1206 of decoder 1200 receives a second video data stream 1216
  • Second video data stream 1216 is received in file format (e.g., USB file format), and typically is received from a storage device (e.g., a USB mass storage device) connected to display device 504.
  • USB interface 1206 provides second video data stream 1216 as second video data stream 1220 to file data path processing logic 1208.
  • File data path processing logic 1208 is configured to process USB file data of second video data stream 1216 to generate second processed video data 1224.
  • Decoding engine 1210 receives and decodes second processed video data 1224 for display, generating decoded video data 1226. For instance, decoded video data 1226 may be provided by decoding engine 1210 to display screen 522 as decoded video data 532, as shown in FIG. 5.
  • decoder 1200 may receive TS or PS (program stream) formatted video data streams (e.g., formatted according to an ES (elementary stream) format, or other format) at stream interface 1202 for processing by stream data path processing logic 1204 and decoding by decoding engine 1210, and may receive USB file formatted video data streams at USB interface 1206 for processing by file data path processing logic 1208 and decoding by decoding engine 1210.
  • data diverter 1212 may be present in decoder 1200.
  • Data diverter 1212 enables stream data (e.g., TS or PS video data streams) received at USB interface 1206 to be processed by stream data path processing logic 1204 and decoded by decoding engine 1210.
  • display device 504 can receive a wireless video data stream at communication module 512 coupled to USB connector 510 that contains stream data, and the stream data can be processed by stream data path processing logic 1204 and decoded by decoding engine 1210 by data diverter 1212 diverting the stream data from USB interface 1206 to stream data path processing logic 1204 (rather than to file data path processing logic 1208).
  • decoder 1200 may operate according to FIG. 13.
  • FIG. 13 shows a flowchart 1300 for rerouting stream data in a decoder, according to an example embodiment. Further structural and operational embodiments will be apparent to persons skilled in the relevant art(s) based on the following description of flowchart 1300.
  • Flowchart 1300 begins with step 1302.
  • the received video data is provided to the file data path processing logic if the video data received through the USB connector is received in the form of a USB file.
  • data diverter 1212 receives second video data stream 1220. If data diverter 1212 determines that second video data stream 1220 from USB interface 1206 includes file data (e.g., USB data of a USB file), data diverter 1212 does not divert second video data stream 1220. Second video data stream 1220 is allowed to be received by file data path processing logic 1208 for processing (and decoding by decoding engine 1210), as described above.
  • Data diverter 1212 may determine that second video data stream 1220 contains file data in any way, including detecting predetermined file header information or other file field information (e.g., USB file field information) in second video data stream 1220, by detecting an absence of stream information, or in other manner, as would be known to persons skilled in the relevant art(s).
  • predetermined file header information or other file field information e.g., USB file field information
  • step 1304 the received video data is routed to the stream data path processing logic if the video data received through the USB connector is stream data.
  • data diverter 1212 receives second video data stream 1220. If data diverter 1212 determines that second video data stream 1220 from USB interface 1206 includes stream data (e.g., TS or ES video data), data diverter 1212 routes or diverts second video data stream 1220 to stream data path processing logic 1204 for processing (and decoding by decoding engine 1210). In such case, second video data stream 1220 is not processed by file data path processing logic 1208.
  • stream data e.g., TS or ES video data
  • data diverter 1212 may provide a disable signal to file data path processing logic 1208, may prevent second video data stream 1220 from reaching file data path processing logic 1208 (e.g., by performing a de-multiplexing or controlled switch function), or otherwise prevent file data path processing logic 1208 from receiving and/or processing second video data stream 1220.
  • Data diverter 1212 may determine that second video data stream 1220 contains stream data in any way, including detecting predetermined stream header information or other stream field information (e.g., MPEG-2 TS field information) in second video data stream 1220, by detecting an absence of file information, or in other manner, as would be known to persons skilled in the relevant art(s).
  • stream-formatted data may be streamed to a display device over a
  • USB link that typically handles file-formatted data
  • the stream-formatted data may be processed as stream data within the video decoder of the display device by being diverted from the USB data path to the stream data path.
  • FIG. 12B shows a block diagram of a video decoder 1230, according to an example embodiment.
  • Video decoder 1230 is an example of decoder 520 of FIG. 5, and is generally similar to decoder 1200 of FIG. 12, with differences described as follows. As shown in FIG.
  • decoder 1230 includes stream interface (or port) 1202, stream data path processing logic 1204, USB interface (or port) 1206, file data path processing logic 1208, decoding engine 1210, data diverter 1212, and a third data path processing logic 1232.
  • Decoder 1230 is described as follows. [0091] As shown in FIG. 12B, stream interface 1202 of decoder 1230 receives first video data stream 1214 in TS (or ES) format. Similarly to the description provided above, stream interface 1202 provides first video data stream 1214 as first video data stream 1218 to stream data path processing logic 1204. Stream data path processing logic 1204 is configured to process the stream data of first video data stream 1218 to generate first processed video data 1222.
  • Decoding engine 1210 receives and decodes first processed video data 1222 for display, generating decoded video data 1226.
  • USB interface 1206 of decoder 1200 receives second video data stream 1216, which is wirelessly received at display device 504 (e.g., by communication module 512).
  • second video data stream 1216 is received in file format (e.g., USB file format), and typically is received from a storage device (e.g., a USB mass storage device) connected to display device 504.
  • second video data stream 1216 is received in a streaming data format.
  • second video data stream 1216 may include video data in transport stream (TS) format, program stream (PS) format, or other streaming data format mentioned elsewhere herein or otherwise known.
  • USB interface 1206 provides second video data stream 1216 as second video data stream 1220 to data diverter 1212.
  • Data diverter 1212 enables USB disk file data of second video data stream 1216 received at USB interface 1206 to be processed by file data path processing logic 1208 and decoded by decoding engine 1210, or enables stream data (e.g., TS or PS video data streams) received at USB interface 1206 to be processed by third data path processing logic 1232 and decoded by decoding engine 1210. As shown in FIG. 12B, data diverter 1212 receives second video data stream 1220.
  • data diverter 1212 determines that second video data stream 1220 from USB interface 1206 includes file data (e.g., USB data of a USB disk file), data diverter 1212 routes or diverts second video data stream 1220 to file data path processing logic 1208. In such case, second video data stream 1220 is received by file data path processing logic 1208 for processing (and decoding by decoding engine 1210), as described above. If data diverter 1212 determines that second video data stream 1220 from USB interface 1206 includes stream data (e.g., TS or ES video data), data diverter 1212 routes or diverts second video data stream 1220 to third data path processing logic 1232 for processing (and decoding by decoding engine 1210).
  • file data e.g., USB data of a USB disk file
  • stream data e.g., TS or ES video data
  • file data path processing logic 1208 is configured to process
  • Decoding engine 1210 receives and decodes second processed video data 1224 for display, generating decoded video data 1226.
  • Third data path processing logic 1232 is configured to process the stream data of second video data stream 1220, when received from data diverter 1212, to generate third processed video data 1234.
  • Decoding engine 1210 receives and decodes third processed video data 1234 for display, generating decoded video data 1226.
  • decoded video data 1226 may be provided by decoding engine 1210 to display screen 522 as decoded video data 532, as shown in FIG. 5.
  • stream data path processing logic 1204, file data path processing logic 1208, and third data path processing logic 1232 may be entirely separate from each other, or any two or more of them may be overlapping (e.g., may share logic).
  • display device 504 can receive a wireless video data stream at communication module 512 coupled to USB connector 510 that contains stream data, and the stream data can be processed by third data path processing logic 1232 and decoded by decoding engine 1210 by data diverter 1212 routing the stream data from USB interface 1206 to third data path processing logic 1232 (rather than to file data path processing logic 1208).
  • third data path processing logic 1232 may include logic for processing stream-formatted data similarly to logic of stream data path processing logic 1204, but provides a data path that is separate from stream data path processing logic 1204.
  • FIG. 14 shows a block diagram of a video streaming system 1400, according to an example embodiment.
  • Video streaming system 1400 is an example of video streaming system 500 of FIG. 5.
  • system 1400 includes video source device 502, display device 504, first communication module 508, second communication module 512, first antenna 514, and second antenna 530.
  • Video source device 502 includes driver 516
  • display device 504 includes decoder 520 and display screen 522.
  • the features of system 1400 are described as follows.
  • System 1400 operates according to the push mode because a video source device controls the video data transfer.
  • a video data stream 1402 is received by video source device 502.
  • Video data stream 1402 is streamed from another video source device (not shown in FIG. 14), in a wired and/or wireless manner, such as an Internet server, a cable head-end, another computer, etc.
  • driver 516 of video source device 502 forwards video data stream 1402 as video data stream 1404 to communication module 508 (through USB connector 506).
  • driver 516 performs a step 1502 shown in FIG. 15.
  • a received video data stream is provided to be wirelessly transmitted from the video source device.
  • driver 516 forwards video data stream 1402 as video data stream 1404 to communication module 508 to be wirelessly transmitted from video source device 502.
  • Communication module 508 transmits video data stream 1404 as wireless video data stream 1406 from antenna 514.
  • Communication module 512 receives wireless video data stream 1406 at antenna 530, and provides video stream 1404 from wireless video data stream 1406 to decoder 520 (through USB connector 510).
  • video data stream 1402 is provided by video source device 502 to display device 504 using USB connectivity, and in an embodiment, data diverter 1212 of FIG. 12A may be used to divert the stream data of video data stream 1402 from the file data path (file data path processing logic 1208) to the stream data path (stream data path processing logic 1204).
  • FIG. 16 shows a block diagram of a video streaming system 1600, according to another example embodiment.
  • Video streaming system 1600 is an example of video streaming system 500 of FIG. 5.
  • system 1600 includes video source device 502, display device 504, first communication module 508, second communication module 512, first antenna 514, and second antenna 530.
  • Video source device 502 includes driver 516 and virtual disk 802, and display device 504 includes decoder 520 and display screen 522.
  • the features of system 1600 are described as follows.
  • System 1600 operates according to the push mode because a video source device controls the video data transfer.
  • data diverter 1212 of FIG. 12A need not be present in decoder 520.
  • a virtual file 1602 in virtual disk 802 is configured to hold the real time push stream data of video data stream 1402 as it is received at video source device 502 (similarly to the pull mode embodiment described above with respect to FIGS. 8-11).
  • a pre-generated FAT (similar to FAT 918 of FIG.
  • video data stream 1402 may be generated for virtual file 1602, and provided to display device 504, such that decoder 520 may operate as if decoder 520 is reading a video file from a U-disk or other USB mass storage device, while video data stream 1402 is written into the virtual file based on the pre-generated FAT on a real time basis.
  • Video data stream 1402 may be written to physical memory of video source device 502 for temporary storage.
  • the pre- generated FAT may mapped to the physical memory in any suitable manner.
  • video data stream 1402 may be streamed from another video source device (not shown in FIG. 16), in a wired and/or wireless manner, such as an Internet server, a cable head-end, another computer, etc.
  • video data stream 1402 is stored in physical memory in stream form, and the physical memory location is mapped to a virtual storage location for virtual file 1602 in virtual disk 802.
  • a pre- generated FAT may be transmitted to decoder 520 of display device 504 that indicates the virtual storage location for virtual file 1602.
  • Decoder 520 may transmit request 526, as described above, to request the video file corresponding to virtual file 1602.
  • Driver 516 of video source device 502 forwards video data stream 1402 as video data stream 1604 to communication module 508 (through USB connector 506).
  • Video data stream 1604 is generated in file stream format (e.g., USB file format), such as by a USB mass storage module of driver 516 (e.g., USB mass storage module 910 of FIGS. 9 and 10) so that decoder 520 may request and receive virtual file 1602 using its USB interface (e.g., USB interface 1206 of FIG. 12A).
  • Communication module 508 transmits video data stream 1604 as wireless video data stream 1606 from antenna 514.
  • Communication module 512 receives wireless video data stream 1606 at antenna 530, and provides the video data of wireless video data stream 1606 to decoder 520 (through USB connector 510).
  • Decoder 520 recognizes the video data stream of wireless video data stream 1606 as corresponding to the virtual storage location of virtual file 1602 provided in the pre-generated FAT.
  • video data stream 1402 is provided by video source device 502 to display device 504 using USB connectivity, and in an embodiment (referring to FIG. 12 A), data diverter 1212 is not needed to divert the file stream data of video data stream 1602 from the file data path of the decoder because file data path processing logic 1208 can process the file stream video data.
  • driver 516 performs a flowchart 1700 shown in FIG. 17.
  • a received video data stream is stored in a virtual file in a virtual file storage location.
  • video data stream 1402 is written into virtual file 1402 in a location indicated by a pre-generated FAT on a real time basis.
  • the received video data stream is provided to be transmitted wirelessly from the video source device at least partially simultaneously with storing the received video stream in the virtual file.
  • driver 516 forwards video data stream 1402 as video data stream 1604 to communication module 508 to be wirelessly transmitted from video source device 502. This may occur partially overlapping in time with video data stream 1402 being stored in virtual file 1602.
  • video data stream 1402 is not written into virtual file 1602. Instead, video data stream 1402 is forwarded to decoder 520 at display device 504 as it is received by driver 516 upon request 526 being received for file content.
  • a FAT may be generated and provided to display device 504 that indicates a virtual file location for a virtual file, although the FAT is provided solely for a dummy purpose since the virtual file does not exist.
  • Driver 516, decoder 520, application 906, application storage manager 908, USB mass storage module 910, driver storage manager 912, decoder 1200, stream interface 1202, stream data path processing logic 1204, USB interface 1206, file data path processing logic 1208, decoding engine 1210, data diverter 1212, decoder 1230, and third data path processing logic 1232 may be implemented in hardware, software, firmware, or any combination thereof.
  • driver 516, decoder 520, application 906, application storage manager 908, USB mass storage module 910, driver storage manager 912, decoder 1200, stream interface 1202, stream data path processing logic 1204, USB interface 1206, file data path processing logic 1208, decoding engine 1210, data diverter 1212, decoder 1230, and third data path processing logic 1232 may be implemented as computer program modules or code configured to be executed in one or more processors.
  • driver 516, decoder 520, application 906, application storage manager 908, USB mass storage module 910, driver storage manager 912, decoder 1200, stream interface 1202, stream data path processing logic 1204, USB interface 1206, file data path processing logic 1208, decoding engine 1210, data diverter 1212, decoder 1230, and third data path processing logic 1232 may be implemented as hardware logic/electrical circuitry.
  • devices in which embodiments may be implemented may include storage, such as storage drives, memory devices, and further types of computer-readable media.
  • Examples of such computer-readable storage media include a hard disk, a removable magnetic disk, a removable optical disk, flash memory cards, digital video disks, random access memories (RAMs), read only memories (ROM), and the like.
  • computer program medium As used herein, the terms "computer program medium,” “computer-readable medium, “ and “computer storage medium” are used to generally refer to the hard disk associated with a hard disk drive, a removable magnetic disk, a removable optical disk (e.g., CDROMs, DVDs, etc.), zip disks, tapes, magnetic storage devices, MEMS (micro-electromechanical systems) storage, nanotechnology-based storage devices, as well as other media such as flash memory cards, digital video discs, RAM devices, ROM devices, and the like.
  • Such computer-readable storage media are distinguished from and non-overlapping with communication media (do not include communication media).
  • Communication media typically embodies computer-readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave.
  • modulated data signal means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal.
  • communication media includes wireless media such as acoustic, RF, infrared and other wireless media. Embodiments are also directed to such communication media.
  • Such computer-readable storage media may store program modules that include computer program logic for implementing the features of driver 516, decoder 520, application 906, application storage manager 908, USB mass storage module 910, driver storage manager 912, stream interface 1202, stream data path processing logic 1204, USB interface 1206, file data path processing logic 1208, decoding engine 1210, data diverter 1212, decoder 1230, third data path processing logic 1232, flowchart 600, flowchart 700, flowchart 1100, flowchart 1300, step 1502, and/or flowchart 1700 (including any step of flowcharts 600, 700, 1100, 1300, and 1700), and/or further embodiments of the present invention described herein.
  • Embodiments of the invention are directed to computer program products comprising such logic (e.g., in the form of program code or software) stored on any computer useable medium.
  • Such program code when executed in one or more processors, causes a device to operate as described herein.
  • Decoder 520 (such as decoder 1200, decoder 1230, or other embodiment of decoder 520) may be implemented in various ways, including multiple integrated circuit devices (e.g., chips), or a single integrated circuit device (e.g., a system-on-chip (SOC) implementation). Decoder 520 may include one or more of memory, processing logic (e.g., one or more processors, etc.), embedded code, etc.
  • processing logic e.g., one or more processors, etc.
  • embedded code etc.
  • stream data path processing logic 1204, file data path processing logic 1208, and third data path processing logic 1232 may be implemented as computer code that is executed by one or more processors, or may be implemented in other ways.
  • Devices in which stream data path processing logic 1204, file data path processing logic 1208, and/or third data path processing logic 1232 are implemented may include buffers/memory (e.g., one or more RAM devices, including dynamic RAM devices, or static RAM devices such as flash memory, etc.) that is used to buffer data processed by logic 1204, logic 1208, and/or logic 1232.
  • stream data path processing logic 1204, file data path processing logic 1208, and/or third data path processing logic 1232 may share some logic, or may each implement entirely separate logic.

Abstract

Methods, systems, and apparatuses are described for enabling the streaming of video data. In implementations, video source devices and are configured to wirelessly stream video data, and display devices are configured to receive the streamed video data, using standard communication interface technologies, such as USB (universal serial bus) interfaces.

Description

WIRELESS VIDEO STREAMING USING USB CONNECTIVITY OF HD DISPLAYS
BACKGROUND OF THE INVENTION
Field of the Invention
[0001] The present invention relates to wireless video data streaming technology.
Background Art
[0002] "Video" refers to a technology for electronically capturing, storing, and playing a sequence of still images that represent scenes in motion. Various devices exist that are capable of capturing, storing, and/or playing video. Different resolutions of video are available that provide different levels of viewing quality. High-definition video (e.g., "HD" video) refers to video of higher resolution (e.g., than standard-definition video). For example, HD video displays may have display resolutions of 1,280 by 720 pixels (720p), 1,920 by 1,080 pixels (1080i/1080p), or other resolutions. Televisions and other display devices are being manufactured with high definition resolutions in increasing numbers. As such, techniques for providing high definition video data to high definition displays are being developed, including wireless techniques.
[0003] The demand for wireless video streaming has been increasing in recent years.
Various wireless technologies have been tried for this purpose, including WiFi (according to wireless local area network (WLAN) devices) technology, UWB (ultra- wideband) technology, WiHD (WirelessHD) technology, WHDI (wireless home digital interface), etc. Wireless video streaming is desired between various types of video source devices, such as set top boxes, desktop computers, and laptops, and various types of display devices, including HD displays
[0004] Push mode and pull mode video streaming techniques have been used to transport video between electronic devices and displays. According to push mode streaming, a portable electronic device initiates and controls the transfer of the video to the display device. One example of push mode video streaming is the broadcast video service in cable or satellite networks. In such broadcast video services, a video stream is initiated and transferred by the video source device (e.g., a satellite or cable service). According to pull mode streaming, the display initiates and controls the transfer of the video from the portable electronic device. [0005] One hurdle in the adoption of wireless HD video streaming technology is that many current solutions use HDMI (High-Definition Multimedia Interface) to provide the connectivity between the video source and HD display device, which results in a high cost. The WiHD and WHDI interface solutions provide a replacement for HDMI, while some solutions based on WiFi and UWB use an external video decoder at the display device to provide HDMI connectivity. To directly replace HDMI, existing replacement technologies either wirelessly stream raw HD video at very high data rates (e.g., in the order of gigabits-per-second), such as in WiHD, or use video codecs (coder-decoders) to reduce the wireless streaming data rate, such as in WHDI. Both of these approaches have cost problems that make them prohibitive for mass production. Cost issues also exist for solutions based on WiFi (e.g., WiDi) or UWB with HDMI connectivity due to the external HD decoder used at the display device.
BRIEF SUMMARY OF THE INVENTION
[0006] Methods, systems, and apparatuses are described for enabling the streaming of video data. In implementations, video source devices and are configured to stream video data, and display devices are configured to receive the streamed video data, using standard communication interface technologies, such as USB (universal serial bus) interfaces.
[0007] For instance, in one implementation, a video source device is provided. The video source device includes a USB connector, a storage device, and a driver. The driver receives a request through the USB connector for a video file that contains video data. The request is wirelessly received at the video source device from a display device. The request indicates a virtual file storage location at the video source device for the video file. The virtual file storage location is a virtual location in a virtual USB mass storage device. The driver determines a mapping for the requested video file from the virtual file storage location to a storage location in the storage device. The driver retrieves the requested video file from the storage location in the storage device, and transmits the video data of the video file through the USB connector to be wirelessly transmitted in a video data stream in response to the request.
[0008] Video source devices implementations are also provided that wirelessly stream video data received by the video source device in real time.
[0009] In another implementation, a display device is provided. The display device includes a USB connector and a decoder. The decoder includes a USB interface and a decoding engine. The USB interface receives video data through the USB connector that is wirelessly received at the display device from a video source device. The decoding engine decodes the received video data for display.
[0010] In one implementation of the display device, the decoder further includes a data diverter, a stream data path processing logic, and a file data path processing logic. The stream data path processing logic is configured to process stream data to generate first processed video data that is received by the decoding engine. The file data path processing logic is configured to process file data to generate second processed video data that is received by the decoding engine. If the video data received through the USB connector is stream data, the data diverter routes the received video data to the stream data path processing logic. If the video data received through the USB connector is received in the form of file data, the received video data is provided to the file data path processing logic.
[0011] Computer program products are also described herein that enable streaming of video data, and that enable further embodiments as described herein. Devices in which embodiments may be implemented may include storage, such as storage drives, memory devices, and further types of computer-readable storage media described herein. Such computer-readable storage media may store program modules that include computer program logic for performing the systems, processes, and further embodiments described herein.
[0012] Further features and advantages of the disclosed technologies, as well as the structure and operation of various embodiments, are described in detail below with reference to the accompanying drawings. It is noted that the invention is not limited to the specific embodiments described herein. Such embodiments are presented herein for illustrative purposes only. Additional embodiments will be apparent to persons skilled in the relevant art(s) based on the teachings contained herein.
BRIEF DESCRIPTION OF THE DRAWINGS/FIGURES
[0013] The accompanying drawings, which are incorporated herein and form a part of the specification, illustrate the present invention and, together with the description, further serve to explain the principles of the invention and to enable a person skilled in the pertinent art to make and use the invention. [0014] FIG. 1 shows a block diagram of a video streaming system that includes a video source device and a display device, according to an example embodiment.
[0015] FIGS. 2-4 show respective block diagrams of example video streaming systems.
[0016] FIG. 5 shows a block diagram of a video streaming system, according to an example embodiment.
[0017] FIG. 6 shows a flowchart for wirelessly streaming video data from a video source device, according to an example embodiment.
[0018] FIG. 7 shows a flowchart for receiving streamed video at a display device, according to an example embodiment.
[0019] FIG. 8 shows a block diagram of a video streaming system, according to an example embodiment.
[0020] FIG. 9 shows a block diagram of a video source device configured to store video data in a virtual storage to be accessible over a USB (universal serial bus) link as a USB file, according to an example embodiment.
[0021] FIG. 10 shows a block diagram of the video source device of FIG. 9 providing the video data over the USB link, according to an example embodiment.
[0022] FIG. 11 shows a flowchart providing a process for streaming video data from virtual storage over a USB link, according to an example embodiment.
[0023] FIGS. 12A and 12B show block diagrams of a decoder, according to example embodiments.
[0024] FIG. 13 shows a flowchart providing a process in a decoder for rerouting a video data stream received at a USB interface of the decoder, according to an example embodiment.
[0025] FIG. 14 shows a block diagram of a video streaming system, according to an example embodiment.
[0026] FIG. 15 shows a process in a video source device for streaming a received video data stream, according to an example embodiment.
[0027] FIG. 16 shows a block diagram of a video streaming system, according to an example embodiment.
[0028] FIG. 17 shows a flowchart providing a process in a video source device for streaming a received video data stream from virtual storage, according to an example embodiment.
[0029] The present invention will now be described with reference to the accompanying drawings. In the drawings, like reference numbers indicate identical or functionally similar elements. Additionally, the left-most digit(s) of a reference number identifies the drawing in which the reference number first appears.
DETAILED DESCRIPTION OF THE INVENTION
I. Introduction
[0030] The present specification discloses one or more embodiments that incorporate the features of the invention. The disclosed embodiment(s) merely exemplify the invention. The scope of the invention is not limited to the disclosed embodiment(s). The invention is defined by the claims appended hereto.
[0031] References in the specification to "one embodiment," "an embodiment," "an example embodiment," etc., indicate that the embodiment described may include a particular feature, structure, or characteristic, but every embodiment may not necessarily include the particular feature, structure, or characteristic. Moreover, such phrases are not necessarily referring to the same embodiment. Further, when a particular feature, structure, or characteristic is described in connection with an embodiment, it is submitted that it is within the knowledge of one skilled in the art to effect such feature, structure, or characteristic in connection with other embodiments whether or not explicitly described.
[0032] Furthermore, it should be understood that spatial descriptions (e.g., "above,"
"below," "up," "left," "right," "down," "top," "bottom," "vertical," "horizontal," etc.) used herein are for purposes of illustration only, and that practical implementations of the structures described herein can be spatially arranged in any orientation or manner.
II. Example Embodiments
[0033] Embodiments of the present invention enable the wireless streaming of video, such as high definition (HD) video, from source devices. Such source devices may store the video (e.g., in the form of video data) locally (e.g., in local storage) or may stream the video from the source device simultaneously with the video being received in a stream from another source device. A display device may receive the streamed video, and may display the streamed video on a display screen.
[0034] For instance, FIG. 1 shows a block diagram of a video streaming system 100. As shown in FIG. 1, system 100 includes a video source device 102 and a display device 104. As shown in FIG. 1, video source device 102 includes an antenna 106 and video information 1 10. Video source device 102 may optionally include a display screen for display of video, a user interface, and/or other features. When present, the display screen may be any type of display screen, including an LCD display screen, an LED display screen, etc. Video source device 102 may store video information 110 in storage in the form of video files, may receive video information 110 from one or more sources external to video source device 102, and/or may have access to video information 1 10 in other ways. As shown in FIG. 1, video source device 102 generates a video data stream 1 16 based on video information 110. Video data stream 116 is streamed from antenna 106 as a stream of video data to display device 104. As shown in FIG. 1, display device 104 includes an antenna 108 and a display screen 1 12. Antenna 108 of display device 104 receives video data stream 1 16, and display device 104 displays video 1 14 on display screen 112 based on video data stream 112.
[0035] In some cases, video 114 may be simultaneously displayed on display screen 1 12 of display device 104 and on a display screen of video source device 102. In such case, display screen 110 may display video 114 imposed on background graphics shown on the display screen of video source 102. Such background graphics may be visible when video 114 is displayed by the display screen of video source device 102 in less than full screen, and may include one or more of a background color and/or pattern, one or more icons, one or more other applications being used (e.g., an Internet browser viewing a web page, a document open in a word processor, an email tool, etc.), etc., displayed on the display screen of video source device 102. Alternatively, video 114 may be displayed by display screen 112 of display device 104 without being simultaneously displayed at video source device 102, and/or without background graphics from the display screen of video source device 102.
[0036] Different conventional techniques exist for wirelessly streaming video from video source devices to display devices. Such techniques include WiFi (according to wireless local area network (WLAN) devices) technology, UWB (ultra-wideband) technology, WiHD (WirelessHD) technology, and WHDI (wireless home digital interface). Most of these techniques use HDMI (HDMI) connectivity for HDTV. HDMI is a compact audio/video interface for transmitting uncompressed digital data. HDMI is becoming a standard for transmitting HD video data for HDTV, enabling the transmission of numerous HD formats, such as 720p, 1080i, 1080p, etc. Because uncompressed data is transmitted according to HDMI, the use of HDMI leads to very high data transmission rates, including data rates in the order of gigabits-per-second (Gbps). Examples of wireless video streaming according to WiFi, UWB, WiHD, and WHDI are described as follows with respect to FIGS. 2-4.
[0037] For instance, FIG. 2 shows a block diagram of a video streaming system 200.
System 200 is an example of system 100 of FIG. 1. As shown in FIG. 2, system 200 includes a video source device 202, a display device 204, an HDMI receiver 208, a WiHD transmitter 210, an HDMI transmitter 214, a WiHD receiver 216, an antenna 218, and an antenna 220. Video source device 202 is an example of video source device 102, and display device 204 is an example of display device 104. These elements of FIG. 2 are described as follows.
[0038] HDMI receiver 208, WiHD transmitter 210, and antenna 218 form a transmitter for video source device 202 for raw video data. As shown in FIG. 2, video source device 202 has an HDMI connector 206. HDMI receiver 208 includes an HDMI connector (not shown in FIG. 2) that mates with HDMI connector 206. Video source device 202 provides video information 1 10 through HDMI connector 206 as raw video data to HDMI receiver 208. HDMI receiver 208 receives the raw video data, and WiHD transmitter 210 transmits the raw video data from antenna 218 as wireless raw video data stream 222 according to the WiHD communication protocol.
[0039] WiHD receiver 216, HDMI transmitter 214, and antenna 220 form a receiver for display device 204 for raw video data. As shown in FIG. 2, display device 204 has an HDMI connector 212. HDMI transmitter 214 includes an HDMI connector (not shown in FIG. 2) that mates with HDMI connector 212. WiHD receiver 216 receives raw video data stream 222 according to the WiHD communication protocol. HDMI transmitter 214 transmits the raw video data of raw video data stream 222 received by WiHD receiver 216 to display device 204 through HDMI connector 212. Display device 204 displays the raw video data as video 114.
[0040] Because raw video data is wirelessly streamed in system 200 of FIG. 2, raw video data stream 222 is transmitted at a very high data rate that is orders of magnitudes higher than the data rates of compressed data, such as video data streams transmitted according to the MPEG-2 (moving picture experts group) or H.264 standards. For example, a raw data rate for 1080P video data is more than 1 Gbps, while the data rate of an MPEG-2 video stream is typically about 20 megabits-per- second (Mbps). Due to the high data throughput requirement related to HDMI connectivity, system 200 has a high cost that discourages mass adoption in consumer markets. [0041] FIG. 3 shows a block diagram of a video streaming system 300. System 300 is an example of system 100 of FIG. 1. As shown in FIG. 3, system 300 includes video source device 202, display device 204, HDMI receiver 208, HDMI transmitter 214, antenna 218, antenna 220, a first WHDI communication module 302, and a second WHDI communication module 304. These elements of FIG. 3 are described as follows.
[0042] First WHDI communication module 302 is a transmitter for video source device
202 for compressed video data. As shown in FIG. 3, first WHDI communication module 302 includes an encoder 306 and a WHDI transmitter 308. Video source device 202 has HDMI connector 206, and as described above, HDMI receiver 208 mates with HDMI connector 206. Video source device 202 provides video information 1 10 through HDMI connector 206 as raw video data to HDMI receiver 208. HDMI receiver 208 receives the raw video data, and encoder 306 compresses the raw video data to generate compressed video data. For example, encoder 306 may compress the raw video data according to MPEG-2, H.264, or other video file standard. WHDI transmitter 308 transmits the compressed video data from antenna 218 as compressed video data stream 314 according to the WHDI communication protocol.
[0043] Second WHDI communication module 304 is a receiver for display device 204 for compressed video data. As shown in FIG. 3, second WHDI communication module 304 includes a WHDI receiver 312 and a decoder 310. As shown in FIG. 3, display device 204 has HDMI connector 212, and as described above, HDMI transmitter 214 mates with HDMI connector 212. WHDI receiver 312 receives compressed video data stream 314 according to the WHDI communication protocol. Decoder 310 decompresses the compressed video data of compressed video data stream 314 to raw video data, and HDMI transmitter 214 transmits the raw video data to display device 204 through HDMI connector 212. Display device 204 displays the raw video data as video 1 14.
[0044] Because compressed video data is wirelessly streamed in system 300 of FIG. 3, compressed video data stream 314 can be transmitted at a lower data rate compared to raw video data stream 222 of FIG. 2. However, encoder 306 is present to compress the raw video data of video information 1 10 (typically in the form of a hardware-based encoder), and decoder 310 is present to decompress the compressed video data of compressed video data stream 314 (typically in the form of a hardware-based decoder), resulting in additional costly hardware to be present. As such, system 300 may be implemented less expensively than system 200 of FIG. 2, but the costs are still high enough to discourage mass production. [0045] FIG. 4 shows a block diagram of a video streaming system 400. System 400 is an example of system 100 of FIG. 1. As shown in FIG. 4, system 400 includes a video source device 402, display device 204, a first transceiver 406, a decoder 408, a second transceiver 412, antenna 218, and antenna 220. Furthermore, video source device 402 includes an encoder 414 and a driver 416. These elements of FIG. 4 are described as follows.
[0046] Transceiver 406 is a transmitter for video source device 402 for compressed video data. For example, transceiver 406 may be a WiFi or UWB transceiver. As shown in FIG. 4, encoder 414 compresses the raw video data of video information 1 10 to generate compressed video data. Video source device 202 has a USB connector 404. Transceiver 406 has a USB connector (not shown in FIG. 4) that mates with USB connector 404. Encoder 414 provides the compressed video data through USB connector 404 as compressed video data to transceiver 406. Transceiver 406 transmits the compressed video data from antenna 218 as compressed video data stream 420 according to the corresponding communication protocol (e.g., either WiFi or UWB).
[0047] Transceiver 412 is a receiver for display device 204 for compressed video data.
Transceiver 412 has a USB connector 410 that mates with a USB connector (not shown in FIG. 4) of decoder 408. Display device 204 has HDMI connector 212. Decoder 408 has an ITDMI connector (not shown in FIG. 4) that mates with ITDMI connector 212. Transceiver 412 receives compressed video data stream 420, and the received compressed video data of compressed video data stream 420 is transmitted through USB connector 410 to decoder 408. Decoder 408 decompresses the compressed video data to raw video data, and transmits the raw video data to display device 204 through ITDMI connector 212. Display device 204 displays the raw video data as video 114.
[0048] System 400 of FIG. 4 uses software encoding by encoder 414 to capture a full display screen of video source device 402 (e.g., including video 114 and background graphics), for display at display device 204. Furthermore, hardware decoding by decoder 408 is performed at the display device end in FIG. 4 to enable HDMI connectivity between video source device 402 and display device 204. As such, system 400 may be implemented less expensively than system 200 of FIG. 2 and system 300 of FIG. 3, but the costs are still high enough to discourage mass production (e.g., due to the hardware- based decoder 408).
[0049] The WiDi type of WiFi is an example of the communication technique used in system 400. Furthermore, some UWB techniques may be used that use software encoding at the video source device and use hardware decoding at the display device to achieve HDMI connectivity. These two techniques result in similarly expensive cost structures.
[0050] As such, HDMI connectivity is a contributing factor toward wireless video streaming techniques having high costs. Systems 200 and 300 both have HDMI connectivity at both ends, while system 400 has HDMI connectivity at the display device end. Embodiments are described in the following sections for wireless video streaming that do not use HDMI connectivity at the video source device and at the display device, resulting in systems that are low enough in cost to enable mass adoption.
A. Example Wireless Video Data Streaming Embodiments
[0051] Embodiments of the present invention enable video data streaming without needing HDMI connectivity. For example, FIG. 5 shows a block diagram of a video streaming system 500, according to an example embodiment. As shown in FIG. 5, system 500 includes a video source device 502, a display device 504, a first communication module 508, a second communication module 512, a first antenna 514, and a second antenna 530. Video source device 502 includes a driver 516, and display device 504 includes a decoder 520 and a display screen 522. In system 500, video source device 502 is a video source device enabled to stream video, such as HD video, according to the pull mode (where display device 504 requests the video content from video source device 502). Display device 504 is an example video sink device configured to display the video streamed by video source device 502.
[0052] System 500 is described as follows with respect to FIGS. 6 and 7. FIG. 6 shows a flowchart 600 for streaming video from a video source device, according to an example embodiment. In an embodiment, video source device 502 of FIG. 5 may operate according to flowchart 600. FIG. 7 shows a flowchart 700 for receiving streamed video at a display device, according to an example embodiment. In an embodiment, display device 504 of FIG 5 may operate according to flowchart 700. Further structural and operational embodiments will be apparent to persons skilled in the relevant art(s) based on the following description of flowcharts 600 and 700 and system 500.
[0053] Flowchart 600 of FIG. 6 begins with step 602. In step 602, video data is received from local storage or a remote source. For example, as shown in FIG. 5, driver 516 may receive video data 518. Video data 518 may be compressed video data that is stored in compressed form, or that may be compressed in real time as provided to driver 516. Video data 518 may be formatted according to one or more different video compression formats mentioned elsewhere herein or otherwise known. Video data 518 may be received by driver 516 from various sources, such as from storage located in or associated with video source device 502, or from a real time source such as a video camera included in video source device 502 (e.g., mounted to a housing of video source device 502, attached to video source device 502, etc.) or a remote source (e.g., that streams compressed video data 302 to video source device 502 in a wired and/or wireless manner) such as the Internet, a cable head-end, or a wireless network of a wireless operator. Table 1 indicates examples of video compression standards for video data 518 and corresponding examples of storage media of storage of video source device 502 in which video files containing video data 518 may be stored:
Table 1
Figure imgf000012_0001
In step 604, the video data is transmitted through a USB connector to be wirelessly transmitted from the video source device by a wireless communication module connected to the video source device at the USB connector. For example, as shown in FIG. 5, driver 516 generates a compressed video data stream 524. Compressed video data stream 524 includes video data 518 converted into the form of a video data stream. As shown in FIG. 5, video source device 502 includes a USB connector 506. USB connector 506 is a connector configured according to the USB specification. Video source device 502 is capable of transmitting USB signals (e.g., serial USB signals over D+ and D- pins) from video source device 502 to devices coupled to USB connector 506. For instance, compressed video data stream 524 may be transmitted through USB connector 506 to be received by communication module 508. Communication module 508 has a USB connector (not shown in FIG. 5) that mates with USB connector 506 so that compressed video data stream 524 may be provided to communication module 508. Communication module 508 is configured to wirelessly transmit compressed video data stream 524 from video source device 502 as wireless video data stream 528. [0055] Referring to FIG. 7, flowchart 700 begins with step 702. In step 702, video data is received through a USB connector, the video data having been wirelessly received at the display device from a video source device. For example, as shown in FIG. 5, communication module 512 is a receiver (and optionally a transmitter) for display device 504. Communication module 512 has a USB connector (not shown in FIG. 5) that mates with a USB connector 510 of display device 504. Communication module 512 receives wireless video data stream 528, and the received compressed video data of wireless video data stream 528 is transmitted through USB connector 510 to decoder 520.
[0056] In step 704, the received video data is decoded for display. For instance, as shown in FIG. 5, decoder 520 decompresses the compressed video data received in wireless video data stream 528 to decoded video data 532, and transmits decoded video data 532 to display screen 522. Display screen 522 generates a display of video 114 based on the video data of decoded video data 532.
[0057] As such, in the embodiment of FIG. 5, video data is wirelessly streamed from video source device 502 in a compressed form without using any UDMI connectivity. This results in a lower cost implementation than techniques that use UDMI connectivity, as such techniques typically either stream raw video data at high data rates (system 200 of FIG. 2), and/or use hardware encoders and/or decoders that are external to the video source device and display device (system 300 of FIG. 3 or system 400 of FIG. 4). In contrast, in system 500 of FIG. 5, wireless video data stream 528 contains compressed data that can be transmitted at a lower data rate, and does not use an external hardware decoder or encoder. For instance, instead of using an external decoder at the display device, decoder 520, which is internal to display device 504, may be used to decode/decompress the received compressed video data. Decoder 520 may be a type of decoder that is integrated in many commercially available display devices, typically implemented in software, relieving the need for an additional external decoder. Decoder 520 may be configured/modified to perform according to the embodiments described herein
[0058] Display device 504 may be any type of display device or device with a display, including a high definition (HD) display device that includes display screen 522 as an LCD (liquid crystal display) display or monitor, a plasma display or monitor, an LED (light emitting diode) display or monitor, an HD projector, or further type of display screen. Video source device 502 may be any type of stationary or mobile video source device, including a desktop computer, a set top box, a digital video disc (DVD) player, a video game console, a smart phone, a handheld computing device (e.g., a personal digital assistant (PDA), a BLACKBERRY device, a PALM device, etc.), a laptop computer, a tablet computer (e.g., an APPLE IP AD), or further type of video source device.
[0059] Video source device 502 may include a camera used to capture images, such as still images and video images that are streamed from the video source device in compressed form. Video source device 502 may include storage that stores compressed video data. Alternatively or additionally, video source devices may receive video data from external sources, such as from a cable television provider, a satellite television provider, an Internet, a wireless network of a wireless operator, or other network-based web site, or other remote source.
[0060] First and second communication modules 508 and 512 may be configured to enable wireless communications for video source device 502 and display device 504, respectively. For example, communication modules 508 and 512 may each include a frequency up-converter (transmitter) and down-converter (receiver). Communication modules 508 and 512 may each enable communication using one or more proprietary or commercially available wireless communication protocols/technologies, including wireless USB, WiFi, UWB, and/or other communication protocols/technologies. Communication modules 508 and 512 each include a USB connector. First communication module 508 receives video data in USB file form over a USB port that interfaces with USB connector 506 of video source device 502, and wirelessly transmits a video data stream over a second port (according to the applicable wireless communication protocol/technology). Second communication module 512 wirelessly receives the video data stream over a second port, and outputs video data in USB file form over a USB port that interfaces with USB connector 516 of display device 504.
[0061] In embodiments, wireless video data stream 528 may be generated and transmitted by communication module 508 in any video data stream form, proprietary or commercially available. For instance, in one embodiment, wireless video data stream 528 may be generated in elementary stream (ES) format, defined by an MPEG communication protocol. An elementary stream is often referred to as "elementary", "data", "audio", or "video" bitstreams or streams. The format of the elementary stream depends upon the codec or data carried in the stream, but will often carry a common header when packetized into a packetized elementary stream. Packetized elementary stream (PES) is a specification in the MPEG-2 Part 1 (Systems) (ISO IEC 13818-1) and ITU-T H.222.0 that defines carrying of elementary streams in packets within an MPEG program stream and MPEG transport stream. The elementary stream is packetized by encapsulating sequential data bytes from the elementary stream inside PES packet headers. In another embodiment, wireless video data stream 528 may be generated in transport stream (TS) format, a standard format for transmission and storage of audio, video, and data. Transport stream format is used in broadcast systems such as DVB (digital video broadcasting) and ATSC (Advanced Television Systems Committee). Transport stream specifies a container format encapsulating packetized elementary streams, with error correction and stream synchronization features for maintaining transmission integrity when the signal is degraded. In further embodiments, wireless video data stream 528 may be generated according to other video data stream format.
[0062] System 500, including source video device 502 and display device 504, may be implemented in various ways to wirelessly stream video data. For example, system 500 may be implemented to wirelessly stream video data according to the pull mode, where display device 504 generates a video content request 526 that is transmitted to video source device 502, and defines the video content to be provided to display device 504. In the pull mode, display device 504 controls the transmitting of video content to display device 504. Alternatively, system 500 may be implemented to wirelessly stream video data according to the push mode, where video source device 502 transmits (e.g., broadcasts) the video content without display device 504 being involved in controlling the transmitting of the video content. Example embodiments for system 500 described in the following sections.
B. Example Wireless Video Data Streaming Pull Mode Embodiments
[0063] Video source device 502 and display device 504 of system 500 may be configured in various ways to enable the streaming of video data according to the pull mode, where display device 504 controls the streaming of video data by video source device 502. For instance, FIG. 8 shows a block diagram of a video streaming system 800, according to an example embodiment. Video streaming system 800 is an example of video streaming system 500 of FIG. 5. In the example of FIG. 8, system 800 includes video source device 502, display device 504, first communication module 508, second communication module 512, first antenna 514, and second antenna 530. Video source device 502 includes driver 516 and a virtual disk 802, and display device 504 includes decoder 520 and display screen 522. The features of system 800 are described as follows. [0064] In system 800, video data 518 is stored in the form of a file in storage of video source device 502 that is accessible in a virtual disk 802 maintained at video source device 502. Virtual disk 802 is configured to mimic the behavior of a USB mass storage device (e.g., a "U-disk", etc.) that may be accessed by display device 504. In the embodiment of FIG. 8, communication modules 508 and 512 may relay a command or control message (e.g., request 526) from display device 504 to video source device 502 to initiate delivery of video data 518 from virtual disk 802 to display device 504. Driver 516 interprets the command and controls the delivery of video data 518 to display device 504 through communication modules 508 and 512 in response to the command. As such, in an embodiment, video decoder 520 reads video data 518 as a video file from virtual disk 802 through the wireless link provided by communication modules 508 and 512. In an embodiment, video decoder 520 is a video decoder that includes USB connectivity and functionality, and therefore can support this reading of the video file from virtual disk 802 emulating a USB mass storage device. Because display device 504 (e.g., decoder 520) controls this video transfer process, the video data streaming of FIG. 8 is performed according to the pull mode.
[0065] Video source device 502 may be configured in various ways to enable video content to be retrieved from virtual disk 802 according to the pull mode. For instance, FIG. 9 shows a block diagram of a video source device 900, according to an example embodiment. Video source device 900 is an example of video source device 502 of FIG. 8. As shown in FIG. 9, video source device 900 includes an application 902, driver 516, virtual disk 802, and a storage device 904. Application 902 includes a graphical user interface 906 and an application storage manager 908. Driver 516 includes a USB mass storage module 910 and a driver storage manager 912. Virtual disk 802 includes a file allocation table (FAT) 914 and a mapping table 916. Video source device 900 is configured to store video data in a virtual storage to be accessible over a USB (universal serial bus) link as a USB file stored a USB mass storage device. Video source device 900 is described as follows.
[0066] In an embodiment, application 902 is an application supported by video source device 900. Application 902 may be implemented in computer code that executes in one or more processors, or in any other form or combination of hardware, software, and/or firmware. Application 902 is configured to enable a user of device 900 to select one or more files stored in storage device 904 to be mapped into virtual disk 802. For instance, application storage manager 908 of application 902 may access storage device 904 to determine the one or more files stored in storage device 904. As shown in FIG. 9, application 902 may generate GUI 906, which may enable the user to select the one or more files determined to be stored in storage device 904 (e.g., by displaying the files in as list, a file/directory navigator, or in other manner). GUI 906 may then enable the user to submit the selected file(s), such as video file 918, to be mapped to virtual disk 802. GUI 906 may have any form, including one or more windows and user interface elements (e.g., displayed menus, buttons, text entry blanks, etc.) that may be interacted with by a user interface of device 900 (e.g., a keyboard, a mouse, a touch screen, a roller ball, a thumb wheel, etc.).
[0067] Application storage manager 908 of application 902 may read physical storage information (e.g., file name, file size, file location in storage device 904, etc.) associated with the selected files, and may transmit the physical storage information to driver storage manager 912 of driver 516. Driver storage manager 912 generates FAT 914 to include at least some of the received physical storage information. FAT 914 is a dummy file allocation table that simulates a real file allocation table, and may contain one or more dummy entries associated with dummy data clusters indicating virtual storage locations for the selected files. For instance, driver storage manager 912 may generate FAT 914 to indicate the virtual locations in virtual disk 802 that the selected files, including video file 918, are stored. Thus, FAT 914 may at least identify each of the selected files and their locations in virtual disk 802. Driver storage manager 912 may additionally generate mapping table 916 to indicate a mapping of the physical storage location in storage device 904 to the virtual storage location in virtual disk 802 for each of the selected files. For instance, for video file 918, mapping table 916 may indicate the physical storage location for video file 918 in storage device 904 and the virtual storage location for video file 918 in virtual disk 802. Using FAT 14 and mapping table 916, a physical storage location in storage device 904 for video file 918 can be determined when the virtual location of video file 918 in virtual disk 802 is provided during a virtual disk operation.
[0068] Storage device 904 may include any suitable type of storage in the form of one or more storage devices, including one or more primary or main memory devices (e.g., random access memory (RAM)), one or more secondary storage devices (e.g., a hard disk drive, a removable storage device or drive, a memory card, a memory stick, a floppy disk drive, a magnetic tape drive, a compact disc (CD) drive for CDs, a digital video disc (DVD) drive for DVDs, an optical storage device), and/or other types of storage. [0069] For instance, FIG. 10 shows a block diagram of video source device 900 of FIG. 9 streaming video data of video file 904 to a requesting display device, according to an example embodiment. As shown in FIG. 10, communication module 508 connected to video source device 900 may transmit stored file information 1002 to display device 504 of FIG. 8. Stored file information 1002 may be transmitted to display device 504 in response to a request by display device 504 (e.g., a request by a user to determine available video content), may be transmitted automatically, or in other manner. Stored file information 1002 may include FAT 914, which indicates virtual storage locations in virtual disk 802 for available video files for display. Display device 504 may enable a user to select a video file from FAT 914 for display. For instance, display device 504 of FIG. 8 may provide a user interface, such as a graphical user interface (e.g., similar to GUI 906 in FIG. 9, etc.), that enables the user to select a video file (e.g., a television show, a movie, a YouTube video, or other video content) from a list or collection of video files. Display device 504 may transmit video content request 526 to request the selected video file. For instance, decoder 520 of display device 504 may have a USB interface that enables display device 504 to generate request 526, and to transmit request 526 for the video file as if a USB mass storage device connected to USB connector 510 is being accessed for the video file. As shown in FIG. 8, communication module 512, which is connected to USB connector 510, transmits request 526 from display device 504.
[0070] Video source device 900 may receive video content request 526 transmitted from display device 504 of FIG. 8, which indicates the selected video file to be provided to display device 504. For instance, request 526 may include the virtual file location from FAT 914 for the selected video file, such as the virtual storage location for video file 918. As shown in FIG. 10, communication module 508 may receive video content request 526 at antenna 526, and may provide file request information 1004 to USB mass storage module 910 of driver 516 (through USB connector 506). File request information 1004 indicates the virtual storage location for video file 918 in virtual disk 802 provided in request 526. USB mass storage module 910 is configured to respond to USB transactions with the USB interface of decoder 520 of display device 504. For example, in an embodiment, the USB interface of decoder 520 may provide request 526 in the form of a USB CBW (command block wrapper), as would be known to persons skilled in the relevant art(s). As shown in FIG. 10, USB mass storage module 910 provides the virtual storage location to driver storage manager 912. Driver storage manager 912 accesses mapping table 916 to map the received virtual storage location to the physical storage location in storage device 904 for video file 918. As shown in FIG. 10, driver storage manager 912 accesses storage device 904 for video file 918 at the mapped physical storage location. USB mass storage module 910 transmits video file 918 to communication module 508 (through USB connector 506) as requested video file 1006, and communication module 508 transmits video file 918 as wireless video data stream 528.
[0071] As described above, wireless video data stream 528 is received at communication module 512, and video file 918 of wireless video data stream 528 is transmitted through USB connector 510 to decoder 520. Decoder 520 decompresses the compressed video data received in wireless video data stream 528 to decoded video data 532, and transmits decoded video data 532 to display screen 522. Display screen 522 generates a display of video 114 based on the video data of decoded video data 532.
[0072] FIG. 11 shows a flowchart 1100 for pull mode streaming of video from a video source device to a display device, according to an example embodiment. In an embodiment, video source device 900 of FIGS. 9 and 10 may operate according to flowchart 1 100. Further structural and operational embodiments will be apparent to persons skilled in the relevant art(s) based on the following description of flowchart 1 100.
[0073] Flowchart 1100 begins with step 1102. In step 1 102, a file allocation table (FAT) is provided that is wirelessly transmitted to a display device, the FAT indicating the virtual storage location for the USB file in the virtual USB mass storage device. For example, as described above, FAT 914 may be generated to map virtual storage locations in virtual disk 802 to physical storage locations in storage 904 for video files such as video file 918. USB mass storage module 910 may provide FAT 914 to communication module 508 (through USB connector 506) connected to video source device 900. As shown in FIG. 10, communication module 508 may transmit stored file information 1002, which contains FAT 914, to display device 504 of FIG. 8.
[0074] In step 1 104, a request for a USB file is received that contains video data, the request being wirelessly received at the video source device. For instance, as described above, video content request 526 may be wirelessly received by communication module 508 from display device 504 (e.g., from a USB interface of decoder 520), and may be handled by USB mass storage module 910. Request 526 indicates a request for a video file, such as video file 918. Video file 918 is referred to as a USB file because USB mass storage module 910 emulates a USB mass storage device connected to display device 504 at USB connector 510 on which video file 918 is stored. [0075] In step 1106, a mapping for the requested USB file from the virtual file storage location to a storage location in the storage device is determined. For example, as described above, driver storage manager 912 may access mapping table 916, which maps the virtual file storage location in virtual disk 802 for video file 918 to the physical storage location in storage device 904.
[0076] In step 1108, the requested video data is retrieved from the storage location in the storage device. For instance, as described above, video file 918 is retrieved from storage device 904 at the physical storage location indicated by the mapping.
[0077] In step 11 10, the requested video data is provided to be wirelessly transmitted in response to the request. For example, as described above, USB mass storage module 910 provides video file 918 to communication module 508, and communication module 508 wirelessly transmits video file 918 in wireless video data stream 528 in response to request 526.
C. Example Wireless Video Data Streaming Push Mode Embodiments
[0078] Video source device 502 and display device 504 of system 500 may be configured in various ways to enable the streaming of video data according to the push mode. In many cases, a video stream transferred to a display device is not provided as a result of file reading, such as in the pull mode described above. In many usage models, a video transfer occurs similarly to a broadcast video program. The video could be transmitted from real time compression of graphical/audio output of a video source device (e.g., a laptop computer), a real time Internet video download, or a stream formed by the real time parsing of a stored file. This type of video data streaming may be referred to as push mode, because the video source device is controlling the video transfer, and the video is provided in the form of a stream. In such a case, the decoder of the display device may actually decode a broadcast type of video program.
[0079] Video decoders in display devices typically have at least two data paths (or
"datapaths") for receiving video data streams. A first data path is referred to as a "stream data path" that is received through one or more transport stream (TS) ports of the decoder, and a second data path is referred to as a "file data path" that is received through one or more USB ports of the decoder. Typically, a push video stream is received through the transport stream (TS) ports instead of the USB ports of the decoder, due to the nature of the video stream being a stream of data packets formed according to the MPEG TS standard format or other TS format (or ES - elementary stream - format, which is typically the output of a video decoder). However, most wireless communication modules (e.g., WiFi, etc.) do not have a TS port for communicating with the TS port of a video decoder, while USB ports are widely integrated with wireless communication modules.
[0080] As such, in an embodiment, a video decoder is formed that is configured to enable
USB connectivity for received video data push stream. For instance, in an embodiment, the firmware of a video decoder may be modified, so that a video data stream delivered to the USB port of the video decoder can be redirected to the stream data path of the decoder.
[0081] For instance, FIG. 12A shows a block diagram of a video decoder 1200, according to an example embodiment. Video decoder 1200 is an example of decoder 520 of FIG. 5. As shown in FIG. 12A, decoder 1200 includes a stream interface (or port) 1202, a stream data path processing logic 1204, a USB interface (or port) 1206, a file data path processing logic 1208, a decoding engine 1210, and a data diverter 1212. Decoder 1200 is described as follows.
[0082] As shown in FIG. 12A, stream interface 1202 of decoder 1200 receives a first video data stream 1214 in TS (or ES) format. First video data stream 1214 may be received from a remote source or from storage associated with display device 504. Stream interface 1202 provides first video data stream 1214 as first video data stream 1218 to stream data path processing logic 1204. Stream data path processing logic 1204 is configured to process the stream data of first video data stream 1218 to generate first processed video data 1222. Decoding engine 1210 receives and decodes first processed video data 1222 for display, generating decoded video data 1226. For instance, decoded video data 1226 may be provided by decoding engine 1210 to display screen 522 as decoded video data 532, as shown in FIG. 5.
[0083] USB interface 1206 of decoder 1200 receives a second video data stream 1216
(through USB connector 510 of display device 504) that is wirelessly received at display device 504 (e.g., by communication module 512). Second video data stream 1216 is received in file format (e.g., USB file format), and typically is received from a storage device (e.g., a USB mass storage device) connected to display device 504. USB interface 1206 provides second video data stream 1216 as second video data stream 1220 to file data path processing logic 1208. File data path processing logic 1208 is configured to process USB file data of second video data stream 1216 to generate second processed video data 1224. Decoding engine 1210 receives and decodes second processed video data 1224 for display, generating decoded video data 1226. For instance, decoded video data 1226 may be provided by decoding engine 1210 to display screen 522 as decoded video data 532, as shown in FIG. 5.
[0084] Thus, decoder 1200 may receive TS or PS (program stream) formatted video data streams (e.g., formatted according to an ES (elementary stream) format, or other format) at stream interface 1202 for processing by stream data path processing logic 1204 and decoding by decoding engine 1210, and may receive USB file formatted video data streams at USB interface 1206 for processing by file data path processing logic 1208 and decoding by decoding engine 1210.
[0085] However, in an embodiment, as shown in FIG. 12A, data diverter 1212 may be present in decoder 1200. Data diverter 1212 enables stream data (e.g., TS or PS video data streams) received at USB interface 1206 to be processed by stream data path processing logic 1204 and decoded by decoding engine 1210. In this manner, display device 504 can receive a wireless video data stream at communication module 512 coupled to USB connector 510 that contains stream data, and the stream data can be processed by stream data path processing logic 1204 and decoded by decoding engine 1210 by data diverter 1212 diverting the stream data from USB interface 1206 to stream data path processing logic 1204 (rather than to file data path processing logic 1208).
[0086] For example, in an embodiment, decoder 1200 may operate according to FIG. 13.
FIG. 13 shows a flowchart 1300 for rerouting stream data in a decoder, according to an example embodiment. Further structural and operational embodiments will be apparent to persons skilled in the relevant art(s) based on the following description of flowchart 1300.
[0087] Flowchart 1300 begins with step 1302. In step 1302, the received video data is provided to the file data path processing logic if the video data received through the USB connector is received in the form of a USB file. For example, with reference to FIG. 12A, data diverter 1212 receives second video data stream 1220. If data diverter 1212 determines that second video data stream 1220 from USB interface 1206 includes file data (e.g., USB data of a USB file), data diverter 1212 does not divert second video data stream 1220. Second video data stream 1220 is allowed to be received by file data path processing logic 1208 for processing (and decoding by decoding engine 1210), as described above. Data diverter 1212 may determine that second video data stream 1220 contains file data in any way, including detecting predetermined file header information or other file field information (e.g., USB file field information) in second video data stream 1220, by detecting an absence of stream information, or in other manner, as would be known to persons skilled in the relevant art(s).
[0088] In step 1304, the received video data is routed to the stream data path processing logic if the video data received through the USB connector is stream data. For example, with reference to FIG. 12A, data diverter 1212 receives second video data stream 1220. If data diverter 1212 determines that second video data stream 1220 from USB interface 1206 includes stream data (e.g., TS or ES video data), data diverter 1212 routes or diverts second video data stream 1220 to stream data path processing logic 1204 for processing (and decoding by decoding engine 1210). In such case, second video data stream 1220 is not processed by file data path processing logic 1208. For example, in an embodiment, data diverter 1212 may provide a disable signal to file data path processing logic 1208, may prevent second video data stream 1220 from reaching file data path processing logic 1208 (e.g., by performing a de-multiplexing or controlled switch function), or otherwise prevent file data path processing logic 1208 from receiving and/or processing second video data stream 1220. Data diverter 1212 may determine that second video data stream 1220 contains stream data in any way, including detecting predetermined stream header information or other stream field information (e.g., MPEG-2 TS field information) in second video data stream 1220, by detecting an absence of file information, or in other manner, as would be known to persons skilled in the relevant art(s).
[0089] In this manner, stream-formatted data may be streamed to a display device over a
USB link that typically handles file-formatted data, and the stream-formatted data may be processed as stream data within the video decoder of the display device by being diverted from the USB data path to the stream data path.
[0090] Note that in another embodiment, rather than diverting a video data stream delivered to the USB port, a video decoder may be formed that includes an additional data path that processes a video data stream delivered to the USB port of the video decoder. For instance, FIG. 12B shows a block diagram of a video decoder 1230, according to an example embodiment. Video decoder 1230 is an example of decoder 520 of FIG. 5, and is generally similar to decoder 1200 of FIG. 12, with differences described as follows. As shown in FIG. 12B, decoder 1230 includes stream interface (or port) 1202, stream data path processing logic 1204, USB interface (or port) 1206, file data path processing logic 1208, decoding engine 1210, data diverter 1212, and a third data path processing logic 1232. Decoder 1230 is described as follows. [0091] As shown in FIG. 12B, stream interface 1202 of decoder 1230 receives first video data stream 1214 in TS (or ES) format. Similarly to the description provided above, stream interface 1202 provides first video data stream 1214 as first video data stream 1218 to stream data path processing logic 1204. Stream data path processing logic 1204 is configured to process the stream data of first video data stream 1218 to generate first processed video data 1222. Decoding engine 1210 receives and decodes first processed video data 1222 for display, generating decoded video data 1226.
[0092] Furthermore, USB interface 1206 of decoder 1200 receives second video data stream 1216, which is wirelessly received at display device 504 (e.g., by communication module 512). In an embodiment, second video data stream 1216 is received in file format (e.g., USB file format), and typically is received from a storage device (e.g., a USB mass storage device) connected to display device 504. In another embodiment, second video data stream 1216 is received in a streaming data format. For instance, second video data stream 1216 may include video data in transport stream (TS) format, program stream (PS) format, or other streaming data format mentioned elsewhere herein or otherwise known.
[0093] USB interface 1206 provides second video data stream 1216 as second video data stream 1220 to data diverter 1212. Data diverter 1212 enables USB disk file data of second video data stream 1216 received at USB interface 1206 to be processed by file data path processing logic 1208 and decoded by decoding engine 1210, or enables stream data (e.g., TS or PS video data streams) received at USB interface 1206 to be processed by third data path processing logic 1232 and decoded by decoding engine 1210. As shown in FIG. 12B, data diverter 1212 receives second video data stream 1220. If data diverter 1212 determines that second video data stream 1220 from USB interface 1206 includes file data (e.g., USB data of a USB disk file), data diverter 1212 routes or diverts second video data stream 1220 to file data path processing logic 1208. In such case, second video data stream 1220 is received by file data path processing logic 1208 for processing (and decoding by decoding engine 1210), as described above. If data diverter 1212 determines that second video data stream 1220 from USB interface 1206 includes stream data (e.g., TS or ES video data), data diverter 1212 routes or diverts second video data stream 1220 to third data path processing logic 1232 for processing (and decoding by decoding engine 1210).
[0094] As described above, file data path processing logic 1208 is configured to process
USB file data of second video data stream 1216 when received as second video data stream 1220 from data diverter 1212 to generate second processed video data 1224. Decoding engine 1210 receives and decodes second processed video data 1224 for display, generating decoded video data 1226. Third data path processing logic 1232 is configured to process the stream data of second video data stream 1220, when received from data diverter 1212, to generate third processed video data 1234. Decoding engine 1210 receives and decodes third processed video data 1234 for display, generating decoded video data 1226. For instance, decoded video data 1226 may be provided by decoding engine 1210 to display screen 522 as decoded video data 532, as shown in FIG. 5. Note that in embodiments, stream data path processing logic 1204, file data path processing logic 1208, and third data path processing logic 1232 may be entirely separate from each other, or any two or more of them may be overlapping (e.g., may share logic).
[0095] In this manner, display device 504 can receive a wireless video data stream at communication module 512 coupled to USB connector 510 that contains stream data, and the stream data can be processed by third data path processing logic 1232 and decoded by decoding engine 1210 by data diverter 1212 routing the stream data from USB interface 1206 to third data path processing logic 1232 (rather than to file data path processing logic 1208). For instance, third data path processing logic 1232 may include logic for processing stream-formatted data similarly to logic of stream data path processing logic 1204, but provides a data path that is separate from stream data path processing logic 1204.
[0096] Systems may be configured in various ways to wirelessly stream video data that includes stream-formatted data to be processed as described with respect to FIGS. 12 and 13. For instance, FIG. 14 shows a block diagram of a video streaming system 1400, according to an example embodiment. Video streaming system 1400 is an example of video streaming system 500 of FIG. 5. In the example of FIG. 14, system 1400 includes video source device 502, display device 504, first communication module 508, second communication module 512, first antenna 514, and second antenna 530. Video source device 502 includes driver 516, and display device 504 includes decoder 520 and display screen 522. The features of system 1400 are described as follows.
[0097] System 1400 operates according to the push mode because a video source device controls the video data transfer. In system 1400, a video data stream 1402 is received by video source device 502. Video data stream 1402 is streamed from another video source device (not shown in FIG. 14), in a wired and/or wireless manner, such as an Internet server, a cable head-end, another computer, etc. In real time (e.g., without storing in physical storage, such as storage device 904 of FIG. 9), driver 516 of video source device 502 forwards video data stream 1402 as video data stream 1404 to communication module 508 (through USB connector 506). Thus, in an embodiment, driver 516 performs a step 1502 shown in FIG. 15. In step 1502, a received video data stream is provided to be wirelessly transmitted from the video source device. As just described, driver 516 forwards video data stream 1402 as video data stream 1404 to communication module 508 to be wirelessly transmitted from video source device 502.
[0098] Communication module 508 transmits video data stream 1404 as wireless video data stream 1406 from antenna 514. Communication module 512 receives wireless video data stream 1406 at antenna 530, and provides video stream 1404 from wireless video data stream 1406 to decoder 520 (through USB connector 510). As such, video data stream 1402 is provided by video source device 502 to display device 504 using USB connectivity, and in an embodiment, data diverter 1212 of FIG. 12A may be used to divert the stream data of video data stream 1402 from the file data path (file data path processing logic 1208) to the stream data path (stream data path processing logic 1204).
[0099] FIG. 16 shows a block diagram of a video streaming system 1600, according to another example embodiment. Video streaming system 1600 is an example of video streaming system 500 of FIG. 5. In the example of FIG. 16, system 1600 includes video source device 502, display device 504, first communication module 508, second communication module 512, first antenna 514, and second antenna 530. Video source device 502 includes driver 516 and virtual disk 802, and display device 504 includes decoder 520 and display screen 522. The features of system 1600 are described as follows.
[0100] System 1600 operates according to the push mode because a video source device controls the video data transfer. However, in system 1600, data diverter 1212 of FIG. 12A need not be present in decoder 520. Instead, a virtual file 1602 in virtual disk 802 is configured to hold the real time push stream data of video data stream 1402 as it is received at video source device 502 (similarly to the pull mode embodiment described above with respect to FIGS. 8-11). A pre-generated FAT (similar to FAT 918 of FIG. 9) may be generated for virtual file 1602, and provided to display device 504, such that decoder 520 may operate as if decoder 520 is reading a video file from a U-disk or other USB mass storage device, while video data stream 1402 is written into the virtual file based on the pre-generated FAT on a real time basis. Video data stream 1402 may be written to physical memory of video source device 502 for temporary storage. The pre- generated FAT may mapped to the physical memory in any suitable manner. [0101] Thus, in system 1600, video data stream 1402 is received by video source device
502. As described above, video data stream 1402 may be streamed from another video source device (not shown in FIG. 16), in a wired and/or wireless manner, such as an Internet server, a cable head-end, another computer, etc. In real time, video data stream 1402 is stored in physical memory in stream form, and the physical memory location is mapped to a virtual storage location for virtual file 1602 in virtual disk 802. A pre- generated FAT may be transmitted to decoder 520 of display device 504 that indicates the virtual storage location for virtual file 1602. Decoder 520 may transmit request 526, as described above, to request the video file corresponding to virtual file 1602. Driver 516 of video source device 502 forwards video data stream 1402 as video data stream 1604 to communication module 508 (through USB connector 506). Video data stream 1604 is generated in file stream format (e.g., USB file format), such as by a USB mass storage module of driver 516 (e.g., USB mass storage module 910 of FIGS. 9 and 10) so that decoder 520 may request and receive virtual file 1602 using its USB interface (e.g., USB interface 1206 of FIG. 12A). Communication module 508 transmits video data stream 1604 as wireless video data stream 1606 from antenna 514. Communication module 512 receives wireless video data stream 1606 at antenna 530, and provides the video data of wireless video data stream 1606 to decoder 520 (through USB connector 510). Decoder 520 recognizes the video data stream of wireless video data stream 1606 as corresponding to the virtual storage location of virtual file 1602 provided in the pre-generated FAT. As such, video data stream 1402 is provided by video source device 502 to display device 504 using USB connectivity, and in an embodiment (referring to FIG. 12 A), data diverter 1212 is not needed to divert the file stream data of video data stream 1602 from the file data path of the decoder because file data path processing logic 1208 can process the file stream video data.
[0102] Thus, in an embodiment, driver 516 performs a flowchart 1700 shown in FIG. 17.
In step 1702, a received video data stream is stored in a virtual file in a virtual file storage location. For instance, as just described, video data stream 1402 is written into virtual file 1402 in a location indicated by a pre-generated FAT on a real time basis. In step 1704, the received video data stream is provided to be transmitted wirelessly from the video source device at least partially simultaneously with storing the received video stream in the virtual file. For example, as just described, driver 516 forwards video data stream 1402 as video data stream 1604 to communication module 508 to be wirelessly transmitted from video source device 502. This may occur partially overlapping in time with video data stream 1402 being stored in virtual file 1602.
[0103] Note that in another embodiment similar to FIGS. 16 and 17, video data stream
1402 is not written into virtual file 1602. Instead, video data stream 1402 is forwarded to decoder 520 at display device 504 as it is received by driver 516 upon request 526 being received for file content. In this embodiment, a FAT may be generated and provided to display device 504 that indicates a virtual file location for a virtual file, although the FAT is provided solely for a dummy purpose since the virtual file does not exist.
D. Further Example Embodiments
[0104] Driver 516, decoder 520, application 906, application storage manager 908, USB mass storage module 910, driver storage manager 912, decoder 1200, stream interface 1202, stream data path processing logic 1204, USB interface 1206, file data path processing logic 1208, decoding engine 1210, data diverter 1212, decoder 1230, and third data path processing logic 1232 may be implemented in hardware, software, firmware, or any combination thereof. For example, driver 516, decoder 520, application 906, application storage manager 908, USB mass storage module 910, driver storage manager 912, decoder 1200, stream interface 1202, stream data path processing logic 1204, USB interface 1206, file data path processing logic 1208, decoding engine 1210, data diverter 1212, decoder 1230, and third data path processing logic 1232 may be implemented as computer program modules or code configured to be executed in one or more processors. Alternatively, driver 516, decoder 520, application 906, application storage manager 908, USB mass storage module 910, driver storage manager 912, decoder 1200, stream interface 1202, stream data path processing logic 1204, USB interface 1206, file data path processing logic 1208, decoding engine 1210, data diverter 1212, decoder 1230, and third data path processing logic 1232 may be implemented as hardware logic/electrical circuitry.
[0105] For instance, devices in which embodiments may be implemented may include storage, such as storage drives, memory devices, and further types of computer-readable media. Examples of such computer-readable storage media include a hard disk, a removable magnetic disk, a removable optical disk, flash memory cards, digital video disks, random access memories (RAMs), read only memories (ROM), and the like. As used herein, the terms "computer program medium," "computer-readable medium, " and "computer storage medium" are used to generally refer to the hard disk associated with a hard disk drive, a removable magnetic disk, a removable optical disk (e.g., CDROMs, DVDs, etc.), zip disks, tapes, magnetic storage devices, MEMS (micro-electromechanical systems) storage, nanotechnology-based storage devices, as well as other media such as flash memory cards, digital video discs, RAM devices, ROM devices, and the like. Such computer-readable storage media are distinguished from and non-overlapping with communication media (do not include communication media). Communication media typically embodies computer-readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave. The term "modulated data signal" means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media includes wireless media such as acoustic, RF, infrared and other wireless media. Embodiments are also directed to such communication media.
[0106] Such computer-readable storage media may store program modules that include computer program logic for implementing the features of driver 516, decoder 520, application 906, application storage manager 908, USB mass storage module 910, driver storage manager 912, stream interface 1202, stream data path processing logic 1204, USB interface 1206, file data path processing logic 1208, decoding engine 1210, data diverter 1212, decoder 1230, third data path processing logic 1232, flowchart 600, flowchart 700, flowchart 1100, flowchart 1300, step 1502, and/or flowchart 1700 (including any step of flowcharts 600, 700, 1100, 1300, and 1700), and/or further embodiments of the present invention described herein. Embodiments of the invention are directed to computer program products comprising such logic (e.g., in the form of program code or software) stored on any computer useable medium. Such program code, when executed in one or more processors, causes a device to operate as described herein.
[0107] Decoder 520 (such as decoder 1200, decoder 1230, or other embodiment of decoder 520) may be implemented in various ways, including multiple integrated circuit devices (e.g., chips), or a single integrated circuit device (e.g., a system-on-chip (SOC) implementation). Decoder 520 may include one or more of memory, processing logic (e.g., one or more processors, etc.), embedded code, etc.
[0108] In an embodiment, stream data path processing logic 1204, file data path processing logic 1208, and third data path processing logic 1232 may be implemented as computer code that is executed by one or more processors, or may be implemented in other ways. Devices in which stream data path processing logic 1204, file data path processing logic 1208, and/or third data path processing logic 1232 are implemented may include buffers/memory (e.g., one or more RAM devices, including dynamic RAM devices, or static RAM devices such as flash memory, etc.) that is used to buffer data processed by logic 1204, logic 1208, and/or logic 1232. In embodiments, stream data path processing logic 1204, file data path processing logic 1208, and/or third data path processing logic 1232 may share some logic, or may each implement entirely separate logic.
Conclusion
While various embodiments of the present invention have been described above, it should be understood that they have been presented by way of example only, and not limitation. It will be apparent to persons skilled in the relevant art that various changes in form and detail can be made therein without departing from the spirit and scope of the invention. Thus, the breadth and scope of the present invention should not be limited by any of the above-described exemplary embodiments, but should be defined only in accordance with the following claims and their equivalents.

Claims

WHAT IS CLAIMED IS:
1. A video source device, comprising:
a universal serial bus (USB) connector;
a storage device; and
a driver that receives a request through the USB connector for a video file that contains video data, the request being wirelessly received at the video source device, and the request indicating a virtual file storage location at the video source device for the video file, the virtual file storage location being a virtual location in a virtual USB mass storage device;
wherein the driver determines a mapping for the requested video file from the virtual file storage location to a storage location in the storage device; and
wherein the driver retrieves the requested video file from the storage location in the storage device, and transmits the video data of the video file through the USB connector to be wirelessly transmitted in a video data stream in response to the request.
2. The video source device of claim 1, wherein the driver provides a file allocation table (FAT) that is wirelessly transmitted to a display device, the FAT indicating the virtual storage location for the video file in the virtual USB mass storage device; and wherein the request for the video file is received from the display device, the display device having determined the virtual file storage location at the video source device for the video file from the FAT.
3. The video source device of claim 1, wherein the driver includes:
a USB mass storage module configured to handle communications through the USB connector; and
a storage manager that accesses a mapping table to determine a mapping for the requested video file between the physical storage location and the virtual file storage location.
4. The video source device of claim 1, wherein the requested video data is wirelessly transmitted from the video source device by a wireless communication module connected to the video source device at the USB connector.
5. A video source device, comprising:
a universal serial bus (USB) connector; and
a driver that transmits video data through the USB connector to be wirelessly transmitted from the video source device by a wireless communication module connected to the video source device at the USB connector.
6. The video source device of claim 5, further comprising:
a storage device that stores the video data; and
wherein the driver retrieves the video data from the storage device to be transmitted by the driver.
7. The video source device of claim 5, wherein the video data transmitted by the driver is received in a push video data stream from a remote source by the video source device.
8. The video source device of claim 7, further comprising;
a storage device;
wherein the push data stream is stored in the storage device as a video file in a physical storage location that is mapped to a virtual file storage location by the driver, the virtual file storage location being a virtual location in a virtual USB mass storage device; and
wherein the video data transmitted by the driver is retrieved from the physical storage location at least partially overlapping in time with the push data stream being stored in the storage device to be transmitted from the video source device.
9. The video source device of claim 8, wherein the driver provides a file allocation table (FAT) that to be wirelessly transmitted to a display device, the FAT indicating the virtual file storage location for the video file in the virtual USB mass storage device.
10. The video source device of claim 8, wherein the driver includes:
a USB mass storage module configured to handle communications through the USB connector; and a storage manager that accesses a mapping table to determine a mapping for the requested video file between the physical storage location and the virtual file storage location.
11. A display device, comprising:
a universal serial bus (USB) connector; and
a decoder that includes a USB interface and a decoding engine;
wherein the USB interface receives video data in the form of stream data through the USB connector that is wirelessly received at the display device from a video source device, and the decoding engine decodes the received video data for display.
12. The display device of claim 1 1, wherein the decoder further includes:
a stream data path processing logic that is configured to process stream data to generate first processed video data that is received by the decoding engine,
a file data path processing logic that is configured to process file data to generate second processed video data that is received by the decoding engine, and
a data diverter;
wherein if the video data received through the USB connector is stream data, the data diverter routes the received video data to the stream data path processing logic, and if the video data received through the USB connector is received in the form of file data, the received video data is provided to the file data path processing logic.
13. The display device of claim 12, wherein the USB interface transmits a request for a video file to the video source device and receives the video data in the form of the video file in response to the request.
14. The display device of claim 13, wherein the USB interface receives a file allocation table (FAT) wirelessly transmitted from the video source device, the FAT indicating the virtual file storage location for the video file in a virtual USB mass storage device.
15. The display device of claim 12, wherein the USB interface receives the video data as stream data according to a push mode.
16. The display device of claim 12, further comprising:
a display screen;
wherein if the received video data is stream data, the first processed video data is decoded by the decoding engine for display by the display screen; and
if the received video data is received in the form of a file data, the second processed video data is decoded by the decoding engine for display by the display screen.
17. The display device of claim 12, wherein the video data is wirelessly received at the display device by a wireless communication module connected to the display device at the USB connector.
PCT/CN2011/075641 2011-06-13 2011-06-13 Wireless video streaming using usb connectivity of hd displays WO2012171156A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/CN2011/075641 WO2012171156A1 (en) 2011-06-13 2011-06-13 Wireless video streaming using usb connectivity of hd displays

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2011/075641 WO2012171156A1 (en) 2011-06-13 2011-06-13 Wireless video streaming using usb connectivity of hd displays

Publications (1)

Publication Number Publication Date
WO2012171156A1 true WO2012171156A1 (en) 2012-12-20

Family

ID=47356486

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2011/075641 WO2012171156A1 (en) 2011-06-13 2011-06-13 Wireless video streaming using usb connectivity of hd displays

Country Status (1)

Country Link
WO (1) WO2012171156A1 (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2014204061A1 (en) * 2013-06-17 2014-12-24 Samsung Electronics Co., Ltd. Content receiving apparatus, display device and content receiving method thereof
CN105282601A (en) * 2014-07-07 2016-01-27 乐视致新电子科技(天津)有限公司 One screen sharing method, apparatus and system
JP2019175260A (en) * 2018-03-29 2019-10-10 エヌ3エヌ、カンパニー、リミテッドN3N Co.,Ltd. Transmitter and receiver for remote control support, receiver, operation method thereof, and remote control system

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1859669A (en) * 2005-11-19 2006-11-08 华为技术有限公司 Method and system for realizing virtual storage in mobile terminal
CN101106681A (en) * 2007-06-26 2008-01-16 广东威创日新电子有限公司 Wireless video transmission device and method
CN101714933A (en) * 2008-10-06 2010-05-26 厦门华侨电子股份有限公司 Method and system for playing remote media

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1859669A (en) * 2005-11-19 2006-11-08 华为技术有限公司 Method and system for realizing virtual storage in mobile terminal
CN101106681A (en) * 2007-06-26 2008-01-16 广东威创日新电子有限公司 Wireless video transmission device and method
CN101714933A (en) * 2008-10-06 2010-05-26 厦门华侨电子股份有限公司 Method and system for playing remote media

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2014204061A1 (en) * 2013-06-17 2014-12-24 Samsung Electronics Co., Ltd. Content receiving apparatus, display device and content receiving method thereof
CN105282601A (en) * 2014-07-07 2016-01-27 乐视致新电子科技(天津)有限公司 One screen sharing method, apparatus and system
EP3096531A4 (en) * 2014-07-07 2017-01-04 LE Holdings (Beijing) Co., Ltd. One-screen sharing method, apparatus, and system
JP2019175260A (en) * 2018-03-29 2019-10-10 エヌ3エヌ、カンパニー、リミテッドN3N Co.,Ltd. Transmitter and receiver for remote control support, receiver, operation method thereof, and remote control system

Similar Documents

Publication Publication Date Title
US11019403B2 (en) Transcoder enabled cloud of remotely controlled devices
US9300754B2 (en) Information processing system, information processing apparatus, information processing method, and program
US8554938B2 (en) Web browser proxy-client video system and method
US20120254931A1 (en) Content Extraction for Television Display
US9276997B2 (en) Web browser proxy—client video system and method
US9131267B2 (en) Apparatus and method of portable terminal for dual display of broadcasting receiver by HDMI signal
US20130135179A1 (en) Control method and device thereof
CN103037256A (en) Method for sharing screen between mobile terminal and TV receiving terminal and TV receiving terminal
JP2015080025A (en) Electronic apparatus and communication control method
US20150067758A1 (en) Methods for content sharing utilizing a compatibility notification to a display forwarding function and associated devices
US9800901B2 (en) Apparatus, systems and methods for remote storage of media content events
KR20210073280A (en) Electronic apparatus and method of controlling the same
US20050273824A1 (en) Set top box for PC/HDTV multimedia center
WO2012171156A1 (en) Wireless video streaming using usb connectivity of hd displays
CN109640136B (en) Method and device for controlling television, electronic equipment and readable medium
KR20140117889A (en) Client apparatus, server apparatus, multimedia redirection system and the method thereof
US20120106918A1 (en) Interface device and data processing method
EP2947843B1 (en) Server apparatus, display apparatus, system, and controlling methods thereof
CN102340519B (en) Multimedia network data sharing system
WO2012149685A1 (en) Wireless hd video streaming with intermediate bridge
WO2012149684A1 (en) Low power and low latency push mode wireless hd video streaming architecture for portable devices
CN117812341A (en) Display equipment and media asset playing method
KR101337952B1 (en) Connection device of pc and tv
CN115914694A (en) Display device and network connection method based on USB link

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 11867611

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 11/04/2014)

122 Ep: pct application non-entry in european phase

Ref document number: 11867611

Country of ref document: EP

Kind code of ref document: A1