US20080025196A1 - Method and system for providing visually related content description to the physical layer - Google Patents
Method and system for providing visually related content description to the physical layer Download PDFInfo
- Publication number
- US20080025196A1 US20080025196A1 US11/492,391 US49239106A US2008025196A1 US 20080025196 A1 US20080025196 A1 US 20080025196A1 US 49239106 A US49239106 A US 49239106A US 2008025196 A1 US2008025196 A1 US 2008025196A1
- Authority
- US
- United States
- Prior art keywords
- data
- media object
- audiovisual scene
- block
- priority level
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/1066—Session management
- H04L65/1101—Session protocols
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/60—Network streaming of media packets
- H04L65/70—Media network packetisation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W80/00—Wireless network protocols or protocol adaptations to wireless operation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W88/00—Devices specially adapted for wireless communication networks, e.g. terminals, base stations or access point devices
- H04W88/02—Terminal devices
Definitions
- Certain embodiments of the invention relate to signal handling for wireless communications. More specifically, certain embodiments of the invention relate to a method and system for providing visually related content description to the physical layer.
- Broadcasting and telecommunications have historically occupied separate fields.
- broadcasting was largely an “over-the-air” medium while wired media carried telecommunications. That distinction may no longer apply as both broadcasting and telecommunications may be delivered over either wired or wireless media.
- Present development may adapt broadcasting to mobility services.
- One limitation for transfer of multimedia data such as, for example, for digital television, video, digital photo, voice over IP, and even pure voice, has been data transmission rate bottleneck.
- this obstacle may be overcome.
- Terrestrial television and radio broadcast networks have made use of high power transmitters covering broad service areas, which enable one-way distribution of content to user equipment such as televisions and radios.
- wireless telecommunications networks have made use of low power transmitters, which have covered relatively small areas known as “cells”.
- wireless networks may be adapted to provide two-way interactive services between subscribers of user equipment such as telephones and computer equipment.
- first generation systems The networks of this period may be commonly known as first generation, or “1G” systems. These systems were based upon analog, circuit-switching technology, the most prominent of these systems may have been the advanced mobile phone system (AMPS). Second generation, or “2G” systems ushered improvements in performance over 1G systems and introduced digital technology to mobile communications. Exemplary 2G systems include the global system for mobile communications (GSM), digital AMPS (D-AMPS), and code division multiple access (CDMA). Many of these systems have been designed according to the paradigm of the traditional telephony architecture, often focused on circuit-switched services, voice traffic, and supported data transfer rates up to 14.4 kbits/s.
- GSM global system for mobile communications
- D-AMPS digital AMPS
- CDMA code division multiple access
- the 2.5G networks began the introduction of packet-switching technology in wireless networks. However, it is the evolution of third generation, or “3G” technology that may introduce fully packet-switched networks, which support high-speed data communications.
- DTTB Digital television terrestrial broadcasting
- ATSC advanced standards technical committee
- DVD-T digital video broadcast terrestrial
- ISDB-T integrated service digital broadcasting terrestrial
- the ATSC system has largely been adopted in North America, South America, Taiwan, and South Korea.
- This system adapts trellis coding and 8-level vestigial sideband (8-VSB) modulation.
- the DVB-T system has largely been adopted in Europe, the Middle East, Australia, as well as parts of Africa and parts of Asia.
- the DVB-T system adapts coded orthogonal frequency division multiplexing (COFDM).
- COFDM coded orthogonal frequency division multiplexing
- the OFDM spread spectrum technique may be utilized to distribute information over many carriers that are spaced apart at specified frequencies.
- the OFDM technique may also be referred to as multi-carrier or discrete multi-tone modulation.
- the spacing between carriers may prevent the demodulators in a radio receiver from seeing frequencies other than their corresponding frequency. This technique may result in spectral efficiency and lower multi-path distortion, for example.
- the ISDB-T system has been adopted in Japan and adapts bandwidth segmented transmission orthogonal frequency division multiplexing (BST-OFDM).
- BST-OFDM bandwidth segmented transmission orthogonal frequency division multiplexing
- the various DTTB systems may differ in important aspects; some systems employ a 6 MHz channel separation, while others may employ 7 MHz or 8 MHz channel separations.
- One of the more notable reasons may be the high data rates that may be supported in DTTB systems.
- DVB-T may support data rates of 15 Mbits/s in an 8 MHz channel in a wide area SFN.
- ISDB-T was originally designed to support broadcast services to mobile user equipment. While DVB-T may not have been originally designed to support mobility broadcast services, a number of adaptations have been made to provide support for mobile broadcast capability.
- the adaptation of DVB-T to mobile broadcasting is commonly known as DVB handheld (DVB-H).
- the broadcasting frequencies for Europe are in UHF (bands IVN) and in the US, the 1670-1675 MHz band that has been allocated for DVB-H operation. Additional spectrum is expected to be allocated in the L-band worldwide.
- DVB-H may support time slicing to reduce power consumption at the user equipment, addition of a 4K mode to enable network operators to make tradeoffs between the advantages of the 2K mode and those of the 8K mode, and an additional level of forward error correction on multi-protocol encapsulated data—forward error correction (MPE-FEC) to make DVB-H transmissions more robust to the challenges presented by mobile reception of signals and to potential limitations in antenna designs for handheld user equipment.
- DVB-H may also use the DVB-T modulation schemes, like QPSK and 16-quadrature amplitude modulation (16-QAM), which may be more resilient to transmission errors.
- MPEG audio and video services may be more resilient to error than data, thus additional forward error correction may not be required to meet DTTB service objectives.
- a signal from a transmitter to the mobile terminal may change in strength as the mobile terminal moves with respect to the transmitter.
- the signal from the mobile terminal may also take different paths by, for example, reflecting from buildings, trees, bodies of water, the ground, and/or other surfaces.
- the transmitted signal may also be attenuated when it passes through an object, for example, various glass surfaces in buildings.
- the throughput of a mobile user equipment may need to be optimized based on the services that are provided.
- developing a mobile user equipment with more flexible architectures that enable better control of content transmission may increase the efficiency with which the integrated voice, video, and data services may be communicated in higher data rate systems.
- a system and/or method is provided for providing visually related content description to the physical layer, substantially as shown in and/or described in connection with at least one of the figures, as set forth more completely in the claims.
- FIG. 1 is a block diagram illustrating an exemplary system that may be utilized for content-aware mapping and error protection, in accordance with an embodiment of the invention.
- FIG. 2A illustrates an exemplary conventional architecture for transmitting data.
- FIG. 2B illustrates an exemplary architecture for source layer optimization for transmitting data, in accordance with an embodiment of the invention.
- FIG. 2C illustrates an exemplary architecture for source layer optimization with an integrated multiplexer and partitioner for transmitting data, in accordance with an embodiment of the invention.
- FIG. 3A illustrates an exemplary feedback from a receiver to a transmitter, in accordance with an embodiment of the invention.
- FIG. 3B illustrates an exemplary multiple antenna architecture with feedback from a receiver to a transmitter, in accordance with an embodiment of the invention.
- FIG. 3C illustrates an exemplary constellation with four constellation points, which may be utilized in connection with an embodiment of the invention.
- FIG. 3D illustrates an exemplary constellation with 16 constellation points, which may be utilized in connection with an embodiment of the invention.
- FIGS. 4A and 4B illustrate an exemplary original image and a reconstruction of the original image after compression, respectively, in connection with an embodiment of the invention.
- FIG. 5A illustrates an exemplary audiovisual scene comprising a plurality of image objects, in connection with an embodiment of the invention.
- FIG. 5B illustrates exemplary radio frequency (RF) modulation and forward error correction (FEC) techniques that may be applied to image objects from FIG. 5A based on a priority, in accordance with an embodiment of the invention.
- RF radio frequency
- FEC forward error correction
- FIG. 6 is a flow diagram illustrating exemplary steps in providing visually related content description to the physical layer for wireless communication, in accordance with an embodiment of the invention.
- Certain embodiments of the invention may be found in a method and system for providing visually related content description to the physical layer.
- Aspects of the invention may comprise a MAC layer and/or a PHY layer in a wireless communication device that may be controlled for processing at least one media object in an audiovisual scene based on metadata information that corresponds to the audiovisual scene.
- the metadata information may result from, for example, MPEG-4 encoding of the audiovisual scene.
- Each media object may be forward error corrected based on a determined priority level that results from the metadata information.
- the priority may refer to whether a media object is part of the background or foreground of the audiovisual scene, for example. In this regard, the number of bits utilized for protection may be based on the priority level of the media object.
- each media object may be RF modulated in accordance with its corresponding priority level.
- the RF modulation may be, for example, QPSK modulation or 16 QAM modulation.
- the metadata information and the processed media objects may then be wireless communicated to other devices.
- priority obtained from the metadata information may also be utilized to select least one antenna from which to transmit at least a portion of the audiovisual scene, for example.
- FIG. 1 is a block diagram illustrating an exemplary system that may be utilized for content-aware mapping and error protection, in accordance with an embodiment of the invention.
- terrestrial network 102 there Q is shown terrestrial network 102 , wireless service provider network 104 , service provider 106 a and 106 b , portal 108 , public switched telephone network (PSTN) 110 , mobile terminals 116 a , 116 b , and 116 c , a WiFi access point 120 , a WiMax transmitter 122 , and a Bluetooth device 124 .
- the terrestrial network 102 may comprise transmitter (Tx) 102 a , multiplexer (Mux) 102 b , and information content source 114 .
- Tx transmitter
- Mux multiplexer
- the content source 114 may also be referred to as a data carousel, which may comprise audio, data and video content.
- the terrestrial network 102 may also comprise DVB antennas 112 a and 112 b that may be adapted to transmit DVB-based information, such as DVB-T or DVB-H, to the mobile terminals 116 a , 166 b , and 116 c .
- the DVB antennas 112 a and 112 b may communicate with each other via DVB-T and with the mobile terminals via DVB-H, for example.
- the wireless service provider network 104 may comprise mobile switching center (MSC) 118 a , and a plurality of cellular base stations 104 a and 104 b.
- MSC mobile switching center
- the WiFi access point 120 may allow a terminal, such as, for example, the mobile terminal 116 a , 116 b , or 116 c , to access a network such as, for example, the Internet. Additionally, the WiMax antenna 122 may also allow a terminal, such as, for example, the mobile terminal 116 a , 116 b , or 116 c , to access a network such as, for example, the Internet.
- the WiFi access point 120 and the WiMax antenna 122 may be serviced by, for example, the service provider 106 b .
- the mobile terminal 116 a , 116 b , and 116 c may also allow communication using Bluetooth protocol.
- the mobile terminal 116 b may communicate with the Bluetooth device 124 , which may be, for example, a wireless headset for a cellular phone.
- the network described in FIG. 1 need not be limited to any specific technology and, accordingly, may also support other communication technologies such as wireless metropolitan area networks (WMAN), wireless local area networks (WLAN), and/or wireless personal area networks (WPAN), for example.
- WMAN wireless metropolitan area networks
- WLAN wireless local area networks
- WPAN wireless personal area networks
- the mobile terminals may possess the capability to handle any one or more of a plurality of exemplary access technologies such as DVB-H, WCDMA, CDMA, CDMA200, GSM, 802.11, 802.16 and Bluetooth.
- the terrestrial network 102 may comprise suitable equipment that may be enable encoding and/or encryption of data for transmission via the transmitter 102 a .
- the transmitter 102 a in the terrestrial network 102 may enable utilizing, for example, DVB channels to communicate information to the mobile terminals.
- the transmitter 102 a may enable DVB-H transmission to the mobile terminals via the UHF band, such as bands IVN, the 1670-1675 MHz band, and/or the L-band, for example.
- the transmitter 102 a may have the capability to determine the type of media that is being communicated and accordingly alter the type of modulation and/or coding used to process the media content so as to provide content-aware mapping and error protection.
- MIMO Multiple-input-multiple-output
- the transmitter 102 a may also utilize, for example, beamforming to transmit information.
- the beamforming may also utilize the feedback information provided by the mobile terminals 116 a , 116 b , and/or 116 c .
- U.S. application Ser. No. ______ (Attorney Docket No. 17289US01) filed on even date herewith, provides an exemplary communication system that utilizes content-aware beamforming encoding and is hereby incorporated herein by reference in its entirety.
- the multiplexer 102 b associated with the terrestrial network 102 may be utilized to multiplex data from a plurality of sources.
- the multiplexer 102 b may be adapted to multiplex various types of information such as audio, video and/or data into a single pipe for transmission by the transmitter 102 a .
- Content media from the portal 108 which may be handled by the service provider 106 a may also be multiplexed by the multiplexer 102 b .
- the portal 108 may be an ISP service provider.
- the mobile terminals 116 a , 116 b , and/or 116 c may receive DVB-H services from the DVB antennas 112 a or 112 b whenever the mobile terminals are within operating range of the DVB antenna.
- the terrestrial network 102 may enable providing one or more digital television (DTV) channels to the service provider 106 a .
- the terrestrial network 102 may comprise suitable high-speed or broadband interfaces that may be utilized to facilitate transfer of the DTV channels from the terrestrial network 102 to the service provider.
- the service provider 106 a may then utilize at least a portion of the DTV channels to provide television (TV) on demand service, or other similar types of services to the wireless service provider network 104 .
- the service provider 106 a may further comprise suitable high-speed or broadband interfaces that may be utilized to facilitate the transfer of related TV on demand information to the MSC 118 a .
- the communication links between the terrestrial network 102 and the service provider 106 a and the communication links between the service provider 106 a and the wireless service provider 104 may be wired and/or wireless communication links.
- the wireless service provider network 104 may be a cellular or personal communication service (PCS) provider that may enable broadcasting UMTS (B-UMTS), for example.
- PCS personal communication service
- B-UMTS broadcasting UMTS
- the term cellular as utilized herein refers to both cellular and PCS frequencies bands. Hence, usage of the term cellular may comprise any band of frequencies that may be utilized for cellular communication and/or any band of frequencies that may be utilized for PCS communication.
- broadcast UMTS (B-UMTS) may also be referred to as MBMS.
- MBMS is a high-speed data service that is overlaid on WCDMA to provide much higher data rates than may be provided by core WCDMA.
- the B-UMTS services may be superimposed on the cellular or PCS network.
- the wireless service provider network 104 may utilize cellular or PCS access technologies such as GSM, CDMA, CDMA2000, WCDMA, AMPS, N-AMPS, and/or TDMA, for example.
- the cellular network may be utilized to offer bi-directional services via uplink and downlink communication channels, while the B-UMTS or MBMS network may be utilized to provide a unidirectional broadband services via a downlink channel.
- content-aware coding, mapping, and/or error protection may be utilized on the downlink.
- the B-UMTS or MBMS unidirectional downlink channel may be utilized to transmit content media and/or multimedia type information to the mobile terminals 116 a and 116 b .
- MBMS provides only unidirectional downlink communication, other bidirectional communication methodologies comprising uplink and downlink capabilities, whether symmetric or asymmetric, may be utilized.
- the wireless service provider network 104 need not be limited to a GSM, CDMA, WCDMA based network and/or variants thereof.
- the wireless service provider network 104 may be, for example, an 802.11, an 802.16, or a wireless local area network (WLAN).
- the wireless service provider network 104 may also be adapted to provide 802.11 or 802.16 based wireless communication in addition to GSM, CDMA, WCDMA, CDMA2000 based network and/or variants thereof.
- the mobile terminals 116 a , 116 b , and 116 c may access a network via a WiFi access point 120 and/or the WiMax antenna 122 .
- the mobile terminals 116 a , 116 b , and 116 c may also be compliant with the 802.11-based wireless network.
- the service provider 106 a may comprise suitable interfaces, circuitry, logic and/or code that may enable communication between the terrestrial network 102 and the wireless communication network 104 .
- the service provider 106 a may enable its interfaces to facilitate exchange control information with the terrestrial network 102 and to exchange control information with the wireless service provider 104 .
- the control information exchanged by the service provider 106 a with the terrestrial network 102 and the wireless communication network 104 may be utilized to control certain operations of the mobile terminals, the terrestrial network 102 and the wireless communication network 104 .
- the portal 108 may comprise suitable logic, circuitry and/or code that may enable providing content media to the service provider 106 a via one or more communication links. These communication links, although not shown, may comprise wired and/or wireless communication links. In various exemplary embodiments of the invention, these communication links may utilize any of a plurality of communication access technologies disclosed herein. Other access technologies not disclosed herein may be utilized without departing from the spirit or scope of the invention.
- the content media that may be provided by the portal 108 may comprise audio, data, video or any combination thereof. In this regard, the portal 108 may provide one or more specialized information services to the service provider 106 a.
- the public switched telephone network (PSTN) 110 may be coupled to the MSC 118 a . Accordingly, the MSC 118 a may enable switching of calls originating from within the PSTN 110 to one or more mobile terminals serviced by the wireless service provider 104 . Similarly, the MSC 118 a may enable switching of calls originating from mobile terminals serviced by the wireless service provider 104 to one or more telephones serviced by the PSTN 110 .
- PSTN public switched telephone network
- the information content source 114 may comprise a data carousel.
- the information content source 114 may provide various information services, which may comprise online data including audio, video and data content.
- the information content source 114 may also comprise file download, and software download capabilities.
- a mobile terminal fails to acquire requested information from the information content source 114 or the requested information is unavailable, then the mobile terminal may acquire the requested information via, for example, B-UMTS from the portal 108 .
- the request may be initiated through an uplink cellular communication path.
- the mobile terminals 116 a , 116 b , and 116 c may comprise suitable logic, circuitry and/or code that may enable handling the processing of uplink and downlink cellular channels for various access technologies and DVB-H technologies.
- the mobile terminals 116 a , 116 b , and 116 c may enable processing of voice, video, and data services, for example.
- the mobile terminals 116 a , 116 b , and 116 c may enable utilizing one or more cellular access technologies such as GSM, GPRS, EDGE, CDMA, WCDMA, CDMA2000, HSDPA and MBMS (B-UMTS).
- the mobile terminal may also enable receiving and processing DVB-H signals in the DVB-H bands.
- a mobile terminal may also enable requesting information via a first cellular service and in response, receive corresponding information via a DVB-H service.
- a mobile terminal may also enable requesting of information from a service provider via a cellular service and in response, receive corresponding information via a data service, which is provided via the cellular service.
- the mobile terminals may also be adapted to receive DVB-H information from the base stations 104 a or 104 b or from the DVB-H antennas 112 a and 112 b . In instances where a mobile terminal receives information from any of the base stations 104 a or 104 b via a downlink MBMS communication channel, then the mobile terminal may communicate corresponding uplink information via an uplink cellular communication channel.
- Data may be compressed prior to transmission to reduce the amount of data that needs to be transmitted.
- the compression may take place, for example, in the transmitter 102 a . Since errors in the compressed data may lead to inability to decompress the data or to give a result that may affect the decompressed data adversely, data protection methods may be used to enable correction of detected errors.
- Data protection methods may differ in the number of extra bits that may be used for error detection and correction. Generally, the more bits that are used, the more the data may be protected from uncorrectable errors. However, the extra bits may also reduce the throughput of the data.
- different portions of the multimedia information may be given different priority, where the priority may be used to determine the data protection method used to protect the data.
- feedback information such as performance metrics, from the mobile terminals 116 a , 116 b , and 116 c that may be utilized to determine which antenna may transmit content with certain priorities.
- antennas 112 a and 112 b are used for MIMO transmission
- feedback information from the mobile terminals may be utilized to determine that high priority data, such as video content, for example, may be transmitted from the antenna 112 a and low priority data, such as audio content, for example, may be transmitted from the antenna 112 b .
- high priority data such as video content
- low priority data such as audio content
- channel estimations may be utilized for determining the appropriate content-aware mapping/error protection, antenna selection, and/or encoding method, for example.
- FIG. 2A illustrates an exemplary conventional architecture for transmitting data.
- a source encoder block 200 may comprise suitable logic, circuitry, and/or code that may be utilized to enable, compression of data that is to be transmitted.
- the compressed data may be video data in MPEG-4 format.
- the PHY/MAC block 204 may comprise suitable logic, circuitry, and/or code that may be utilized to enable conversion of input data in a digital format to output suitably modulated RF signal.
- the PHY/MAC block 204 may apply a forward error correction (FEC) code to the digital data.
- FEC forward error correction
- the PHY/MAC block 204 may also convert the digital data to an analog signal, and then RF modulate the analog signal.
- the PHY/MAC block 204 may communicate the modulated analog signal to the transmit antenna 208 for transmission.
- the parameter control block 206 may comprise suitable logic, circuitry, and/or code that may be utilized to enable controlling of various operations on the digital data in the PHY/MAC block 204 before the digital data is output for transmission.
- the parameter control block 206 may configure the PHY/MAC block 204 to use a specific FEC code and/or RF modulation.
- the source encoder 200 may, for example, compress video data to MPEG-4 format and store the compressed data in the memory block 202 .
- the PHY/MAC 204 may read portions of the compressed data from the memory block 202 .
- the PHY/MAC 204 may then perform various operations to generate a suitable RF signal that may be transmitted via the transmit antenna 208 to mobile terminals, for example, the mobile terminals 116 a , 116 b , and 116 c.
- FIG. 2B illustrates an exemplary architecture for source layer optimization for transmitting data, in accordance with an embodiment of the invention.
- the transmit block 215 may comprise a source encoder block 220 , a memory block 222 , a source layer multiplexer block 224 , a PHY/MAC block 226 , a cross-layer partitioner block 228 , a parameter control block 230 , and transmit antennas 232 a , . . . , 232 b .
- the transmit block 215 may be, for example, part of the cellular base stations 104 a or 104 b.
- the source encoder block 220 may comprise suitable logic, circuitry, and/or code that may be utilized to enable compression of data prior to transmission.
- the compressed data may be video data in MPEG-4 format.
- the source encoder block 220 may also communicate information about the compressed data to the cross-layer partitioner block 228 .
- the information communicated may relate to the type of compression.
- the source encoder block 220 may communicate the specific type of compression, such as MPEG-1, MPEG-2, MPEG-4, H.261, H.263, or H.264.
- the source encoder block 220 may also communicate the type of chroma subsampling used, such as, for example, 4-4-4, 4-2-2, or 4-2-0 chroma subsampling.
- the source encoder may generate metadata information associated with the contents of the video data.
- the source encoder may generate metadata information associated with media objects in an audiovisual scene comprised within the video stream.
- the metadata information may indicate, for example, whether a media object is associated with the foreground or the background of the audiovisual scene.
- the metadata information need not be so limited and may also indicate other visually related content description of the audiovisual scene.
- the audiovisual scene need not be limited to having foreground and background layers of the scene but may comprise a plurality of descriptive layers to which a media object may be associated.
- Video encoding or compression operations such as MPEG-2 format, for example, may also be expanded to generate metadata information describing visually related content.
- the compressed video data may comprise the metadata information associated with audiovisual scenes in the video data.
- the source encoder block 220 may be enabled to communicate information about the metadata information to the cross-layer partitioner block 228 .
- the source layer multiplexer block 224 may comprise suitable logic, circuitry, and/or code that may be utilized to enable reading data from, for example, the memory block 222 and communicating various portions of the data to the PHY/MAC 226 .
- the data may be split in to the various portions according to information from the cross-layer partitioner block 228 .
- the information from the cross-layer partitioner block 228 may comprise, for example, priority for the various portions of the data. The priority may be based on, for example, perceived importance of the information in the memory block 222 . The number of priorities may be design and/or implementation dependent.
- the cross-layer partitioner block 228 may also indicate that portions of data with certain priority may be communicated via certain outputs of the source layer multiplexer block 224 .
- the PHY/MAC block 226 may comprise suitable logic, circuitry, and/or code that may be utilized to enable conversion of input data in a digital format to output suitably modulated analog data ready for transmission.
- the PHY/MAC block 226 may apply a FEC code to the digital data.
- the PHY/MAC block 226 may also apply a specific RF modulation to the analog signal, which may have been converted from the digital data.
- the PHY/MAC block 226 may additionally communicate analog signals to different transmit antennas 232 a , . . . , 232 b , in a part of a multiple-antenna architecture. Accordingly, the transmission from the transmit antennas 232 a , . . . , 232 b may be a MIMO transmission and/or beamformed transmission.
- the PHY/MAC block 226 may receive one or more streams of digital data.
- the PHY/MAC block 226 may then operate on the multiple streams as indicated by, for example, the parameter control block 230 .
- the PHY/MAC block 226 may, for example, apply a specific FEC code to each digital stream.
- Each digital stream may then be converted to analog RF signal, which may be modulated by a specific RF modulation scheme.
- Each modulated RF signal may then be communicated to one or more antennas to be transmitted.
- the cross-layer partitioner block 228 may comprise suitable logic, circuitry, and/or code that may be utilized to enable assigning a priority to portions of data in the memory block 222 .
- the priority may be based on, for example, perceived importance of the information in the memory block 222 .
- a portion of the data that comprises information about an entire frame, such as, example, an I-frame may have a high priority.
- Other frames, such as, for example, P-frames may have a lower priority than I-frames since P-frames may depend on the I-frames for additional information.
- P-frames that depend on other primary P-frames may be, for example, assigned a lower priority than the P-frames that may only depend on I-frames.
- a B-frame that depends on a prior and subsequent frame may be assigned, for example, a lowest priority.
- the number of priorities may be design and/or implementation dependent.
- the cross-layer partitioner block 228 may also indicate to the source layer multiplexer block 224 that data with certain priorities may be communicated to the PHY/MAC block 226 via specific outputs of the source layer multiplexer block 224 .
- the cross-layer partitioner block 228 may then communicate to the parameter control block 230 those operations that may be performed on the various streams of data communicated by the source layer multiplexer 224 .
- the priority indicated by the cross-layer partitioner block 228 to the source layer multiplexer block 224 may relate to whether a portion of the data being read from the memory 222 corresponds to a media object that is associated with, for example, the foreground or the background layer of an audiovisual scene.
- the cross-layer partitioner block 228 may utilize metadata information to indicate to the source layer multiplexer block 224 to direct the content associated with the media object to an appropriate output for communication to the PHY/MAC 226 .
- the cross-layer partitioner block 228 may then communicate to the parameter control block 230 those operations that may be performed on the various media objects associated with audiovisual scenes in the video data that are communicated by the source layer multiplexer 224 to the PHY/MAC 226 .
- specific streams of data may be communicated to specific transmit antennas.
- the cross-layer partitioner block 228 may have information regarding the propagation path from each transmit antenna 232 a , . . . , 232 b to a receive antenna, where data transmitted via one transmit antenna may be received with fewer bit errors, for example, than data transmitted by another transmit antenna. Accordingly, this information may be used to determine which data may be transmitted via which transmit antenna.
- the information regarding the propagation path for each transmit antenna may be generated, for example, from feedback information from the receiving devices. Alternatively, the information may be generated from feedback information from a receiver co-located with the transmit block 215 .
- the parameter control block 230 may comprise suitable logic, circuitry, and/or code that may be utilized to enable controlling of various operations to the digital data in the PHY/MAC block 226 .
- the parameter control block 230 may determine the FEC code and/or the RF modulation that may be used by the PHY/MAC block 226 for specific portions of data.
- the parameter control block 230 may determine the FEC code and/or the RF modulation that may be utilized for each media object depending on whether the media object is associated with, for example, the foreground or background of an audiovisual scene.
- the parameter control block 230 may also determine which antennas may be used to transmit which portions of data by controlling routing of the data within the PHY/MAC block 226 to the specific antennas.
- some embodiments of the invention may select the antenna used to transmit data by selecting the source layer multiplexer 224 output used to communicate data from the source layer multiplexer 224 to the PHY/MAC 226 .
- Data communicated to the PHY/MAC 226 via specific outputs to the PHY/MAC 226 may be transmitted via specific transmit antennas.
- the data, Output 1 may be transmitted by the transmit antenna 232 a
- the data, Output 2 may be transmitted by the transmit antenna 232 b.
- the source encoder block 220 may compress data and store the compressed data in the memory block 222 .
- the video data may be compressed using, for example, the MPEG-4 format, resulting in the generation of metadata information associated with media objects in the audiovisual scenes in the video data.
- the media objects may be associated with two priority levels, for example, a high priority level associated with foreground image objects and a low priority level associated with background image objects.
- an Output 1 data and Output 2 data may be communicated from the source-layer multiplexer 224 to the PHY/MAC block 226 for transmission by the transmit antennas 232 a and 232 b .
- the source encoder block 220 may communicate to the cross-layer partitioner block 228 information associated with the video data compressed using the MPEG-4 format.
- the source encoder block 220 may also communicate, for example, start and end memory addresses for the stored video data corresponding to a frame, the frame number, and the type of frame that may be stored.
- the type of frame may be, for example, I-frame, P-frame, and B-frame.
- Other information may also be communicated, such as, for example, the chroma sub-sampling format.
- the cross-layer partitioner block 228 may then determine a priority to assign to each media object for the compressed video image.
- the corresponding priority for the media objects in the video data being read from the memory block 222 may be communicated to the source layer multiplexer block 224 .
- the source layer multiplexer 224 may output, for example, high priority data, that is, media objects associated with the foreground of an audiovisual scene, as Output 1 and the low priority data, that is, media objects associated with the background of an audiovisual scene, as Output 2 .
- the cross-layer partitioner block 228 may also communicate to the parameter control block 230 the operations to be applied to each stream of data, namely, Output 1 and Output 2 , based on their corresponding priority.
- the parameter control block 230 may indicate that the high priority data, Output 1 , may have applied to it a forward error correction (FEC) code A that may have a greater overhead in the number of bits used than a FEC code B, which may be applied to Output 2 .
- FEC forward error correction
- using the FEC code A may allow a receiving unit, for example, a mobile terminal such as the mobile terminal 116 a , to correct a larger number of faulty bits than when using the FEC code B.
- the cross-layer partitioner block 228 may also communicate to the parameter control block 230 to use, for example, quadrature phase shift keying (QPSK) RF modulation rather than 16 quadrature amplitude modulation (16 QAM) RF modulation for the high priority data Output 1 .
- QPSK quadrature phase shift keying
- the QPSK RF modulation may have a smaller data throughput than the 16 QAM RF modulation, however, the QPSK RF modulation may be more reliable for a given transmission environment.
- Other exemplary modulation types may comprise binary phase shift keying (BPSK), 64 level QAM (64 QAM), and 256 level QAM (256 QAM).
- the information associated with the media objects may be transmitted using MIMO technology and/or beamforming technology.
- the transmit antenna 232 a may exhibit more reliable transmission characteristics than the transmit antenna 232 b . If the transmission environment changes such that the transmit antenna 232 b exhibits a more reliable transmission characteristics than the transmit antenna 232 a , then the cross-layer partitioner block 228 may indicate that the higher priority data be output as Output 2 .
- both the source-layer multiplexer 224 and the parameter control 230 may receive indications from the cross-layer partitioner block 228 to address the change in priority data output and antenna selection.
- the cross-layer partitioner block 228 may also take in to account feedback information from the receiving device, for example, the mobile terminal 116 a , to maximize throughput for transmission of the high priority and low priority data. This may allow, for example, the cross-layer partitioner block 228 to select from a plurality of FEC codes and from a plurality of RF modulation schemes for a plurality of priority levels. Similarly, MIMO and/or beamforming transmission may allow choosing a transmission method where a plurality of antennas may be selected for transmission of particular stream of data. Various transmission methods are discussed in U.S. application Ser. No. ______ (Attorney Docket No. 17289US01), which is hereby incorporated herein by reference in its entirety.
- feedback information from a receiving device may be used for transmission, the invention need not be so limited.
- feedback data from a receiver that is co-located with the transmitting device may also be used.
- the processor 210 may communicate the feedback data and/or instructions to the transmit block 215 .
- the processor 210 may process the feedback data from a co-located receiving device, and communicate information to the transmit block 215 .
- the information may be used, for example, to control the operations on the data streams by the PHY/MAC block 226 .
- FIG. 2C illustrates an exemplary architecture for source layer optimization with an integrated multiplexer and partitioner for transmitting data, in accordance with an embodiment of the invention.
- the integrated multiplexer and partitioner 240 may comprise suitable logic, circuitry, and/or code that may enable implementing the operations described for the source-layer multiplexer 224 and for the cross-layer partitioner 228 in FIG. 2B , for example.
- the integrated multiplexer and partitioner 240 may be, for example, an application specific integrated circuit wherein a portion enables fast multiplexing operations while another portion provides sufficient processing capabilities to perform the operations associated with the cross-layer partitioner 228 .
- FIG. 3A illustrates an exemplary feedback from a receiver to a transmitter, in accordance with an embodiment of the invention.
- the transmitting terminal 312 may transmit, for example, video data as described with respect to FIG. 2B , and the mobile terminal 310 may receive the transmitted video data.
- the mobile terminal 310 may process the received video data and generate metrics that may indicate, for example, bit error rate and/or signal-to-noise ratio (SNR) for a particular propagation path.
- the propagation path may be, for example, a specific path taken by a transmitted signal from one or more transmit antennas to the mobile terminal. Alternatively, the propagation path may be a combined path where the same data may have been transmitted via a plurality of MIMO transmit antennas.
- the mobile terminal 310 may feed back the received-signal metrics to the transmitting terminal 312 .
- the transmitting terminal 312 may then use the metrics to determine what specific operations may need to be performed by, for example, the PHY/MAC block 226 .
- the mobile terminal 310 may also use an embodiment of the invention in order to optimize throughput during transmission.
- FIG. 3B illustrates an exemplary multiple antenna architecture with feedback from a receiver to a transmitter, in accordance with an embodiment of the invention.
- the transmitting terminal 312 and the mobile terminal 310 may receive the data transmitted by the transmitting terminal 312 .
- the transmitting terminal 312 may transmit signals via the transmit antennas 312 a and 312 b , and the mobile terminal may receive signals via the antennas 310 a and 310 b .
- the transmitting terminal 312 may generate the RF signals tx 1 and tx 2 , which may be transmitted via the transmit antennas 312 a and 312 b , respectively.
- the transmitted RF signals may be represented by s 1 and s 2 .
- the signals received by the receive antennas 310 a and 310 b may be represented by r 1 and r 2 , respectively.
- a receive antenna may receive signals from a plurality of transmit antennas, for example, the transmit antennas 312 a and 312 b .
- the transmit antennas 312 a and 312 b may transmit the same data.
- the transmit antennas 312 a and 312 b may transmit different data.
- the mobile terminal 310 may process the received signals r 1 and r 2 to estimate what information may have been transmitted by the transmitting terminal 312 .
- the mobile terminal 310 may also generate various signal metrics such as, for example, the SNR and bit error rate. The signal metrics may be fed back to the transmitting terminal 312 .
- the transmitting terminal 312 may transmit different data via the transmit antennas 312 a and 312 b .
- the mobile terminal 310 may also process the received signal r 1 for the data transmitted by the transmit antenna 312 a , and the received signal r 2 for the data transmitted by the transmit antenna 312 b . Accordingly, if the bit error rate for the received signal r 1 is less than the bit error rate for the received signal r 2 , this information may be fed back to the transmitting station 312 .
- the transmitting station 312 may then assign, for example, the transmit antenna 312 a for the high priority data and the transmit antenna 312 b for the low priority data.
- the bit error rate and other metrics fed back to the transmitting terminal 312 may be used by the transmitting terminal 312 to select, for example, different FEC codes and/or RF modulation schemes.
- FIG. 3C illustrates an exemplary constellation with four constellation points, which may be utilized in connection with an embodiment of the invention.
- a constellation 320 with four constellation points 320 a , 320 b , 320 c , and 320 d , and a symbol 320 e .
- QPSK modulated RF signals may be received by the mobile terminal 310 and may be demodulated and processed to generate a baseband signal. The resulting signals may be mapped to one of the four constellation points.
- each symbol of a demodulated signal may be mapped directly to one of the four constellation points.
- a symbol may not be able to be mapped directly to a constellation point.
- the symbol 320 e may need to be mapped. Accordingly, the mobile terminal 310 may try to map the symbol 320 e to, for example, the constellation point closest to the symbol. Since the symbol 320 e is the closest to the constellation point 320 a , the symbol 320 e may be mapped to the constellation point 320 a.
- 16 QAM modulation may comprise 16 constellation points. Accordingly, for a given propagation path, data transmitted using the QPSK modulation may allow fewer errors than, for example, data transmitted using the 16 QAM modulation.
- FIG. 3D illustrates an exemplary constellation with 16 constellation points, which may be utilized in connection with an embodiment of the invention.
- a constellation 322 with 16 constellation points and a symbol 322 e Four of these 16 constellation points are labeled 322 a , 322 b , 322 c , and 322 d .
- the mobile terminal 310 may receive 16 QAM modulated RF signals.
- the RF signal may be demodulated and processed to generate a baseband signal.
- the resulting symbols may be mapped to one of the 16 constellation points.
- each symbol of a demodulated signal may map directly to one of the 16 constellation points.
- a symbol may not be able to be mapped directly to a constellation point.
- the symbol 322 e may need to be mapped to a symbol.
- the mobile terminal 310 may try to map the symbol 320 e to, for example, the constellation point closest to the symbol. Since the symbol 322 e is the closest to the constellation point 322 a , the symbol 322 e may be mapped to the constellation point 322 a.
- the likelihood of error may be greater than if there were less than 16 constellation points.
- QPSK modulation as illustrated with respect to FIG. 3C , may comprise four constellation points. Accordingly, for a given propagation path, data transmitted using the 16 QAM modulation may have more errors than, for example, data transmitted using the QPSK modulation.
- selection of an RF modulation scheme may be based on the priority assigned to the media object.
- an spectrally efficient modulation scheme may be utilized for media objects that require a lot of bandwidth, such as background image objects that comprise a lot of detail or high spatial frequency content.
- FIGS. 4A and 4B illustrate an exemplary original image and a reconstruction of the original image after compression, respectively, in connection with an embodiment of the invention.
- a first audiovisual scene or image 402 that corresponds to an original image before compression.
- the first image 402 comprises a foreground and a background, the foreground comprising a media object of the upper torso and face of a person while the background comprises a media object of trees and leaves, for example.
- the background is highly detailed and contains high spatial frequency content that may result in significant amounts of data even after compression.
- a second audiovisual scene or image 404 that corresponds to a reconstruction of the first image 402 after compression.
- the second image 404 shows blockiness in both the background and the foreground that may result from errors that may occur, for example, during the reconstruction process. Blockiness in the foreground is more noticeable, and therefore less desirable, than in the background.
- the forward error correction and/or RF modulation scheme utilized for the transmission of foreground and/or background image objects of an audiovisual scene may be selected in a manner that enables more efficient use of the transmission bandwidth and/or to reduce errors that may result in undesirable artifacts after reconstruction.
- FIG. 5A illustrates an exemplary audiovisual scene comprising a plurality of image objects, in connection with an embodiment of the invention.
- an audiovisual scene 500 may comprise an image object of a person 502 , an image object of a desk 504 , an image object of a globe 506 , and an image object of a blackboard 508 .
- the image objects in the audiovisual scene 500 may be assigned a priority level in accordance with the metadata information generated during the compression operation.
- persons or images of people may be assigned the highest priority level.
- persons or images of people may be said to correspond to the foreground layer of the audiovisual scene 500 .
- the person 502 may correspond to a foreground image object.
- Image objects other than persons or images of people, for example, may be assigned the lowest priority level.
- image objects other than those of persons or images of people may be said to correspond to the background layer of the audiovisual scene 500 .
- the desk 504 , the globe 506 , and the blackboard 508 may correspond to background image objects.
- FIG. 5B illustrates exemplary radio frequency (RF) modulation and forward error correction (FEC) techniques that may be applied to image objects from FIG. 5A based on a priority level, in accordance with an embodiment of the invention.
- RF radio frequency
- FEC forward error correction
- the video data contents corresponding to the desk 504 and to the person 502 may be stored in the memory 222 in FIG. 2B .
- the source encoder 220 may provide metadata information associated with the image objects stored in the memory 222 to the cross-layer partitioner 228 , for example.
- the cross-layer partitioner 228 may indicate to the source-layer multiplexer 224 to stream the contents associated with the person 502 to the higher priority output, Output 1 , and the contents associated with the desk 504 to the lower priority output, Output 2 .
- the cross-layer partitioner 228 may then communicate metadata information to the parameter control 230 to indicate to the PHY/MAC block 226 which FEC code and/or which RF modulation scheme to utilize for the contents in the higher priority output and the lower priority output from the source-layer multiplexer 224 .
- the parameter control 230 may indicate to the PHY/MAC block 226 to forward error correct the contents associated with the desk 504 image object based on an FEC code B that may utilize fewer number of bits for protection than an FEC code A. Selecting an FEC code that utilizes fewer bits for protection may result in reduced bandwidth for background layer image objects. While fewer protection bits may result in a higher number of errors not being corrected, errors in the background layer of an audiovisual scene may be less noticeable than in the foreground layer.
- the parameter control 230 may indicate to the PHY/MAC block 226 to RF modulate the contents associated with the desk 504 image object utilizing the more spectrally efficient 16-QAM, for example, to reduce bandwidth utilization even when a higher number of symbol mapping errors may occur.
- the parameter control 230 may also indicate to the PHY/MAC block 226 to forward error correct the contents associated with the person 502 image object based on the FEC code A that may utilize a larger number of bits for protection than the FEC code B utilized for the desk 504 . Selecting an FEC code that utilizes a larger number of bits for protection may result in higher bandwidth for foreground layer image objects, however, fewer errors may result in fewer noticeable artifacts in the reconstructed image objects in the foreground layer. Similarly, the parameter control 230 may indicate to the PHY/MAC block 226 to RF modulate the contents associated with the person 502 image object utilizing QPSK, for example, to reduce the number of symbol mapping errors that may occur at the expense of higher bandwidth.
- FIG. 6 is a flow diagram illustrating exemplary steps in providing visually related content description to the physical layer for wireless communication, in accordance with an embodiment of the invention.
- a flow diagram 600 After start step 602 , the priority levels associated with media objects in an audiovisual scene may be determined based on the type of metadata information that may be generated from an encoding or compression operation. Multiple priority levels may result from a compression operation. In some instances, a two-priority level may be utilized where the higher priority may be associated with foreground image objects and the lower priority may be associated with background image objects, for example.
- the image objects in an audiovisual scene in video data may be identified and associated with their corresponding metadata information during the video data compression operation.
- each image object identified in the video data may be assigned a priority level that may correspond to a specified or determine FEC code and/or RF modulation scheme.
- the processed media objects, the metadata information, and/or the priority level information may be wirelessly communicated to a remote device for reconstruction of the video data.
- the approach described herein may enable mobile user equipment with more flexible architectures that enable better control of content transmission and that may increase the efficiency with which integrated voice, video, and/or data services may be communicated in higher data rate systems.
- the present invention may be realized in hardware, software, or a combination of hardware and software.
- the present invention may be realized in a centralized fashion in at least one computer system, or in a distributed fashion where different elements are spread across several interconnected computer systems. Any kind of computer system or other apparatus adapted for carrying out the methods described herein is suited.
- a typical combination of hardware and software may be a general-purpose computer system with a computer program that, when being loaded and executed, controls the computer system such that it carries out the methods described herein.
- the present invention may also be embedded in a computer program product, which comprises all the features enabling the implementation of the methods described herein, and which when loaded in a computer system is able to carry out these methods.
- Computer program in the present context means any expression, in any language, code or notation, of a set of instructions intended to cause a system having an information processing capability to perform a particular function either directly or after either or both of the following: a) conversion to another language, code or notation; b) reproduction in a different material form.
Abstract
Description
- This patent application makes reference to:
- U.S. patent application Ser. No. ______ (Attorney Docket No. 17287US01) filed on even date herewith;
U.S. patent application Ser. No. ______ (Attorney Docket No. 17289US01) filed on even date herewith;
U.S. patent application Ser. No. ______ (Attorney Docket No. 17290US01) filed on even date herewith; and
U.S. patent application Ser. No. ______ (Attorney Docket No. 17380US01) filed on even date herewith. - Each of the above stated applications is hereby incorporated by reference in its entirety.
- Certain embodiments of the invention relate to signal handling for wireless communications. More specifically, certain embodiments of the invention relate to a method and system for providing visually related content description to the physical layer.
- Broadcasting and telecommunications have historically occupied separate fields. In the past, broadcasting was largely an “over-the-air” medium while wired media carried telecommunications. That distinction may no longer apply as both broadcasting and telecommunications may be delivered over either wired or wireless media. Present development may adapt broadcasting to mobility services. One limitation for transfer of multimedia data, such as, for example, for digital television, video, digital photo, voice over IP, and even pure voice, has been data transmission rate bottleneck. However, with emerging developments in high-speed wireless communications technology, even this obstacle may be overcome.
- Terrestrial television and radio broadcast networks have made use of high power transmitters covering broad service areas, which enable one-way distribution of content to user equipment such as televisions and radios. By contrast, wireless telecommunications networks have made use of low power transmitters, which have covered relatively small areas known as “cells”. Unlike broadcast networks, wireless networks may be adapted to provide two-way interactive services between subscribers of user equipment such as telephones and computer equipment.
- The introduction of cellular communications systems in the late 1970's and early 1980's represented a significant advance in mobile communications. The networks of this period may be commonly known as first generation, or “1G” systems. These systems were based upon analog, circuit-switching technology, the most prominent of these systems may have been the advanced mobile phone system (AMPS). Second generation, or “2G” systems ushered improvements in performance over 1G systems and introduced digital technology to mobile communications. Exemplary 2G systems include the global system for mobile communications (GSM), digital AMPS (D-AMPS), and code division multiple access (CDMA). Many of these systems have been designed according to the paradigm of the traditional telephony architecture, often focused on circuit-switched services, voice traffic, and supported data transfer rates up to 14.4 kbits/s. Higher data rates were achieved through the deployment of “2.5G” networks, many of which were adapted to existing 2G network infrastructures. The 2.5G networks began the introduction of packet-switching technology in wireless networks. However, it is the evolution of third generation, or “3G” technology that may introduce fully packet-switched networks, which support high-speed data communications.
- Standards for digital television terrestrial broadcasting (DTTB) have evolved around the world with different systems being adopted in different regions. The three leading DTTB systems are, the advanced standards technical committee (ATSC) system, the digital video broadcast terrestrial (DVB-T) system, and the integrated service digital broadcasting terrestrial (ISDB-T) system. The ATSC system has largely been adopted in North America, South America, Taiwan, and South Korea. This system adapts trellis coding and 8-level vestigial sideband (8-VSB) modulation. The DVB-T system has largely been adopted in Europe, the Middle East, Australia, as well as parts of Africa and parts of Asia. The DVB-T system adapts coded orthogonal frequency division multiplexing (COFDM). The OFDM spread spectrum technique may be utilized to distribute information over many carriers that are spaced apart at specified frequencies. The OFDM technique may also be referred to as multi-carrier or discrete multi-tone modulation. The spacing between carriers may prevent the demodulators in a radio receiver from seeing frequencies other than their corresponding frequency. This technique may result in spectral efficiency and lower multi-path distortion, for example. The ISDB-T system has been adopted in Japan and adapts bandwidth segmented transmission orthogonal frequency division multiplexing (BST-OFDM). The various DTTB systems may differ in important aspects; some systems employ a 6 MHz channel separation, while others may employ 7 MHz or 8 MHz channel separations.
- While 3G systems are evolving to provide integrated voice, video, and data services to mobile user equipment, there may be compelling reasons for adapting DTTB systems for this purpose. One of the more notable reasons may be the high data rates that may be supported in DTTB systems. For example, DVB-T may support data rates of 15 Mbits/s in an 8 MHz channel in a wide area SFN. There are also significant challenges in deploying broadcast services to mobile user equipment. Because of form factor constraints, many handheld portable devices, for example, may require that PCB area be minimized and that services consume minimum power to extend battery life to a level that may be acceptable to users. Another consideration is the Doppler effect in moving user equipment, which may cause inter-symbol interference in received signals. Among the three major DTTB systems, ISDB-T was originally designed to support broadcast services to mobile user equipment. While DVB-T may not have been originally designed to support mobility broadcast services, a number of adaptations have been made to provide support for mobile broadcast capability. The adaptation of DVB-T to mobile broadcasting is commonly known as DVB handheld (DVB-H). The broadcasting frequencies for Europe are in UHF (bands IVN) and in the US, the 1670-1675 MHz band that has been allocated for DVB-H operation. Additional spectrum is expected to be allocated in the L-band worldwide.
- To meet requirements for mobile broadcasting the DVB-H specification may support time slicing to reduce power consumption at the user equipment, addition of a 4K mode to enable network operators to make tradeoffs between the advantages of the 2K mode and those of the 8K mode, and an additional level of forward error correction on multi-protocol encapsulated data—forward error correction (MPE-FEC) to make DVB-H transmissions more robust to the challenges presented by mobile reception of signals and to potential limitations in antenna designs for handheld user equipment. DVB-H may also use the DVB-T modulation schemes, like QPSK and 16-quadrature amplitude modulation (16-QAM), which may be more resilient to transmission errors. MPEG audio and video services may be more resilient to error than data, thus additional forward error correction may not be required to meet DTTB service objectives.
- However, the environment for a mobile user equipment, or a mobile terminal, may change as the mobile terminal moves. A signal from a transmitter to the mobile terminal may change in strength as the mobile terminal moves with respect to the transmitter. The signal from the mobile terminal may also take different paths by, for example, reflecting from buildings, trees, bodies of water, the ground, and/or other surfaces. The transmitted signal may also be attenuated when it passes through an object, for example, various glass surfaces in buildings.
- The throughput of a mobile user equipment that may be used, for example, in wireless systems such as those described by the IEEE 802.11 and 802.16 standards, in digital television terrestrial broadcasting, and/or in 3G systems, may need to be optimized based on the services that are provided. In this regard, developing a mobile user equipment with more flexible architectures that enable better control of content transmission may increase the efficiency with which the integrated voice, video, and data services may be communicated in higher data rate systems.
- Further limitations and disadvantages of conventional and traditional approaches will become apparent to one of skill in the art, through comparison of such systems with some aspects of the present invention as set forth in the remainder of the present application with reference to the drawings.
- A system and/or method is provided for providing visually related content description to the physical layer, substantially as shown in and/or described in connection with at least one of the figures, as set forth more completely in the claims.
- These and other advantages, aspects and novel features of the present invention, as well as details of an illustrated embodiment thereof, will be more fully understood from the following description and drawings.
-
FIG. 1 is a block diagram illustrating an exemplary system that may be utilized for content-aware mapping and error protection, in accordance with an embodiment of the invention. -
FIG. 2A illustrates an exemplary conventional architecture for transmitting data. -
FIG. 2B illustrates an exemplary architecture for source layer optimization for transmitting data, in accordance with an embodiment of the invention. -
FIG. 2C illustrates an exemplary architecture for source layer optimization with an integrated multiplexer and partitioner for transmitting data, in accordance with an embodiment of the invention. -
FIG. 3A illustrates an exemplary feedback from a receiver to a transmitter, in accordance with an embodiment of the invention. -
FIG. 3B illustrates an exemplary multiple antenna architecture with feedback from a receiver to a transmitter, in accordance with an embodiment of the invention. -
FIG. 3C illustrates an exemplary constellation with four constellation points, which may be utilized in connection with an embodiment of the invention. -
FIG. 3D illustrates an exemplary constellation with 16 constellation points, which may be utilized in connection with an embodiment of the invention. -
FIGS. 4A and 4B illustrate an exemplary original image and a reconstruction of the original image after compression, respectively, in connection with an embodiment of the invention. -
FIG. 5A illustrates an exemplary audiovisual scene comprising a plurality of image objects, in connection with an embodiment of the invention. -
FIG. 5B illustrates exemplary radio frequency (RF) modulation and forward error correction (FEC) techniques that may be applied to image objects fromFIG. 5A based on a priority, in accordance with an embodiment of the invention. -
FIG. 6 is a flow diagram illustrating exemplary steps in providing visually related content description to the physical layer for wireless communication, in accordance with an embodiment of the invention. - Certain embodiments of the invention may be found in a method and system for providing visually related content description to the physical layer. Aspects of the invention may comprise a MAC layer and/or a PHY layer in a wireless communication device that may be controlled for processing at least one media object in an audiovisual scene based on metadata information that corresponds to the audiovisual scene. The metadata information may result from, for example, MPEG-4 encoding of the audiovisual scene. Each media object may be forward error corrected based on a determined priority level that results from the metadata information. The priority may refer to whether a media object is part of the background or foreground of the audiovisual scene, for example. In this regard, the number of bits utilized for protection may be based on the priority level of the media object. Moreover, each media object may be RF modulated in accordance with its corresponding priority level. The RF modulation may be, for example, QPSK modulation or 16 QAM modulation. The metadata information and the processed media objects may then be wireless communicated to other devices. In some instances, priority obtained from the metadata information may also be utilized to select least one antenna from which to transmit at least a portion of the audiovisual scene, for example.
-
FIG. 1 is a block diagram illustrating an exemplary system that may be utilized for content-aware mapping and error protection, in accordance with an embodiment of the invention. Referring toFIG. 1 , there Q is shownterrestrial network 102, wirelessservice provider network 104,service provider mobile terminals WiFi access point 120, aWiMax transmitter 122, and aBluetooth device 124. Theterrestrial network 102 may comprise transmitter (Tx) 102 a, multiplexer (Mux) 102 b, and information content source 114. The content source 114 may also be referred to as a data carousel, which may comprise audio, data and video content. Theterrestrial network 102 may also compriseDVB antennas mobile terminals DVB antennas service provider network 104 may comprise mobile switching center (MSC) 118 a, and a plurality ofcellular base stations - The
WiFi access point 120 may allow a terminal, such as, for example, themobile terminal WiMax antenna 122 may also allow a terminal, such as, for example, themobile terminal WiFi access point 120 and theWiMax antenna 122 may be serviced by, for example, theservice provider 106 b. Themobile terminal mobile terminal 116 b may communicate with theBluetooth device 124, which may be, for example, a wireless headset for a cellular phone. The network described inFIG. 1 need not be limited to any specific technology and, accordingly, may also support other communication technologies such as wireless metropolitan area networks (WMAN), wireless local area networks (WLAN), and/or wireless personal area networks (WPAN), for example. Accordingly, the mobile terminals may possess the capability to handle any one or more of a plurality of exemplary access technologies such as DVB-H, WCDMA, CDMA, CDMA200, GSM, 802.11, 802.16 and Bluetooth. - The
terrestrial network 102 may comprise suitable equipment that may be enable encoding and/or encryption of data for transmission via thetransmitter 102 a. Thetransmitter 102 a in theterrestrial network 102 may enable utilizing, for example, DVB channels to communicate information to the mobile terminals. In this regard, thetransmitter 102 a may enable DVB-H transmission to the mobile terminals via the UHF band, such as bands IVN, the 1670-1675 MHz band, and/or the L-band, for example. Thetransmitter 102 a may have the capability to determine the type of media that is being communicated and accordingly alter the type of modulation and/or coding used to process the media content so as to provide content-aware mapping and error protection. - Multiple-input-multiple-output (MIMO) communication utilizing multiple antennas in the
transmitter 102 a, themobile terminals transmitter 102 a may also utilize, for example, beamforming to transmit information. The beamforming may also utilize the feedback information provided by themobile terminals - The
multiplexer 102 b associated with theterrestrial network 102 may be utilized to multiplex data from a plurality of sources. For example, themultiplexer 102 b may be adapted to multiplex various types of information such as audio, video and/or data into a single pipe for transmission by thetransmitter 102 a. Content media from the portal 108, which may be handled by theservice provider 106 a may also be multiplexed by themultiplexer 102 b. The portal 108 may be an ISP service provider. Themobile terminals DVB antennas - In one aspect of the invention, the
terrestrial network 102 may enable providing one or more digital television (DTV) channels to theservice provider 106 a. In this regard, theterrestrial network 102 may comprise suitable high-speed or broadband interfaces that may be utilized to facilitate transfer of the DTV channels from theterrestrial network 102 to the service provider. Theservice provider 106 a may then utilize at least a portion of the DTV channels to provide television (TV) on demand service, or other similar types of services to the wirelessservice provider network 104. Accordingly, theservice provider 106 a may further comprise suitable high-speed or broadband interfaces that may be utilized to facilitate the transfer of related TV on demand information to theMSC 118 a. The communication links between theterrestrial network 102 and theservice provider 106 a and the communication links between theservice provider 106 a and thewireless service provider 104 may be wired and/or wireless communication links. - The wireless
service provider network 104 may be a cellular or personal communication service (PCS) provider that may enable broadcasting UMTS (B-UMTS), for example. The term cellular as utilized herein refers to both cellular and PCS frequencies bands. Hence, usage of the term cellular may comprise any band of frequencies that may be utilized for cellular communication and/or any band of frequencies that may be utilized for PCS communication. Notwithstanding, broadcast UMTS (B-UMTS) may also be referred to as MBMS. MBMS is a high-speed data service that is overlaid on WCDMA to provide much higher data rates than may be provided by core WCDMA. In this regard, the B-UMTS services may be superimposed on the cellular or PCS network. - The wireless
service provider network 104 may utilize cellular or PCS access technologies such as GSM, CDMA, CDMA2000, WCDMA, AMPS, N-AMPS, and/or TDMA, for example. The cellular network may be utilized to offer bi-directional services via uplink and downlink communication channels, while the B-UMTS or MBMS network may be utilized to provide a unidirectional broadband services via a downlink channel. In accordance with an embodiment of the invention, content-aware coding, mapping, and/or error protection may be utilized on the downlink. The B-UMTS or MBMS unidirectional downlink channel may be utilized to transmit content media and/or multimedia type information to themobile terminals - The wireless
service provider network 104 need not be limited to a GSM, CDMA, WCDMA based network and/or variants thereof. In this regard, the wirelessservice provider network 104 may be, for example, an 802.11, an 802.16, or a wireless local area network (WLAN). The wirelessservice provider network 104 may also be adapted to provide 802.11 or 802.16 based wireless communication in addition to GSM, CDMA, WCDMA, CDMA2000 based network and/or variants thereof. For example, themobile terminals WiFi access point 120 and/or theWiMax antenna 122. In this case, themobile terminals - The
service provider 106 a may comprise suitable interfaces, circuitry, logic and/or code that may enable communication between theterrestrial network 102 and thewireless communication network 104. Theservice provider 106 a may enable its interfaces to facilitate exchange control information with theterrestrial network 102 and to exchange control information with thewireless service provider 104. The control information exchanged by theservice provider 106 a with theterrestrial network 102 and thewireless communication network 104 may be utilized to control certain operations of the mobile terminals, theterrestrial network 102 and thewireless communication network 104. - The portal 108 may comprise suitable logic, circuitry and/or code that may enable providing content media to the
service provider 106 a via one or more communication links. These communication links, although not shown, may comprise wired and/or wireless communication links. In various exemplary embodiments of the invention, these communication links may utilize any of a plurality of communication access technologies disclosed herein. Other access technologies not disclosed herein may be utilized without departing from the spirit or scope of the invention. The content media that may be provided by the portal 108 may comprise audio, data, video or any combination thereof. In this regard, the portal 108 may provide one or more specialized information services to theservice provider 106 a. - The public switched telephone network (PSTN) 110 may be coupled to the
MSC 118 a. Accordingly, theMSC 118 a may enable switching of calls originating from within thePSTN 110 to one or more mobile terminals serviced by thewireless service provider 104. Similarly, theMSC 118 a may enable switching of calls originating from mobile terminals serviced by thewireless service provider 104 to one or more telephones serviced by thePSTN 110. - The information content source 114 may comprise a data carousel. In this regard, the information content source 114 may provide various information services, which may comprise online data including audio, video and data content. The information content source 114 may also comprise file download, and software download capabilities. In instances where a mobile terminal fails to acquire requested information from the information content source 114 or the requested information is unavailable, then the mobile terminal may acquire the requested information via, for example, B-UMTS from the portal 108. The request may be initiated through an uplink cellular communication path.
- The
mobile terminals mobile terminals mobile terminals base stations H antennas base stations - Transmission of data that requires wide bandwidth, for example, video data, may need to be optimized as to the amount of data transmitted. Accordingly, data may be compressed prior to transmission to reduce the amount of data that needs to be transmitted. The compression may take place, for example, in the
transmitter 102 a. Since errors in the compressed data may lead to inability to decompress the data or to give a result that may affect the decompressed data adversely, data protection methods may be used to enable correction of detected errors. Data protection methods may differ in the number of extra bits that may be used for error detection and correction. Generally, the more bits that are used, the more the data may be protected from uncorrectable errors. However, the extra bits may also reduce the throughput of the data. - Accordingly, different portions of the multimedia information, such as video, audio, or data content, may be given different priority, where the priority may be used to determine the data protection method used to protect the data. Additionally, if MIMO transmission and/or beamforming with a plurality of antennas is used, feedback information, such as performance metrics, from the
mobile terminals - For example, if
antennas antenna 112 a and low priority data, such as audio content, for example, may be transmitted from theantenna 112 b. Even in cases where feedback information may not be available from the receiving mobile terminal, content-aware mapping/error protection, antenna selection, and/or encoding methods may be utilized. In this regard, channel estimations may be utilized for determining the appropriate content-aware mapping/error protection, antenna selection, and/or encoding method, for example. -
FIG. 2A illustrates an exemplary conventional architecture for transmitting data. Referring toFIG. 2A , there is shown asource encoder block 200, amemory block 202, a physical layer/media access control layer (PHY/MAC) block 204, aparameter control block 206, and a transmitantenna 208. Thesource encoder block 200 may comprise suitable logic, circuitry, and/or code that may be utilized to enable, compression of data that is to be transmitted. For example, the compressed data may be video data in MPEG-4 format. - The PHY/MAC block 204 may comprise suitable logic, circuitry, and/or code that may be utilized to enable conversion of input data in a digital format to output suitably modulated RF signal. For example, the PHY/MAC block 204 may apply a forward error correction (FEC) code to the digital data. The PHY/MAC block 204 may also convert the digital data to an analog signal, and then RF modulate the analog signal. The PHY/MAC block 204 may communicate the modulated analog signal to the transmit
antenna 208 for transmission. - The
parameter control block 206 may comprise suitable logic, circuitry, and/or code that may be utilized to enable controlling of various operations on the digital data in the PHY/MAC block 204 before the digital data is output for transmission. For example, theparameter control block 206 may configure the PHY/MAC block 204 to use a specific FEC code and/or RF modulation. - In operation, the
source encoder 200 may, for example, compress video data to MPEG-4 format and store the compressed data in thememory block 202. The PHY/MAC 204 may read portions of the compressed data from thememory block 202. The PHY/MAC 204 may then perform various operations to generate a suitable RF signal that may be transmitted via the transmitantenna 208 to mobile terminals, for example, themobile terminals -
FIG. 2B illustrates an exemplary architecture for source layer optimization for transmitting data, in accordance with an embodiment of the invention. Referring toFIG. 2B , there is shown aprocessor 210 and a transmitblock 215. The transmitblock 215 may comprise asource encoder block 220, amemory block 222, a sourcelayer multiplexer block 224, a PHY/MAC block 226, across-layer partitioner block 228, aparameter control block 230, and transmitantennas 232 a, . . . , 232 b. The transmitblock 215 may be, for example, part of thecellular base stations - The
source encoder block 220 may comprise suitable logic, circuitry, and/or code that may be utilized to enable compression of data prior to transmission. For example, the compressed data may be video data in MPEG-4 format. Thesource encoder block 220 may also communicate information about the compressed data to thecross-layer partitioner block 228. The information communicated may relate to the type of compression. For example, if the compressed data comprises video data, thesource encoder block 220 may communicate the specific type of compression, such as MPEG-1, MPEG-2, MPEG-4, H.261, H.263, or H.264. Thesource encoder block 220 may also communicate the type of chroma subsampling used, such as, for example, 4-4-4, 4-2-2, or 4-2-0 chroma subsampling. - When comprising video data, for example, the source encoder may generate metadata information associated with the contents of the video data. For example, for MPEG-4 format, the source encoder may generate metadata information associated with media objects in an audiovisual scene comprised within the video stream. The metadata information may indicate, for example, whether a media object is associated with the foreground or the background of the audiovisual scene. The metadata information need not be so limited and may also indicate other visually related content description of the audiovisual scene. Moreover, the audiovisual scene need not be limited to having foreground and background layers of the scene but may comprise a plurality of descriptive layers to which a media object may be associated.
- Other video encoding or compression operations, such as MPEG-2 format, for example, may also be expanded to generate metadata information describing visually related content. In this regard, the compressed video data may comprise the metadata information associated with audiovisual scenes in the video data. The
source encoder block 220 may be enabled to communicate information about the metadata information to thecross-layer partitioner block 228. - The source
layer multiplexer block 224 may comprise suitable logic, circuitry, and/or code that may be utilized to enable reading data from, for example, thememory block 222 and communicating various portions of the data to the PHY/MAC 226. The data may be split in to the various portions according to information from thecross-layer partitioner block 228. The information from thecross-layer partitioner block 228 may comprise, for example, priority for the various portions of the data. The priority may be based on, for example, perceived importance of the information in thememory block 222. The number of priorities may be design and/or implementation dependent. Thecross-layer partitioner block 228 may also indicate that portions of data with certain priority may be communicated via certain outputs of the sourcelayer multiplexer block 224. - The PHY/MAC block 226 may comprise suitable logic, circuitry, and/or code that may be utilized to enable conversion of input data in a digital format to output suitably modulated analog data ready for transmission. For example, the PHY/MAC block 226 may apply a FEC code to the digital data. The PHY/MAC block 226 may also apply a specific RF modulation to the analog signal, which may have been converted from the digital data. The PHY/MAC block 226 may additionally communicate analog signals to different transmit
antennas 232 a, . . . , 232 b, in a part of a multiple-antenna architecture. Accordingly, the transmission from the transmitantennas 232 a, . . . , 232 b may be a MIMO transmission and/or beamformed transmission. - In an embodiment of the invention, the PHY/MAC block 226 may receive one or more streams of digital data. The PHY/MAC block 226 may then operate on the multiple streams as indicated by, for example, the
parameter control block 230. Accordingly, the PHY/MAC block 226 may, for example, apply a specific FEC code to each digital stream. Each digital stream may then be converted to analog RF signal, which may be modulated by a specific RF modulation scheme. Each modulated RF signal may then be communicated to one or more antennas to be transmitted. - The
cross-layer partitioner block 228 may comprise suitable logic, circuitry, and/or code that may be utilized to enable assigning a priority to portions of data in thememory block 222. The priority may be based on, for example, perceived importance of the information in thememory block 222. For example, if the data in thememory block 222 comprises video data relating to video frames, a portion of the data that comprises information about an entire frame, such as, example, an I-frame, may have a high priority. Other frames, such as, for example, P-frames may have a lower priority than I-frames since P-frames may depend on the I-frames for additional information. P-frames that depend on other primary P-frames may be, for example, assigned a lower priority than the P-frames that may only depend on I-frames. A B-frame that depends on a prior and subsequent frame may be assigned, for example, a lowest priority. The number of priorities may be design and/or implementation dependent. - The
cross-layer partitioner block 228 may also indicate to the sourcelayer multiplexer block 224 that data with certain priorities may be communicated to the PHY/MAC block 226 via specific outputs of the sourcelayer multiplexer block 224. Thecross-layer partitioner block 228 may then communicate to theparameter control block 230 those operations that may be performed on the various streams of data communicated by thesource layer multiplexer 224. - For compressed video data applications, for example, the priority indicated by the
cross-layer partitioner block 228 to the sourcelayer multiplexer block 224 may relate to whether a portion of the data being read from thememory 222 corresponds to a media object that is associated with, for example, the foreground or the background layer of an audiovisual scene. In this regard, thecross-layer partitioner block 228 may utilize metadata information to indicate to the sourcelayer multiplexer block 224 to direct the content associated with the media object to an appropriate output for communication to the PHY/MAC 226. Thecross-layer partitioner block 228 may then communicate to theparameter control block 230 those operations that may be performed on the various media objects associated with audiovisual scenes in the video data that are communicated by thesource layer multiplexer 224 to the PHY/MAC 226. - In some instances, specific streams of data may be communicated to specific transmit antennas. The
cross-layer partitioner block 228 may have information regarding the propagation path from each transmitantenna 232 a, . . . , 232 b to a receive antenna, where data transmitted via one transmit antenna may be received with fewer bit errors, for example, than data transmitted by another transmit antenna. Accordingly, this information may be used to determine which data may be transmitted via which transmit antenna. The information regarding the propagation path for each transmit antenna may be generated, for example, from feedback information from the receiving devices. Alternatively, the information may be generated from feedback information from a receiver co-located with the transmitblock 215. - The
parameter control block 230 may comprise suitable logic, circuitry, and/or code that may be utilized to enable controlling of various operations to the digital data in the PHY/MAC block 226. For example, theparameter control block 230 may determine the FEC code and/or the RF modulation that may be used by the PHY/MAC block 226 for specific portions of data. For example, for compressed video data with metadata information available, theparameter control block 230 may determine the FEC code and/or the RF modulation that may be utilized for each media object depending on whether the media object is associated with, for example, the foreground or background of an audiovisual scene. Theparameter control block 230 may also determine which antennas may be used to transmit which portions of data by controlling routing of the data within the PHY/MAC block 226 to the specific antennas. - However, there may be other embodiments of the invention that route signals to specific antennas using other methods. For example, some embodiments of the invention may select the antenna used to transmit data by selecting the
source layer multiplexer 224 output used to communicate data from thesource layer multiplexer 224 to the PHY/MAC 226. Data communicated to the PHY/MAC 226 via specific outputs to the PHY/MAC 226 may be transmitted via specific transmit antennas. For example, the data, Output1, may be transmitted by the transmitantenna 232 a, and the data, Output2, may be transmitted by the transmitantenna 232 b. - In operation, the
source encoder block 220 may compress data and store the compressed data in thememory block 222. The video data may be compressed using, for example, the MPEG-4 format, resulting in the generation of metadata information associated with media objects in the audiovisual scenes in the video data. The media objects may be associated with two priority levels, for example, a high priority level associated with foreground image objects and a low priority level associated with background image objects. As a result of a two-priority level system, an Output1 data and Output2 data may be communicated from the source-layer multiplexer 224 to the PHY/MAC block 226 for transmission by the transmitantennas source encoder block 220 may communicate to thecross-layer partitioner block 228 information associated with the video data compressed using the MPEG-4 format. Thesource encoder block 220 may also communicate, for example, start and end memory addresses for the stored video data corresponding to a frame, the frame number, and the type of frame that may be stored. The type of frame may be, for example, I-frame, P-frame, and B-frame. Other information may also be communicated, such as, for example, the chroma sub-sampling format. - The
cross-layer partitioner block 228 may then determine a priority to assign to each media object for the compressed video image. The corresponding priority for the media objects in the video data being read from thememory block 222 may be communicated to the sourcelayer multiplexer block 224. Thesource layer multiplexer 224 may output, for example, high priority data, that is, media objects associated with the foreground of an audiovisual scene, as Output1 and the low priority data, that is, media objects associated with the background of an audiovisual scene, as Output2. - The
cross-layer partitioner block 228 may also communicate to theparameter control block 230 the operations to be applied to each stream of data, namely, Output1 and Output2, based on their corresponding priority. For example, theparameter control block 230 may indicate that the high priority data, Output1, may have applied to it a forward error correction (FEC) code A that may have a greater overhead in the number of bits used than a FEC code B, which may be applied to Output 2. However, using the FEC code A may allow a receiving unit, for example, a mobile terminal such as themobile terminal 116 a, to correct a larger number of faulty bits than when using the FEC code B. - The
cross-layer partitioner block 228 may also communicate to theparameter control block 230 to use, for example, quadrature phase shift keying (QPSK) RF modulation rather than 16 quadrature amplitude modulation (16 QAM) RF modulation for the high priority data Output1. The QPSK RF modulation may have a smaller data throughput than the 16 QAM RF modulation, however, the QPSK RF modulation may be more reliable for a given transmission environment. Other exemplary modulation types may comprise binary phase shift keying (BPSK), 64 level QAM (64 QAM), and 256 level QAM (256 QAM). - In some instances, the information associated with the media objects may be transmitted using MIMO technology and/or beamforming technology. For example, the transmit
antenna 232 a may exhibit more reliable transmission characteristics than the transmitantenna 232 b. If the transmission environment changes such that the transmitantenna 232 b exhibits a more reliable transmission characteristics than the transmitantenna 232 a, then thecross-layer partitioner block 228 may indicate that the higher priority data be output as Output2. In this regard, both the source-layer multiplexer 224 and theparameter control 230 may receive indications from thecross-layer partitioner block 228 to address the change in priority data output and antenna selection. - The
cross-layer partitioner block 228 may also take in to account feedback information from the receiving device, for example, themobile terminal 116 a, to maximize throughput for transmission of the high priority and low priority data. This may allow, for example, thecross-layer partitioner block 228 to select from a plurality of FEC codes and from a plurality of RF modulation schemes for a plurality of priority levels. Similarly, MIMO and/or beamforming transmission may allow choosing a transmission method where a plurality of antennas may be selected for transmission of particular stream of data. Various transmission methods are discussed in U.S. application Ser. No. ______ (Attorney Docket No. 17289US01), which is hereby incorporated herein by reference in its entirety. - Although feedback information from a receiving device may be used for transmission, the invention need not be so limited. For example, feedback data from a receiver that is co-located with the transmitting device may also be used. Accordingly, for example, the
processor 210 may communicate the feedback data and/or instructions to the transmitblock 215. For example, theprocessor 210 may process the feedback data from a co-located receiving device, and communicate information to the transmitblock 215. The information may be used, for example, to control the operations on the data streams by the PHY/MAC block 226. - Although an embodiment of the invention may have been described using a plurality of functional blocks, the invention need not be so limited. Accordingly, other embodiments of the invention may use different blocks that may encompass various functionalities.
-
FIG. 2C illustrates an exemplary architecture for source layer optimization with an integrated multiplexer and partitioner for transmitting data, in accordance with an embodiment of the invention. Referring toFIG. 2C , there is shown the transmitblock 215 inFIG. 2B with an integrated multiplexer andpartitioner 240. The integrated multiplexer andpartitioner 240 may comprise suitable logic, circuitry, and/or code that may enable implementing the operations described for the source-layer multiplexer 224 and for thecross-layer partitioner 228 inFIG. 2B , for example. The integrated multiplexer andpartitioner 240 may be, for example, an application specific integrated circuit wherein a portion enables fast multiplexing operations while another portion provides sufficient processing capabilities to perform the operations associated with thecross-layer partitioner 228. -
FIG. 3A illustrates an exemplary feedback from a receiver to a transmitter, in accordance with an embodiment of the invention. Referring toFIG. 3A , there is shown amobile terminal 310 and a transmittingterminal 312. The transmittingterminal 312 may transmit, for example, video data as described with respect toFIG. 2B , and themobile terminal 310 may receive the transmitted video data. Themobile terminal 310 may process the received video data and generate metrics that may indicate, for example, bit error rate and/or signal-to-noise ratio (SNR) for a particular propagation path. The propagation path may be, for example, a specific path taken by a transmitted signal from one or more transmit antennas to the mobile terminal. Alternatively, the propagation path may be a combined path where the same data may have been transmitted via a plurality of MIMO transmit antennas. - The
mobile terminal 310 may feed back the received-signal metrics to the transmittingterminal 312. The transmittingterminal 312 may then use the metrics to determine what specific operations may need to be performed by, for example, the PHY/MAC block 226. Although an embodiment of the invention may have been described with respect to the transmittingterminal 312, the invention need not be so limited. For example, themobile terminal 310 may also use an embodiment of the invention in order to optimize throughput during transmission. -
FIG. 3B illustrates an exemplary multiple antenna architecture with feedback from a receiver to a transmitter, in accordance with an embodiment of the invention. Referring toFIG. 3B , there is shown the transmittingterminal 312 and themobile terminal 310, which may receive the data transmitted by the transmittingterminal 312. The transmittingterminal 312 may transmit signals via the transmitantennas antennas terminal 312 may generate the RF signals tx1 and tx2, which may be transmitted via the transmitantennas antennas - In operation, a receive antenna, for example, the receive
antenna 310 a, may receive signals from a plurality of transmit antennas, for example, the transmitantennas antennas antennas mobile terminal 310 may process the received signals r1 and r2 to estimate what information may have been transmitted by the transmittingterminal 312. Themobile terminal 310 may also generate various signal metrics such as, for example, the SNR and bit error rate. The signal metrics may be fed back to the transmittingterminal 312. - For example, the transmitting
terminal 312 may transmit different data via the transmitantennas mobile terminal 310 may also process the received signal r1 for the data transmitted by the transmitantenna 312 a, and the received signal r2 for the data transmitted by the transmitantenna 312 b. Accordingly, if the bit error rate for the received signal r1 is less than the bit error rate for the received signal r2, this information may be fed back to the transmittingstation 312. The transmittingstation 312 may then assign, for example, the transmitantenna 312 a for the high priority data and the transmitantenna 312 b for the low priority data. Similarly, the bit error rate and other metrics fed back to the transmittingterminal 312 may be used by the transmittingterminal 312 to select, for example, different FEC codes and/or RF modulation schemes. -
FIG. 3C illustrates an exemplary constellation with four constellation points, which may be utilized in connection with an embodiment of the invention. Referring toFIG. 3C , there is shown aconstellation 320 with fourconstellation points symbol 320 e. QPSK modulated RF signals may be received by themobile terminal 310 and may be demodulated and processed to generate a baseband signal. The resulting signals may be mapped to one of the four constellation points. - In one aspect of the invention, each symbol of a demodulated signal may be mapped directly to one of the four constellation points. However, because of noise in the propagation path, and interference from other RF sources, among other factors, a symbol may not be able to be mapped directly to a constellation point. For example, the
symbol 320 e may need to be mapped. Accordingly, themobile terminal 310 may try to map thesymbol 320 e to, for example, the constellation point closest to the symbol. Since thesymbol 320 e is the closest to theconstellation point 320 a, thesymbol 320 e may be mapped to theconstellation point 320 a. - With respect to
FIG. 3C , since there may only be four possibilities for mapping, the likelihood of error may be less than if there were more than four constellation points. For example, 16 QAM modulation, as illustrated with respect toFIG. 3D , may comprise 16 constellation points. Accordingly, for a given propagation path, data transmitted using the QPSK modulation may allow fewer errors than, for example, data transmitted using the 16 QAM modulation. -
FIG. 3D illustrates an exemplary constellation with 16 constellation points, which may be utilized in connection with an embodiment of the invention. Referring toFIG. 3D , there is shown aconstellation 322 with 16 constellation points and asymbol 322 e. Four of these 16 constellation points are labeled 322 a, 322 b, 322 c, and 322 d. Themobile terminal 310 may receive 16 QAM modulated RF signals. The RF signal may be demodulated and processed to generate a baseband signal. The resulting symbols may be mapped to one of the 16 constellation points. - In one aspect of the invention, each symbol of a demodulated signal may map directly to one of the 16 constellation points. However, because of noise in the propagation path, and interference from other RF sources, among other factors, a symbol may not be able to be mapped directly to a constellation point. For example, the
symbol 322 e may need to be mapped to a symbol. Accordingly, themobile terminal 310 may try to map thesymbol 320 e to, for example, the constellation point closest to the symbol. Since thesymbol 322 e is the closest to theconstellation point 322 a, thesymbol 322 e may be mapped to theconstellation point 322 a. - With respect to
FIG. 3D , since there may be 16 possibilities for mapping, the likelihood of error may be greater than if there were less than 16 constellation points. For example, QPSK modulation, as illustrated with respect toFIG. 3C , may comprise four constellation points. Accordingly, for a given propagation path, data transmitted using the 16 QAM modulation may have more errors than, for example, data transmitted using the QPSK modulation. - For media objects in an audiovisual scene, for example, selection of an RF modulation scheme, whether 16 QAM or QPSK modulation, may be based on the priority assigned to the media object. For example, an spectrally efficient modulation scheme may be utilized for media objects that require a lot of bandwidth, such as background image objects that comprise a lot of detail or high spatial frequency content.
-
FIGS. 4A and 4B illustrate an exemplary original image and a reconstruction of the original image after compression, respectively, in connection with an embodiment of the invention. Referring toFIG. 4A , there is shown a first audiovisual scene orimage 402 that corresponds to an original image before compression. Thefirst image 402 comprises a foreground and a background, the foreground comprising a media object of the upper torso and face of a person while the background comprises a media object of trees and leaves, for example. The background is highly detailed and contains high spatial frequency content that may result in significant amounts of data even after compression. - Referring to
FIG. 4B , there is shown a second audiovisual scene orimage 404 that corresponds to a reconstruction of thefirst image 402 after compression. Thesecond image 404 shows blockiness in both the background and the foreground that may result from errors that may occur, for example, during the reconstruction process. Blockiness in the foreground is more noticeable, and therefore less desirable, than in the background. In this regard, the forward error correction and/or RF modulation scheme utilized for the transmission of foreground and/or background image objects of an audiovisual scene may be selected in a manner that enables more efficient use of the transmission bandwidth and/or to reduce errors that may result in undesirable artifacts after reconstruction. -
FIG. 5A illustrates an exemplary audiovisual scene comprising a plurality of image objects, in connection with an embodiment of the invention. Referring toFIG. 5 , there is shown anaudiovisual scene 500 that may comprise an image object of aperson 502, an image object of adesk 504, an image object of aglobe 506, and an image object of ablackboard 508. The image objects in theaudiovisual scene 500 may be assigned a priority level in accordance with the metadata information generated during the compression operation. In one embodiment of the invention, persons or images of people may be assigned the highest priority level. In this regard, for a two-priority level system comprising foreground and background layers, persons or images of people may be said to correspond to the foreground layer of theaudiovisual scene 500. For example, theperson 502 may correspond to a foreground image object. Image objects, other than persons or images of people, for example, may be assigned the lowest priority level. In this regard, for a two-priority level system comprising foreground and background layers, image objects other than those of persons or images of people may be said to correspond to the background layer of theaudiovisual scene 500. For example, thedesk 504, theglobe 506, and theblackboard 508 may correspond to background image objects. -
FIG. 5B illustrates exemplary radio frequency (RF) modulation and forward error correction (FEC) techniques that may be applied to image objects fromFIG. 5A based on a priority level, in accordance with an embodiment of the invention. Referring toFIG. 5B , the video data contents corresponding to thedesk 504 and to theperson 502 may be stored in thememory 222 inFIG. 2B . The source encoder 220 may provide metadata information associated with the image objects stored in thememory 222 to thecross-layer partitioner 228, for example. When the video data contents associated with thedesk 504 and theperson 502 are read from thememory 222, thecross-layer partitioner 228 may indicate to the source-layer multiplexer 224 to stream the contents associated with theperson 502 to the higher priority output,Output 1, and the contents associated with thedesk 504 to the lower priority output, Output 2. Thecross-layer partitioner 228 may then communicate metadata information to theparameter control 230 to indicate to the PHY/MAC block 226 which FEC code and/or which RF modulation scheme to utilize for the contents in the higher priority output and the lower priority output from the source-layer multiplexer 224. - In an embodiment of the invention, the
parameter control 230 may indicate to the PHY/MAC block 226 to forward error correct the contents associated with thedesk 504 image object based on an FEC code B that may utilize fewer number of bits for protection than an FEC code A. Selecting an FEC code that utilizes fewer bits for protection may result in reduced bandwidth for background layer image objects. While fewer protection bits may result in a higher number of errors not being corrected, errors in the background layer of an audiovisual scene may be less noticeable than in the foreground layer. Similarly, theparameter control 230 may indicate to the PHY/MAC block 226 to RF modulate the contents associated with thedesk 504 image object utilizing the more spectrally efficient 16-QAM, for example, to reduce bandwidth utilization even when a higher number of symbol mapping errors may occur. - The
parameter control 230 may also indicate to the PHY/MAC block 226 to forward error correct the contents associated with theperson 502 image object based on the FEC code A that may utilize a larger number of bits for protection than the FEC code B utilized for thedesk 504. Selecting an FEC code that utilizes a larger number of bits for protection may result in higher bandwidth for foreground layer image objects, however, fewer errors may result in fewer noticeable artifacts in the reconstructed image objects in the foreground layer. Similarly, theparameter control 230 may indicate to the PHY/MAC block 226 to RF modulate the contents associated with theperson 502 image object utilizing QPSK, for example, to reduce the number of symbol mapping errors that may occur at the expense of higher bandwidth. -
FIG. 6 is a flow diagram illustrating exemplary steps in providing visually related content description to the physical layer for wireless communication, in accordance with an embodiment of the invention. Referring toFIG. 6 , there is shown a flow diagram 600. Afterstart step 602, the priority levels associated with media objects in an audiovisual scene may be determined based on the type of metadata information that may be generated from an encoding or compression operation. Multiple priority levels may result from a compression operation. In some instances, a two-priority level may be utilized where the higher priority may be associated with foreground image objects and the lower priority may be associated with background image objects, for example. - In
step 606, the image objects in an audiovisual scene in video data may be identified and associated with their corresponding metadata information during the video data compression operation. Instep 608, each image object identified in the video data may be assigned a priority level that may correspond to a specified or determine FEC code and/or RF modulation scheme. Instep 610, the processed media objects, the metadata information, and/or the priority level information may be wirelessly communicated to a remote device for reconstruction of the video data. - The approach described herein may enable mobile user equipment with more flexible architectures that enable better control of content transmission and that may increase the efficiency with which integrated voice, video, and/or data services may be communicated in higher data rate systems.
- Accordingly, the present invention may be realized in hardware, software, or a combination of hardware and software. The present invention may be realized in a centralized fashion in at least one computer system, or in a distributed fashion where different elements are spread across several interconnected computer systems. Any kind of computer system or other apparatus adapted for carrying out the methods described herein is suited. A typical combination of hardware and software may be a general-purpose computer system with a computer program that, when being loaded and executed, controls the computer system such that it carries out the methods described herein.
- The present invention may also be embedded in a computer program product, which comprises all the features enabling the implementation of the methods described herein, and which when loaded in a computer system is able to carry out these methods. Computer program in the present context means any expression, in any language, code or notation, of a set of instructions intended to cause a system having an information processing capability to perform a particular function either directly or after either or both of the following: a) conversion to another language, code or notation; b) reproduction in a different material form.
- While the present invention has been described with reference to certain embodiments, it will be understood by those skilled in the art that various changes may be made and equivalents may be substituted without departing from the scope of the present invention. In addition, many modifications may be made to adapt a particular situation or material to the teachings of the present invention without departing from its scope. Therefore, it is intended that the present invention not be limited to the particular embodiment disclosed, but that the present invention will include all embodiments falling within the scope of the appended claims.
Claims (21)
Priority Applications (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US11/492,667 US8917674B2 (en) | 2006-07-25 | 2006-07-25 | Method and system for content-aware mapping/error protection |
US11/492,391 US20080025196A1 (en) | 2006-07-25 | 2006-07-25 | Method and system for providing visually related content description to the physical layer |
US13/846,776 US9066220B2 (en) | 2006-07-25 | 2013-03-18 | Method and system for distributed infrastructure for streaming data via multiple access points |
US14/661,650 US20150229369A1 (en) | 2006-07-25 | 2015-03-18 | Method and system for content-aware mapping/error protection using different spatial streams |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US11/492,391 US20080025196A1 (en) | 2006-07-25 | 2006-07-25 | Method and system for providing visually related content description to the physical layer |
Publications (1)
Publication Number | Publication Date |
---|---|
US20080025196A1 true US20080025196A1 (en) | 2008-01-31 |
Family
ID=38986143
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US11/492,391 Abandoned US20080025196A1 (en) | 2006-07-25 | 2006-07-25 | Method and system for providing visually related content description to the physical layer |
Country Status (1)
Country | Link |
---|---|
US (1) | US20080025196A1 (en) |
Cited By (22)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20090017813A1 (en) * | 2007-07-10 | 2009-01-15 | Kabushiki Kaisha Toshiba | Information processing apparatus |
EP2091205A1 (en) * | 2008-02-14 | 2009-08-19 | NTT DoCoMo, Inc. | Method and an apparatus for delivering a video stream |
US20090213935A1 (en) * | 2002-12-10 | 2009-08-27 | Van Der Laan Roger | System and Method For Compressing Video By Allocating Bits To Image Tiles Based On Detected Intraframe Motion Or Scene Complexity |
US20090220002A1 (en) * | 2002-12-10 | 2009-09-03 | Laan Roger Van Der | System and method for compressing video based on detected intraframe motion |
US20090225220A1 (en) * | 2002-12-10 | 2009-09-10 | Van Der Laan Roger | System and Method For Compressing Video By Adjusting Tile Size Based On Detected Intraframe Motion Or Scene Complexity |
US20100167816A1 (en) * | 2002-12-10 | 2010-07-01 | Perlman Stephen G | System and Method for Multi-Stream Video Compression |
US20100166066A1 (en) * | 2002-12-10 | 2010-07-01 | Steve Perlman | System and Method for Video Compression Using Feedback Including Data Related to the Successful Receipt of Video Content |
US20100167809A1 (en) * | 2002-12-10 | 2010-07-01 | Perlman Steve G | System and Method for Accelerated Machine Switching |
US20100166063A1 (en) * | 2002-12-10 | 2010-07-01 | Perlman Steve G | System and method for compressing video frames or portions thereof based on feedback information from a client device |
US20100166064A1 (en) * | 2002-12-10 | 2010-07-01 | Perlman Steve G | System and Method for Utilizing Forward Error Correction with Video Compression |
US20110310769A1 (en) * | 2010-06-18 | 2011-12-22 | Via Telecom, Inc. | Methods and Apparatuses for Machine Type Communication |
US8711923B2 (en) | 2002-12-10 | 2014-04-29 | Ol2, Inc. | System and method for selecting a video encoding format based on feedback data |
US8769594B2 (en) | 2002-12-10 | 2014-07-01 | Ol2, Inc. | Video compression system and method for reducing the effects of packet loss over a communication channel |
CN104170488A (en) * | 2012-03-16 | 2014-11-26 | 瑞典爱立信有限公司 | Method, network node, computer program and computer program product for determining resource allocation between downlink and uplink |
US8964830B2 (en) | 2002-12-10 | 2015-02-24 | Ol2, Inc. | System and method for multi-stream video compression using multiple encoding formats |
US9061207B2 (en) | 2002-12-10 | 2015-06-23 | Sony Computer Entertainment America Llc | Temporary decoder apparatus and method |
US9084936B2 (en) | 2002-12-10 | 2015-07-21 | Sony Computer Entertainment America Llc | System and method for protecting certain types of multimedia data transmitted over a communication channel |
US9168457B2 (en) | 2010-09-14 | 2015-10-27 | Sony Computer Entertainment America Llc | System and method for retaining system state |
US9192859B2 (en) | 2002-12-10 | 2015-11-24 | Sony Computer Entertainment America Llc | System and method for compressing video based on latency measurements and other feedback |
US9446305B2 (en) | 2002-12-10 | 2016-09-20 | Sony Interactive Entertainment America Llc | System and method for improving the graphics performance of hosted applications |
GB2550877A (en) * | 2016-05-26 | 2017-12-06 | Univ Surrey | Object-based audio rendering |
US20190373270A1 (en) * | 2018-05-31 | 2019-12-05 | Apple Inc. | Efficient coding of video data in the presence of video annotations |
Citations (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20020006137A1 (en) * | 2000-05-08 | 2002-01-17 | Rabenko Theodore F. | System and method for supporting multiple voice channels |
US20020122395A1 (en) * | 2001-03-05 | 2002-09-05 | Yair Bourlas | Method and apparatus for implementing a mac coprocessor in a communication system |
US20030053680A1 (en) * | 2001-09-17 | 2003-03-20 | Koninklijke Philips Electronics N.V. | Three-dimensional sound creation assisted by visual information |
US20050249240A1 (en) * | 2002-06-11 | 2005-11-10 | Boyce Jill M | Multimedia server with simple adaptation to dynamic network loss conditions |
US20060092963A1 (en) * | 2004-10-28 | 2006-05-04 | Ajay Bakre | Architecture and method for efficient application of QoS in a WLAN |
US20060133414A1 (en) * | 2004-12-22 | 2006-06-22 | Juha-Pekka Luoma | Wireless gateway for enabling wireless devices to discover and interact with various short-range services/devices |
US20060282874A1 (en) * | 1998-12-08 | 2006-12-14 | Canon Kabushiki Kaisha | Receiving apparatus and method |
US7206456B2 (en) * | 2001-11-27 | 2007-04-17 | Nokia Corporation | Video coding and decoding |
US20070234385A1 (en) * | 2006-03-31 | 2007-10-04 | Rajendra Bopardikar | Cross-layer video quality manager |
US7852368B2 (en) * | 2005-03-16 | 2010-12-14 | Lg Electronics Inc. | Method and apparatus for composing images during video communications |
-
2006
- 2006-07-25 US US11/492,391 patent/US20080025196A1/en not_active Abandoned
Patent Citations (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20060282874A1 (en) * | 1998-12-08 | 2006-12-14 | Canon Kabushiki Kaisha | Receiving apparatus and method |
US20020006137A1 (en) * | 2000-05-08 | 2002-01-17 | Rabenko Theodore F. | System and method for supporting multiple voice channels |
US20020122395A1 (en) * | 2001-03-05 | 2002-09-05 | Yair Bourlas | Method and apparatus for implementing a mac coprocessor in a communication system |
US20030053680A1 (en) * | 2001-09-17 | 2003-03-20 | Koninklijke Philips Electronics N.V. | Three-dimensional sound creation assisted by visual information |
US7206456B2 (en) * | 2001-11-27 | 2007-04-17 | Nokia Corporation | Video coding and decoding |
US20050249240A1 (en) * | 2002-06-11 | 2005-11-10 | Boyce Jill M | Multimedia server with simple adaptation to dynamic network loss conditions |
US20060092963A1 (en) * | 2004-10-28 | 2006-05-04 | Ajay Bakre | Architecture and method for efficient application of QoS in a WLAN |
US20060133414A1 (en) * | 2004-12-22 | 2006-06-22 | Juha-Pekka Luoma | Wireless gateway for enabling wireless devices to discover and interact with various short-range services/devices |
US7852368B2 (en) * | 2005-03-16 | 2010-12-14 | Lg Electronics Inc. | Method and apparatus for composing images during video communications |
US20070234385A1 (en) * | 2006-03-31 | 2007-10-04 | Rajendra Bopardikar | Cross-layer video quality manager |
Cited By (48)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9314691B2 (en) | 2002-12-10 | 2016-04-19 | Sony Computer Entertainment America Llc | System and method for compressing video frames or portions thereof based on feedback information from a client device |
US10130891B2 (en) | 2002-12-10 | 2018-11-20 | Sony Interactive Entertainment America Llc | Video compression system and method for compensating for bandwidth limitations of a communication channel |
US20090213935A1 (en) * | 2002-12-10 | 2009-08-27 | Van Der Laan Roger | System and Method For Compressing Video By Allocating Bits To Image Tiles Based On Detected Intraframe Motion Or Scene Complexity |
US20090215540A1 (en) * | 2002-12-10 | 2009-08-27 | Perlman Stephen G | System and Method for Intelligently Allocating Client Requests to Server Centers |
US20090215531A1 (en) * | 2002-12-10 | 2009-08-27 | Perlman Stephen G | System and Method for Storing Program Code and Data Within an Application Hosting Center |
US20090213927A1 (en) * | 2002-12-10 | 2009-08-27 | Perlman Stephen G | System and Method for Compressing Video Based on Detected Data Rate of a Communication Channel |
US20090220001A1 (en) * | 2002-12-10 | 2009-09-03 | Van Der Laan Roger | Tile-Based System and method For Compressing Video |
US20090220002A1 (en) * | 2002-12-10 | 2009-09-03 | Laan Roger Van Der | System and method for compressing video based on detected intraframe motion |
US20090225828A1 (en) * | 2002-12-10 | 2009-09-10 | Perlman Stephen G | Video Compression System and Method for Compensating for Bandwidth Limitations of a Communication Channel |
US20090225220A1 (en) * | 2002-12-10 | 2009-09-10 | Van Der Laan Roger | System and Method For Compressing Video By Adjusting Tile Size Based On Detected Intraframe Motion Or Scene Complexity |
US20100167816A1 (en) * | 2002-12-10 | 2010-07-01 | Perlman Stephen G | System and Method for Multi-Stream Video Compression |
US20100166066A1 (en) * | 2002-12-10 | 2010-07-01 | Steve Perlman | System and Method for Video Compression Using Feedback Including Data Related to the Successful Receipt of Video Content |
US20100167809A1 (en) * | 2002-12-10 | 2010-07-01 | Perlman Steve G | System and Method for Accelerated Machine Switching |
US20100166063A1 (en) * | 2002-12-10 | 2010-07-01 | Perlman Steve G | System and method for compressing video frames or portions thereof based on feedback information from a client device |
US8769594B2 (en) | 2002-12-10 | 2014-07-01 | Ol2, Inc. | Video compression system and method for reducing the effects of packet loss over a communication channel |
US10780344B2 (en) * | 2002-12-10 | 2020-09-22 | Sony Interactive Entertainment LLC | System and method for storing program code and data within an application hosting center |
US8366552B2 (en) | 2002-12-10 | 2013-02-05 | Ol2, Inc. | System and method for multi-stream video compression |
US10201760B2 (en) | 2002-12-10 | 2019-02-12 | Sony Interactive Entertainment America Llc | System and method for compressing video based on detected intraframe motion |
US8526490B2 (en) | 2002-12-10 | 2013-09-03 | Ol2, Inc. | System and method for video compression using feedback including data related to the successful receipt of video content |
US8606942B2 (en) | 2002-12-10 | 2013-12-10 | Ol2, Inc. | System and method for intelligently allocating client requests to server centers |
US8711923B2 (en) | 2002-12-10 | 2014-04-29 | Ol2, Inc. | System and method for selecting a video encoding format based on feedback data |
US8881215B2 (en) | 2002-12-10 | 2014-11-04 | Ol2, Inc. | System and method for compressing video based on detected data rate of a communication channel |
US20100166064A1 (en) * | 2002-12-10 | 2010-07-01 | Perlman Steve G | System and Method for Utilizing Forward Error Correction with Video Compression |
US20180221775A1 (en) * | 2002-12-10 | 2018-08-09 | Sony Interactive Entertainment America Llc | System and Method for Storing Program Code and Data Within an Application Hosting Center |
US9155962B2 (en) | 2002-12-10 | 2015-10-13 | Sony Computer Entertainment America Llc | System and method for compressing video by allocating bits to image tiles based on detected intraframe motion or scene complexity |
US8953675B2 (en) | 2002-12-10 | 2015-02-10 | Ol2, Inc. | Tile-based system and method for compressing video |
US9956490B2 (en) * | 2002-12-10 | 2018-05-01 | Sony Interactive Entertainment America Llc | System and method for storing program code and data within an application hosting center |
US8964830B2 (en) | 2002-12-10 | 2015-02-24 | Ol2, Inc. | System and method for multi-stream video compression using multiple encoding formats |
US9061207B2 (en) | 2002-12-10 | 2015-06-23 | Sony Computer Entertainment America Llc | Temporary decoder apparatus and method |
US9077991B2 (en) | 2002-12-10 | 2015-07-07 | Sony Computer Entertainment America Llc | System and method for utilizing forward error correction with video compression |
US9084936B2 (en) | 2002-12-10 | 2015-07-21 | Sony Computer Entertainment America Llc | System and method for protecting certain types of multimedia data transmitted over a communication channel |
US9138644B2 (en) | 2002-12-10 | 2015-09-22 | Sony Computer Entertainment America Llc | System and method for accelerated machine switching |
US9446305B2 (en) | 2002-12-10 | 2016-09-20 | Sony Interactive Entertainment America Llc | System and method for improving the graphics performance of hosted applications |
US9420283B2 (en) | 2002-12-10 | 2016-08-16 | Sony Interactive Entertainment America Llc | System and method for selecting a video encoding format based on feedback data |
US9192859B2 (en) | 2002-12-10 | 2015-11-24 | Sony Computer Entertainment America Llc | System and method for compressing video based on latency measurements and other feedback |
US9272209B2 (en) | 2002-12-10 | 2016-03-01 | Sony Computer Entertainment America Llc | Streaming interactive video client apparatus |
US20090017813A1 (en) * | 2007-07-10 | 2009-01-15 | Kabushiki Kaisha Toshiba | Information processing apparatus |
US8428572B2 (en) * | 2007-07-10 | 2013-04-23 | Fujitsu Mobile Communications Limited | Information processing apparatus |
EP2091205A1 (en) * | 2008-02-14 | 2009-08-19 | NTT DoCoMo, Inc. | Method and an apparatus for delivering a video stream |
US8737265B2 (en) * | 2010-06-18 | 2014-05-27 | Via Telecom Co., Ltd. | Methods and apparatuses for machine type communication |
US20110310769A1 (en) * | 2010-06-18 | 2011-12-22 | Via Telecom, Inc. | Methods and Apparatuses for Machine Type Communication |
US9168457B2 (en) | 2010-09-14 | 2015-10-27 | Sony Computer Entertainment America Llc | System and method for retaining system state |
CN104170488A (en) * | 2012-03-16 | 2014-11-26 | 瑞典爱立信有限公司 | Method, network node, computer program and computer program product for determining resource allocation between downlink and uplink |
US20150043464A1 (en) * | 2012-03-16 | 2015-02-12 | Telefonaktiebolaget L M Ericsson (Publ) | Method, network node, computer program and computer program product for determining resource allocation between downlink and uplink |
GB2550877A (en) * | 2016-05-26 | 2017-12-06 | Univ Surrey | Object-based audio rendering |
US20190373270A1 (en) * | 2018-05-31 | 2019-12-05 | Apple Inc. | Efficient coding of video data in the presence of video annotations |
CN110784719A (en) * | 2018-05-31 | 2020-02-11 | 苹果公司 | Efficient encoding of video data in the presence of video annotations |
US11109042B2 (en) * | 2018-05-31 | 2021-08-31 | Apple Inc. | Efficient coding of video data in the presence of video annotations |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US20080025196A1 (en) | Method and system for providing visually related content description to the physical layer | |
US8917674B2 (en) | Method and system for content-aware mapping/error protection | |
US8272017B2 (en) | Method and system for concurrent communicating of high definition television (HDTV) and standard definition television (SDTV) information in a multistandard wireless communication system | |
US7653415B2 (en) | Method and system for increasing data rate in a mobile terminal using spatial multiplexing for DVB-H communication | |
KR101407674B1 (en) | Multimedia broadcast forwarding systems and methods | |
US20120076204A1 (en) | Method and apparatus for scalable multimedia broadcast using a multi-carrier communication system | |
US7920895B2 (en) | Method and system for providing broadcast services through a cellular and/or wireless network to a plurality of mobile devices via service provider integration | |
US8934529B2 (en) | Method and apparatus for providing robust reception in a wireless communications system | |
US8027704B2 (en) | Method and system for increasing data rate in a mobile terminal using spatial multiplexing for DVB-H communication | |
US20080009312A1 (en) | Method and system for joint broadcast receiving and cellular communication at mobile terminal or device without service control | |
US20090130973A1 (en) | Method and System For Cellular Network and Integrated Broadcast Television (TV) Downlink With Intelligent Service Control | |
KR100923927B1 (en) | Method and system for content-aware mapping/error protection | |
US20090067388A1 (en) | Method and System for Mobile Receiver Architecture for US Band Cellular and VHF/UHF Broadcasting | |
US20150312890A1 (en) | Method and System for Communicating Information in a Wireless Communication System | |
WO2018003540A1 (en) | Reception device, transmission device, and data processing method | |
US20070070179A1 (en) | Method and system for a reconfigurable OFDM radio | |
US8302148B2 (en) | Transmission of composite digital broadcast information | |
Kou et al. | Next generation broadcast wireless systems: Challenges and trends | |
Schafer | Terrestrial transmission of DTVB signals-the European specification | |
Maračić et al. | Reception of digital broadcast services using mobile devices |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: BROADCOM CORPORATION, CALIFORNIA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:KARAOGUZ, JEYHAN;HONARY, HOOMAN;SESHADRI, NAMBIRAJAN;AND OTHERS;REEL/FRAME:018143/0633;SIGNING DATES FROM 20060608 TO 20060723 |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |
|
AS | Assignment |
Owner name: BANK OF AMERICA, N.A., AS COLLATERAL AGENT, NORTH CAROLINA Free format text: PATENT SECURITY AGREEMENT;ASSIGNOR:BROADCOM CORPORATION;REEL/FRAME:037806/0001 Effective date: 20160201 Owner name: BANK OF AMERICA, N.A., AS COLLATERAL AGENT, NORTH Free format text: PATENT SECURITY AGREEMENT;ASSIGNOR:BROADCOM CORPORATION;REEL/FRAME:037806/0001 Effective date: 20160201 |
|
AS | Assignment |
Owner name: AVAGO TECHNOLOGIES GENERAL IP (SINGAPORE) PTE. LTD., SINGAPORE Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:BROADCOM CORPORATION;REEL/FRAME:041706/0001 Effective date: 20170120 Owner name: AVAGO TECHNOLOGIES GENERAL IP (SINGAPORE) PTE. LTD Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:BROADCOM CORPORATION;REEL/FRAME:041706/0001 Effective date: 20170120 |
|
AS | Assignment |
Owner name: BROADCOM CORPORATION, CALIFORNIA Free format text: TERMINATION AND RELEASE OF SECURITY INTEREST IN PATENTS;ASSIGNOR:BANK OF AMERICA, N.A., AS COLLATERAL AGENT;REEL/FRAME:041712/0001 Effective date: 20170119 |