EP3892003A1 - Distribution à faible latence de contenu audio à l'aide d'une seule radio - Google Patents
Distribution à faible latence de contenu audio à l'aide d'une seule radioInfo
- Publication number
- EP3892003A1 EP3892003A1 EP19892433.4A EP19892433A EP3892003A1 EP 3892003 A1 EP3892003 A1 EP 3892003A1 EP 19892433 A EP19892433 A EP 19892433A EP 3892003 A1 EP3892003 A1 EP 3892003A1
- Authority
- EP
- European Patent Office
- Prior art keywords
- audio
- media device
- wireless network
- display device
- content
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000009826 distribution Methods 0.000 title description 3
- 238000000034 method Methods 0.000 claims abstract description 23
- 230000001360 synchronised effect Effects 0.000 claims description 5
- 238000004519 manufacturing process Methods 0.000 abstract description 6
- 238000004590 computer program Methods 0.000 abstract description 5
- 238000001514 detection method Methods 0.000 description 19
- 238000003860 storage Methods 0.000 description 17
- 238000012545 processing Methods 0.000 description 16
- 238000004891 communication Methods 0.000 description 9
- 230000006855 networking Effects 0.000 description 6
- 238000013459 approach Methods 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 5
- 230000006870 function Effects 0.000 description 4
- 230000001934 delay Effects 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 230000003139 buffering effect Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000011664 signaling Effects 0.000 description 2
- 241000278713 Theora Species 0.000 description 1
- 230000003044 adaptive effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000003292 diminished effect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/436—Interfacing a local distribution network, e.g. communicating with another STB or one or more peripheral devices inside the home
- H04N21/43615—Interfacing a Home Network, e.g. for connecting the client to a plurality of peripherals
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
- G10L25/48—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use
- G10L25/51—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/4302—Content synchronisation processes, e.g. decoder synchronisation
- H04N21/4307—Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/4302—Content synchronisation processes, e.g. decoder synchronisation
- H04N21/4307—Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen
- H04N21/43076—Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen of the same content streams on multiple devices, e.g. when family members are watching the same movie on different devices
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/436—Interfacing a local distribution network, e.g. communicating with another STB or one or more peripheral devices inside the home
- H04N21/4363—Adapting the video stream to a specific local network, e.g. a Bluetooth® network
- H04N21/43632—Adapting the video stream to a specific local network, e.g. a Bluetooth® network involving a wired protocol, e.g. IEEE 1394
- H04N21/43635—HDMI
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/436—Interfacing a local distribution network, e.g. communicating with another STB or one or more peripheral devices inside the home
- H04N21/4363—Adapting the video stream to a specific local network, e.g. a Bluetooth® network
- H04N21/43637—Adapting the video stream to a specific local network, e.g. a Bluetooth® network involving a wireless protocol, e.g. Bluetooth, RF or wireless LAN [IEEE 802.11]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/439—Processing of audio elementary streams
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/439—Processing of audio elementary streams
- H04N21/4394—Processing of audio elementary streams involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/442—Monitoring of processes or resources, e.g. detecting the failure of a recording device, monitoring the downstream bandwidth, the number of times a movie has been viewed, the storage space available from the internal hard disk
- H04N21/44213—Monitoring of end-user related data
- H04N21/44222—Analytics of user selections, e.g. selection of programs or purchase activity
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/442—Monitoring of processes or resources, e.g. detecting the failure of a recording device, monitoring the downstream bandwidth, the number of times a movie has been viewed, the storage space available from the internal hard disk
- H04N21/44227—Monitoring of local network, e.g. connection or bandwidth variations; Detecting new devices in the local network
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
- H04N21/81—Monomedia components thereof
- H04N21/8106—Monomedia components thereof involving special audio data, e.g. different tracks for different languages
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R3/00—Circuits for transducers, loudspeakers or microphones
- H04R3/12—Circuits for transducers, loudspeakers or microphones for distributing signals to two or more loudspeakers
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R5/00—Stereophonic arrangements
- H04R5/04—Circuit arrangements, e.g. for selective connection of amplifier inputs/outputs to loudspeakers, for loudspeaker detection, or for adaptation of settings to personal preferences or hearing impairments
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R2420/00—Details of connection covered by H04R, not provided for in its groups
- H04R2420/07—Applications of wireless loudspeakers or wireless microphones
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R2499/00—Aspects covered by H04R or H04S not otherwise provided for in their subgroups
- H04R2499/10—General applications
- H04R2499/15—Transducers incorporated in visual displaying devices, e.g. televisions, computer displays, laptops
Definitions
- This disclosure is generally directed to wirelessly distributing audio when playing content, such as a movie or TV show.
- the home wireless network is being used to transmit the audio to wireless speakers.
- a media player or other device
- a non-streaming source such as cable, satellite, DVD/Blu-Ray player, etc.
- many devices could be using the home network to access the Internet. Due to this burden on the home network, there may be delays or latency with delivering the audio to the wireless speakers, and this latency may make it difficult to synchronize the audio and video playback.
- Some embodiments operate by: determining that the media device is not a source of the content, such that the media device does not need to stream the content over a first wireless network (such as a home wireless network); informing the audio speakers to switch from the first wireless network to a second wireless network (in other words, informing the audio speakers to move to a“clean channel”); disconnecting the media device from the first wireless network; establishing the second wireless network; and providing the audio to the wireless speakers for playback over the second wireless network.
- a first wireless network such as a home wireless network
- the media device determines whether or not the media device is a source of the content: (a) it may be determined whether first audio provided by the media device to the display device corresponds to second audio being output from the display device; and/or (b) one or more heuristics may be used to further determine whether the media device is the source. In some embodiments, it may be concluded that the media device is the source if (a) indicates that the media device is the source, and at least one of the heuristics of (b) also indicates that the media device is the source.
- FIG. 1 illustrates a block diagram of a whole home entertainment system
- FIG. 2 illustrates a block diagram of a display device, according to some embodiments.
- FIGS. 3-5 collectively illustrate a flowchart in a media device for distributing audio to wireless speakers in a whole home entertainment system, where the media device has a single wireless radio, according to some embodiments.
- FIG. 6 illustrates an example computer system useful for implementing various embodiments.
- the audio is transmitted over a wireless network to wireless speakers.
- the wireless network may be a home WIFI network.
- the home WIFI network may also be used to stream the content, and/or enable what could be a multitude of electronic devices to access the Internet. Due to this heavy use of the home WIFI network, it may take some time to transmit the audio to the wireless speakers when using the home WIFI network, and this may make it difficult to maintain synchronization of the audio and video playback.
- a separate wireless connection/network may be established with the wireless speakers.
- This separate wireless connection/network (or“clean channel”) may then be used to transmit the audio to the wireless speakers. Since this separate wireless connection/network is not subject to the burdens of the home WIFI network, the audio may be transmitted significantly faster, thereby better enabling audio/video sync during playback of the content. According to some embodiments, this functionality is achieved by a media device that has only a single wireless radio.
- FIG. 1 illustrates a block diagram of a whole home entertainment system 102, according to some embodiments.
- whole home entertainment system 102 is directed to playing content such as movies, TV shows, games, audio books, and music, to name just some examples.
- System 102 may be located in a user’s home or any other location, such as a business, a park, a stadium, a restaurant, a bar, or a government office, to name just some examples.
- Whole home entertainment system 102 may include one or more display devices
- Display devices 104 may be any combination of monitors, televisions (TVs), computers, smart phones, tablets, wearables (such as a watch), appliances, and/or projectors, to name just some examples.
- Display devices 104 may include internal speakers 105.
- Each display device 104 may be connected to a media device 106.
- Each media device 106 may be separate from its respective display device 104, or may be part of or integrated with the display device 104.
- Each media device 106 may be a streaming media device (that is, a streaming source) that may stream content from content sources 124, and may provide such content to its respective display device 104 for playback to users.
- a given media device 106 may communicate via a home (or local) wireless network 126 to stream content from content sources 124 via the Internet 122.
- Wireless network 126 may be any wireless network, wireless medium or communication protocol such as WIFI, Bluetooth, infrared, cellular, etc., or any combination thereof.
- the media device 106 may be connected to its respective display device 104 via a HDMI (high-definition multimedia interface) ARC (audio return channel) connection 155.
- the media device 106 may transmit audio and video to the display device 104 over the HDMI ARC connection 155.
- the media device 106 may receive from the display device 104 over the HDMI ARC connection 155 the audio that is being played by the display device 104.
- the media device 106 receives the audio in real time from the display device 104, that is, as the audio is played on the display device 104.
- Each content source 124 may store content 140 and metadata 142.
- Content 140 may include any combination of music, videos, movies, TV programs, multimedia, images, still pictures, text, graphics, gaming applications, advertisements, programming content, public service content, government content, local community content, software, and/or any other content or data objects in electronic form.
- Metadata 142 may include data about content 140.
- metadata 142 may include associated or ancillary information indicating or related to writer, director, producer, composer, artist, actor, summary, chapters, production, history, year, trailers, alternate versions, related content, applications, and/or any other information pertaining or relating to the content 140.
- Metadata 142 may also or alternatively include links to any such information pertaining or relating to the content 140.
- Metadata 142 may also or alternatively include one or more indexes of content 140, such as but not limited to a trick mode index.
- Each display device 104 may also receive content for playback from any number of non-streaming sources 119 in addition to media device 106, such as cable or satellite 120, an over-the-air antenna 122, a Blu-Ray/DVD player 124, etc., to name just some examples.
- media device 106 such as cable or satellite 120, an over-the-air antenna 122, a Blu-Ray/DVD player 124, etc., to name just some examples.
- Each display device 104 may also or alternatively wirelessly receive content from any number of other electronic devices 113 over the home wireless network 126, such as from computers 114, smart phones 116, appliances and internet of things (IoT) devices 118, etc., to name just some examples. Any or all of these electronic devices 113 may also use the home wireless network 126 to access the Internet 122 in a well-known manner.
- IoT internet of things
- the whole home entertainment system 102 may include wireless speakers 108.
- the audio portion of the content is provided to the wireless speakers 108 for playback (the audio may also or alternatively be provided to the internal speakers 105 in display devices 104 for playback).
- the media device 106 may receive content from content sources 124 via the Internet 122 and the home wireless network 126. Then, the media device 106 may transmit the content to the display device 104 for video playback via the HDMI/ARC connection 155, and may transmit the audio of the content to the wireless speakers 108 via the home wireless network 126 for audio playback.
- the home wireless network 126 may be burdened by having to stream the content to the media device 106 from content sources 124 (as just described), as well as having to provide connectivity to the Internet 122 to a multitude of electronic devices 113. Due to this burden on the home wireless network 126, it may take some time to transmit the audio to the wireless speakers 108 over the home wireless network 126. For example, it may take 100s of milliseconds to transmit the audio to the wireless speakers 108 over the home wireless network 126, and this latency may greatly vary depending on the load on the home wireless network 126 at any given moment. This latency may make it difficult to maintain audio and video synchronization when content is played. As persons skilled in the relevant art(s) will appreciate, loss of audio/video sync may greatly detract from users’ experience when consuming content.
- a new wireless connection or network 112 may be established that is separate from the home wireless network 126.
- This wireless network/connection 112 may be used to transmit audio from the media device 106 to the wireless speakers 108. Since the wireless network/connection 112 is not burdened like the home wireless network 126 (that is, since the wireless network/connection 112 is a “clean channel”), it make take significantly less time to transmit the audio. For example, in some embodiments, the audio may be transmitted from the media device 106 to the wireless speakers 108 via this wireless connection/network 112 in approximately 20-30 milliseconds. Accordingly, use of the wireless connection/network 112 in this manner makes it much easier to maintain audio/video sync. Also, embodiments of this disclosure achieve these technological advantages while using only a single wireless radio in the media device 106. This shall now be described further.
- FIG. 2 illustrates an example media device 106, according to some embodiments.
- the media device 106 may include one or more video processing modules 202, and one or more audio processing modules 204.
- Each video processing module 202 may be configured to decode, encode and/or translate video of one or more video formats, such as but not limited to MP4 (mp4, m4a, m4v, f4v, f4a, m4b, m4r, f4b, mov), 3GP (3gp, 3gp2, 3g2, 3gpp, 3gpp2), OGG (ogg, oga, ogv, ogx), WMV (wmv, wma, asf), WEBM, FLV, AVI, QuickTime, HDV, MXF (OP la, OP-Atom), MPEG-TS, MPEG-2 PS, MPEG-2 TS, WAV, Broadcast WAV, LXF, GXF, and/or VOB, to name just some examples.
- MP4 mp4, m4a, m4v, f4v, f4a, m4b, m4r, f4b, mov
- 3GP 3g
- Each video processing module 202 may include one or more video codecs, such as but not limited to H.263, H.264, HEV, MPEGl, MPEG2, MPEG-TS, MPEG-4, Theora, 3GP, DV, DVCPRO, DVCPRO, DVCProHD, IMX, XDCAM HD, XDCAM HD422, and/or XDCAM EX, to name just some examples.
- video codecs such as but not limited to H.263, H.264, HEV, MPEGl, MPEG2, MPEG-TS, MPEG-4, Theora, 3GP, DV, DVCPRO, DVCPRO, DVCProHD, IMX, XDCAM HD, XDCAM HD422, and/or XDCAM EX, to name just some examples.
- each audio processing module 204 may be configured to decode, encode and/or translate audio of one or more audio formats, such as but not limited to AAC, HE-AAC, AC3 (Dolby Digital), EAC3 (Dolby Digital Plus), WMA, WAV, PCM, MP3, OGG GSM, FLAC, AU, AIFF, and/or VOX, to name just some examples.
- Media device 106 may also include buffer 206 for buffering video, as well as other types of data.
- the media device 106 may include a microphone 208 for receiving audio input, such as spoken audio commands from users, as well as audio output by the internal speakers 105 and/or the wireless speakers 108 while content is being played.
- the media device 106 may include an audio detection module 212 for detecting the presence or absence of audio being received by the microphone 208.
- the audio detection module 212 may also be configured to compare and process two or more audio streams; this is further described below.
- the media device 106 includes a single wireless radio 214, for communicating wirelessly in a well-known manner.
- Media device 106 may include a networking module 210 for connecting and
- the networking module 210 may be also configured to create, establish and maintain a wireless connection or network 112 using the radio 214, where the wireless
- connection/network 112 is separate from the home wireless network 126.
- the networking module 210 may perform this function in any well-known manner, such as via the well-known SoftAP (software enabled access point) technology.
- SoftAP software enabled access point
- the networking module 210 is not limited to this example embodiment for creating, establishing and maintaining the wireless connection/network 112.
- FIGS. 3 and 4 collectively illustrate a method 300 for wirelessly distributing audio while playing content that includes such audio, according to some embodiments.
- Method 300 can be performed by processing logic that can comprise hardware
- Method 300 shall be described with reference to FIGS. 1 and 2. However, method 300 is not limited to those example embodiments.
- Method 300 may operate while the display device 104 is playing content, such as a movie, TV show, game, music, etc., to name just some examples. This is indicated by 301.
- the media device 106 may determine if it is the source of the content currently being played by the display device 104. Operation of 302 is described below with reference to FIG. 5. If the media device 106 is the source, then 304 is performed.
- the media device 106 may stream the content from a content source 124.
- the media device 106 may receive the content from the content source 124 via the Internet 122 and the home wireless network 126.
- the content may include both video and audio.
- the video processing module 202 of the media device 106 may decode the video, and optionally buffer the decoded video in the buffer 206. Such buffering 206 may be performed to compensate for delays in transmitting the audio to the wireless speakers 108 over the home wireless network 126, to thereby better achieve sync between the audio and video during playback of the content.
- the audio processing module 204 may decode the audio of the content, and then the media device 106 may transmit the decoded audio to the wireless speakers 108 via the home wireless network 126. It is noted that, since the media device 106 includes just a single radio 214, the home wireless network 126 must be used to deliver the audio to the wireless speakers 108, because the home wireless network 126 is also needed by the media device 106 to receive the content from the content source 124 (that is, the media device 106 cannot disconnect from the home wireless network 126 and establish a separate wireless connection/network 112 in the manner that is described below).
- the video may be provided from the buffer 206 to the display device 104 over the HDMI/ARC connection 155 for playback, and the audio is provided to the wireless speakers 108 over the home wireless network 126 for playback. Since the video was buffered in the buffer 206 to compensate for delays in the home wireless network 126, such playback of video and audio is in sync.
- the display device 104 may be receiving the content (both audio and video) from one of the non-streaming sources 119, or any other source other than the media device 106.
- the media device 106 may inform the wireless speakers 108 that it should begin communicating via a new wireless connection/network 112. Such operation may be achieved in a well-known manner using signaling or messaging (or any other means) via the home wireless network 126 or any other communication medium, means,
- the networking module 210 of the media device 106 may disconnect the radio 214 from the home wireless network 126, in a well-known manner. Such disconnection is possible in this case, since the media device 106 is not the source of the content being played on the display device 104 and, thus, the home wireless network 126 is not needed to stream content from the content source 124.
- the networking module 210 may create and/or otherwise establish the new wireless connection and/or network 112 in a well-known manner (using SoftAP technology, for example).
- This wireless connection/network 112 may be different and independent of the home wireless network 126, such that any burdens on the home wireless network 126 (as discussed above) do not impact the wireless connection/network 112.
- the wireless connection/network 112 may be dedicated for the transmission of audio data from the media device 106 to the wireless speakers 108.
- the media device 106 may receive the audio of the content from the
- the audio is received via the HDMI/ARC connection 155.
- the media device 106 may receive the audio from the display device 104 via other means during playback of the content, such as via SPDIF (SONY/PHILLIPS Digital Interface), analog audio, etc.
- the audio processing module 204 may decode the audio of the content (to the extent necessary), and then the media device 106 may transmit the decoded audio to the wireless speakers 108 via the wireless connection/network 112.
- the audio plays on the wireless speakers 108. Since the audio was provided to the wireless speakers 108 much faster over the wireless connection/network 112 compared to the home wireless network 126 (that is, for example, 20-30 milliseconds versus 100s of milliseconds), the audio and video playback are synchronized. This is the case, even though the video was playing on the display device 104 when the audio was provided to the media device 106 over the HDMI/ARC connection 155 (or via other means, as described above). In practice, there may be some latency between such video playback and audio playback, but due to the greater speed of the wireless
- connection/network 112 such latency is not so great to be discernable by users experiencing the content.
- the media device 106 may cause the wireless speakers 108 to switch back to the home wireless network 126 to receive audio data. This may occur, for example, if the media device 106 determines that, based on current conditions, traffic on the home wireless network 126 would not prevent audio/video playback sync if the home wireless network 126 was used to transmit the audio to the wireless speakers 108,
- the media device 106 may determine if it is the source of the content currently being played by the display device 104.
- the source of the content may be a non-streaming source 119 (such as cable or satellite 120, antenna 122, Blu-Ray/DVD 124, etc), or any other source. Operation of 302 shall now be described below with reference to FIG. 5.
- the display device 104 may signal or otherwise indicate to the media
- the display device 104 may provide such signaling via a hot plug detect (HPD) pin of the HDMI/ARC connection 155, or through any other well-known means, approach, functionality, mechanism or technology. However, some display devices 104 may leave the HPD pin active (or inactive) irrespective of the current source.
- HPD hot plug detect
- the display device 104 indicates through the HPD pin that the media device 106 is (or is not) the source, 504 and/or 506 may still be performed.
- the audio detection module 212 of the media device 106 compares the audio being provided by the media device 106 to the display device 104 over the
- the audio detection module 212 concludes that the media device 106 is the source. In some embodiments, the audio detection module 212 performs such comparison using any one or combination of well-known correlation methods, techniques, procedures, approaches and/or technologies.
- the comparison of 504 may be possible even when the HDMI connection 155 is not ARC (or the audio is being delivered via other means such as SPDIF, analog audio, etc.).
- the audio detection module 212 may instead compare the audio being provided by the media device 106 to the display device 104, to the audio output by the display device 104 using internal speakers 105 (or wireless speakers 108) and received by the media device 106 via microphone 208.
- the display device 104 may process the audio before playing it back (to enhance quality or add sound effects, for example). Therefore, even when the audio detection module 212 in 504 concludes that it is likely that the media device 106 is the source, 506 may be performed.
- the audio detection module 212 performs one or more heuristics to
- the audio detection module 212 may cause silence to be sent to the display device 104, and determine whether the display device 104 is outputting silence. If the display device 104 is outputting silence, then it is more likely that the media device 106 is the source; otherwise, it is more likely the media device 106 is not the source.
- the audio detection module 212 may cause known data to be sent to the display device 104, and determine whether the display device 104 is outputting such data. If the display device 104 is outputting the known data, then it is more likely that the media device 106 is the source; otherwise, it is more likely the media device 106 is not the source.
- the audio detection module 212 may detect if users have interacted with the media device 106 within a given time period. For example, the audio detection module 212 may determine if users have used a remote control (not shown in FIG. 1) to interact with a user interface of the media device 106, or have used
- buttons of the remote control to control playback by the media device 106. If users have interacted with the media device 106 within the given time period, then it is more likely the media device 106 is the source; otherwise, it is less likely the media device 106 is the source.
- the audio detection module 212 concludes that the media device 106 is the source. Otherwise, the audio detection module 212 concludes that the media device 106 is not the source.
- At least two of 502, 504 and 506 would have to indicate that the media device 106 is the source, in order for the audio detection module 212 to conclude that the media device 106 is the source. Otherwise, the audio detection module 212 concludes that the media device 106 is not the source.
- 504 and at least one of the heuristics of 506 would have to indicate that the media device 106 is the source, in order for the audio detection module 212 to conclude that the media device 106 is the source. Otherwise, the audio detection module 212 concludes that the media device 106 is not the source.
- Computer system 600 can be any computer or computing device capable of performing the functions described herein.
- one or more computer systems 600 or portions thereof can be used to implement any embodiments of FIGS. 1-5, and/or any combination or sub-combination thereof.
- Computer system 600 includes one or more processors (also called central
- processor 604 is connected to a communication infrastructure or bus 606.
- One or more processors 604 can each be a graphics processing unit (GPU).
- a GPU is a processor that is a specialized electronic circuit designed to process mathematically intensive applications.
- the GPU can have a parallel structure that is efficient for parallel processing of large blocks of data, such as mathematically intensive data common to computer graphics applications, images, videos, etc.
- Computer system 600 also includes user input/output device(s) 603, such as monitors, keyboards, pointing devices, etc., that communicate with communication infrastructure 606 through user input/output interface(s) 602.
- Computer system 600 also includes a main or primary memory 608, such as random access memory (RAM).
- Main memory 608 can include one or more levels of cache.
- Main memory 608 has stored therein control logic (i.e., computer software) and/or data.
- Computer system 600 can also include one or more secondary storage devices or memory 610.
- Secondary memory 610 can include, for example, a hard disk drive 612 and/or a removable storage device or drive 614.
- Removable storage drive 614 can be a floppy disk drive, a magnetic tape drive, a compact disk drive, an optical storage device, tape backup device, and/or any other storage device/drive.
- Removable storage drive 614 can interact with a removable storage unit 618.
- Removable storage unit 618 includes a computer usable or readable storage device having stored thereon computer software (control logic) and/or data.
- Removable storage unit 618 can be a floppy disk, magnetic tape, compact disk, DVD, optical storage disk, and/ any other computer data storage device.
- Removable storage drive 614 reads from and/or writes to removable storage unit 618 in a well-known manner.
- secondary memory 610 can include other means, instrumentalities or other approaches for allowing computer programs and/or other instructions and/or data to be accessed by computer system 600.
- Such means, instrumentalities or other approaches can include, for example, a removable storage unit 622 and an interface 620.
- the removable storage unit 622 and the interface 620 can include a program cartridge and cartridge interface (such as that found in video game devices), a removable memory chip (such as an EPROM or PROM) and associated socket, a memory stick and USB port, a memory card and associated memory card slot, and/or any other removable storage unit and associated interface.
- Computer system 600 can further include a communication or network interface
- Communication interface 624 enables computer system 600 to communicate and interact with any combination of remote devices, remote networks, remote entities, etc. (individually and collectively referenced by reference number 628).
- communication interface 624 can allow computer system 600 to communicate with remote devices 628 over communications path 626, which can be wired and/or wireless, and which can include any combination of LANs, WANs, the Internet, etc.
- Control logic and/or data can be transmitted to and from computer system 600 via communication path 626.
- a non-transitory, tangible apparatus or article of manufacture comprising a tangible computer useable or readable medium having control logic (software) stored thereon is also referred to herein as a computer program product or program storage device.
- control logic when executed by one or more data processing devices (such as computer system 600), causes such data processing devices to operate as described herein.
- embodiments or similar phrases, indicate that the embodiment described can include a particular feature, structure, or characteristic, but every embodiment can not necessarily include the particular feature, structure, or characteristic. Moreover, such phrases are not necessarily referring to the same embodiment. Further, when a particular feature, structure, or characteristic is described in connection with an embodiment, it would be within the knowledge of persons skilled in the relevant art(s) to incorporate such feature, structure, or characteristic into other embodiments whether or not explicitly mentioned or described herein. Additionally, some embodiments can be described using the expression “coupled” and“connected” along with their derivatives. These terms are not necessarily intended as synonyms for each other. For example, some embodiments can be described using the terms“connected” and/or“coupled” to indicate that two or more elements are in direct physical or electrical contact with each other. The term“coupled,” however, can also mean that two or more elements are not in direct contact with each other, but yet still co-operate or interact with each other.
Landscapes
- Engineering & Computer Science (AREA)
- Signal Processing (AREA)
- Multimedia (AREA)
- Computer Networks & Wireless Communication (AREA)
- Health & Medical Sciences (AREA)
- Physics & Mathematics (AREA)
- Acoustics & Sound (AREA)
- General Health & Medical Sciences (AREA)
- Social Psychology (AREA)
- Databases & Information Systems (AREA)
- Computational Linguistics (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Human Computer Interaction (AREA)
- Otolaryngology (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
Applications Claiming Priority (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US201862775450P | 2018-12-05 | 2018-12-05 | |
US16/689,749 US11197054B2 (en) | 2018-12-05 | 2019-11-20 | Low latency distribution of audio using a single radio |
PCT/US2019/064445 WO2020117922A1 (fr) | 2018-12-05 | 2019-12-04 | Distribution à faible latence de contenu audio à l'aide d'une seule radio |
Publications (2)
Publication Number | Publication Date |
---|---|
EP3892003A1 true EP3892003A1 (fr) | 2021-10-13 |
EP3892003A4 EP3892003A4 (fr) | 2022-09-07 |
Family
ID=70971321
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP19892433.4A Pending EP3892003A4 (fr) | 2018-12-05 | 2019-12-04 | Distribution à faible latence de contenu audio à l'aide d'une seule radio |
Country Status (3)
Country | Link |
---|---|
US (2) | US11197054B2 (fr) |
EP (1) | EP3892003A4 (fr) |
WO (1) | WO2020117922A1 (fr) |
Families Citing this family (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10091264B2 (en) * | 2015-12-26 | 2018-10-02 | Intel Corporation | Technologies for streaming device role reversal |
US11197054B2 (en) | 2018-12-05 | 2021-12-07 | Roku, Inc. | Low latency distribution of audio using a single radio |
WO2023282959A1 (fr) * | 2021-07-09 | 2023-01-12 | Arris Enterprises Llc | Système et procédé visant à synchroniser le rendu d'une présentation audio à plusieurs canaux avec une présentation vidéo |
Family Cites Families (23)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN102523063A (zh) * | 2004-08-09 | 2012-06-27 | 尼尔森(美国)有限公司 | 用于监视来自各种源的音频/视觉内容的方法及装置 |
KR101195885B1 (ko) * | 2008-03-03 | 2012-10-30 | 삼성전자주식회사 | 무선 오디오 출력장치 및 이를 포함하는 무선 오디오출력시스템과 이의 제어방법 |
WO2010068151A1 (fr) | 2008-12-08 | 2010-06-17 | Telefonaktiebolaget L M Ericsson (Publ) | Dispositif et procédé pour synchroniser des données audio reçues avec des données vidéo |
KR101728911B1 (ko) | 2010-01-05 | 2017-04-20 | 엘지전자 주식회사 | 화상 통화 연결 방법, 그를 이용한 화상 통신 장치 및 디스플레이 장치 |
US8379151B2 (en) * | 2010-03-24 | 2013-02-19 | Floreat, Inc. | Synchronization of audio and video streams |
US8977310B2 (en) * | 2010-12-30 | 2015-03-10 | Motorola Solutions, Inc. | Methods for coordinating wireless coverage between different wireless networks for members of a communication group |
US9204123B2 (en) | 2011-01-14 | 2015-12-01 | Comcast Cable Communications, Llc | Video content generation |
WO2013032402A1 (fr) * | 2011-09-01 | 2013-03-07 | Mp4Sls Pte Ltd | Dispositif de communication et procédé de réception de données multimédias |
CN102917406B (zh) * | 2012-08-02 | 2016-04-06 | 华为技术有限公司 | 数据流量分流方法和设备 |
CN103747391B (zh) * | 2013-06-26 | 2018-03-06 | 苏州联科盛世科技有限公司 | 具有双无线模式的音箱及其控制方法 |
US9706319B2 (en) * | 2015-04-20 | 2017-07-11 | Sonos, Inc. | Wireless radio switching |
US10397684B2 (en) * | 2016-01-05 | 2019-08-27 | Voxx International Corporation | Wireless speaker system |
US9699684B1 (en) * | 2016-07-28 | 2017-07-04 | Blackfire Research Corporation | Low-latency multimedia using dual wireless adapters |
US10075908B2 (en) * | 2016-06-13 | 2018-09-11 | Erik Young | Group speakers |
US10820087B2 (en) * | 2016-06-24 | 2020-10-27 | Dolby Laboratories Licensing Corporation | Wireless audio source switching |
KR102656013B1 (ko) * | 2016-10-28 | 2024-04-11 | 삼성전자주식회사 | 컨텐츠 출력 장치 및 그 제어 방법 |
US10291964B2 (en) | 2016-12-06 | 2019-05-14 | At&T Intellectual Property I, L.P. | Multimedia broadcast system |
US10129594B2 (en) * | 2017-03-21 | 2018-11-13 | Amplivy, Inc. | Content-activated intelligent, autonomous audio/video source controller |
US10805658B2 (en) | 2018-09-12 | 2020-10-13 | Roku, Inc. | Adaptive switching in a whole home entertainment system |
US10812751B2 (en) | 2018-09-18 | 2020-10-20 | Roku, Inc. | Dynamically switching to/from a first network during audio playback over HDMI/ARC |
US11172180B2 (en) * | 2018-09-19 | 2021-11-09 | Canon Kabushiki Kaisha | Control apparatus, control method and non-transitory computer-readable medium |
US10735881B2 (en) * | 2018-10-09 | 2020-08-04 | Sony Corporation | Method and apparatus for audio transfer when putting on/removing headphones plus communication between devices |
US11197054B2 (en) | 2018-12-05 | 2021-12-07 | Roku, Inc. | Low latency distribution of audio using a single radio |
-
2019
- 2019-11-20 US US16/689,749 patent/US11197054B2/en active Active
- 2019-12-04 WO PCT/US2019/064445 patent/WO2020117922A1/fr unknown
- 2019-12-04 EP EP19892433.4A patent/EP3892003A4/fr active Pending
-
2021
- 2021-10-26 US US17/510,954 patent/US11943497B2/en active Active
Also Published As
Publication number | Publication date |
---|---|
US11943497B2 (en) | 2024-03-26 |
EP3892003A4 (fr) | 2022-09-07 |
US20220046311A1 (en) | 2022-02-10 |
WO2020117922A1 (fr) | 2020-06-11 |
US20200186864A1 (en) | 2020-06-11 |
US11197054B2 (en) | 2021-12-07 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
EP3850859B1 (fr) | Réglage dynamique de vidéo pour améliorer la synchronisation avec l'audio | |
US11943497B2 (en) | Network-based audio playback | |
US11611788B2 (en) | Adaptive switching in a whole home entertainment system | |
US20200089464A1 (en) | Audio Cancellation and Content Recognition of Audio Received Over HDMI/ARC | |
US11611797B2 (en) | Providing over-the-air content to any device | |
US20220303617A1 (en) | Server-side streaming content stitching | |
US20230010754A1 (en) | Non-television experience triggers | |
US11627368B1 (en) | Automatic offering and switching to a higher quality media stream | |
US11792568B2 (en) | Power control for speaker devices in a wireless media system | |
US11985377B2 (en) | Combined media capability for multiple media devices | |
US11785300B2 (en) | HDMI customized ad insertion | |
US20240155178A1 (en) | Private listening system for streaming audio and video | |
EP4250748A1 (fr) | Préservation de verrouillage de canal et réalisation d'insertion de publicité dynamique (dai) dans une configuration multi-dispositifs | |
US20240205495A1 (en) | Advertisement placement for stream media |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE |
|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE |
|
17P | Request for examination filed |
Effective date: 20210701 |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
DAV | Request for validation of the european patent (deleted) | ||
DAX | Request for extension of the european patent (deleted) | ||
A4 | Supplementary search report drawn up and despatched |
Effective date: 20220804 |
|
RIC1 | Information provided on ipc code assigned before grant |
Ipc: H04R 5/04 20060101ALI20220801BHEP Ipc: H04N 21/81 20110101ALI20220801BHEP Ipc: H04N 21/442 20110101ALI20220801BHEP Ipc: H04N 21/439 20110101ALI20220801BHEP Ipc: H04N 21/43 20110101ALI20220801BHEP Ipc: H04N 21/4363 20110101ALI20220801BHEP Ipc: H04N 21/436 20110101AFI20220801BHEP |
|
GRAP | Despatch of communication of intention to grant a patent |
Free format text: ORIGINAL CODE: EPIDOSNIGR1 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: GRANT OF PATENT IS INTENDED |
|
RIC1 | Information provided on ipc code assigned before grant |
Ipc: H04R 5/04 20060101ALI20240424BHEP Ipc: H04N 21/81 20110101ALI20240424BHEP Ipc: H04N 21/442 20110101ALI20240424BHEP Ipc: H04N 21/439 20110101ALI20240424BHEP Ipc: H04N 21/43 20110101ALI20240424BHEP Ipc: H04N 21/4363 20110101ALI20240424BHEP Ipc: H04N 21/436 20110101AFI20240424BHEP |
|
INTG | Intention to grant announced |
Effective date: 20240516 |