WO2020125110A1 - 音频发送、播放的方法及智能电视 - Google Patents

音频发送、播放的方法及智能电视 Download PDF

Info

Publication number
WO2020125110A1
WO2020125110A1 PCT/CN2019/107265 CN2019107265W WO2020125110A1 WO 2020125110 A1 WO2020125110 A1 WO 2020125110A1 CN 2019107265 W CN2019107265 W CN 2019107265W WO 2020125110 A1 WO2020125110 A1 WO 2020125110A1
Authority
WO
WIPO (PCT)
Prior art keywords
audio data
smart
invalid
control device
data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2019/107265
Other languages
English (en)
French (fr)
Inventor
宋子全
庞秀娟
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Qingdao Hisense Electronics Co Ltd
Original Assignee
Qingdao Hisense Electronics Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from CN201811565099.4A external-priority patent/CN109495776B/zh
Priority claimed from CN201811565963.0A external-priority patent/CN111355996A/zh
Priority claimed from CN201811565526.9A external-priority patent/CN111352605A/zh
Application filed by Qingdao Hisense Electronics Co Ltd filed Critical Qingdao Hisense Electronics Co Ltd
Priority to CN201980079748.6A priority Critical patent/CN113261300B/zh
Publication of WO2020125110A1 publication Critical patent/WO2020125110A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/439—Processing of audio elementary streams
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/81—Monomedia components thereof
    • H04N21/8106—Monomedia components thereof involving special audio data, e.g. different tracks for different languages
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/16—Sound input; Sound output
    • G06F3/165—Management of the audio stream, e.g. setting of volume, audio stream path
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H04N21/4104—Peripherals receiving signals from specially adapted client devices
    • H04N21/4126—The peripheral being portable, e.g. PDAs or mobile phones
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H04N21/4104—Peripherals receiving signals from specially adapted client devices
    • H04N21/4126—The peripheral being portable, e.g. PDAs or mobile phones
    • H04N21/41265—The peripheral being portable, e.g. PDAs or mobile phones having a remote control device for bidirectional communication between the remote control device and client device
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H04N21/422—Input-only peripherals, i.e. input devices connected to specially adapted client devices, e.g. global positioning system [GPS]
    • H04N21/42204—User interfaces specially adapted for controlling a client device through a remote control device; Remote control devices therefor
    • H04N21/42206—User interfaces specially adapted for controlling a client device through a remote control device; Remote control devices therefor characterized by hardware details
    • H04N21/4222—Remote control device emulator integrated into a non-television apparatus, e.g. a PDA, media center or smart toy
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/4302—Content synchronisation processes, e.g. decoder synchronisation
    • H04N21/4307—Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen
    • H04N21/43072—Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen of multiple content streams on the same device
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/436—Interfacing a local distribution network, e.g. communicating with another STB or one or more peripheral devices inside the home
    • H04N21/4363—Adapting the video stream to a specific local network, e.g. a Bluetooth® network
    • H04N21/43637—Adapting the video stream to a specific local network, e.g. a Bluetooth® network involving a wireless protocol, e.g. Bluetooth®, RF or wireless LAN [IEEE 802.11]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/442—Monitoring of processes or resources, e.g. detecting the failure of a recording device, monitoring the downstream bandwidth, the number of times a movie has been viewed, the storage space available from the internal hard disk
    • H04N21/44209—Monitoring of downstream path of the transmission network originating from a server, e.g. bandwidth variations of a wireless network
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/485—End-user interface for client configuration
    • H04N21/4852—End-user interface for client configuration for modifying audio parameters, e.g. switching between mono and stereo
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/488—Data services, e.g. news ticker
    • H04N21/4882—Data services, e.g. news ticker for displaying messages, e.g. warnings, reminders
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/85—Assembly of content; Generation of multimedia applications
    • H04N21/854—Content authoring
    • H04N21/8547—Content authoring involving timestamps for synchronizing content

Definitions

  • This application relates to the technical field of smart device interaction, in particular to a method of audio transmission and playback and a smart TV.
  • TV products are increasingly recognized and used by users.
  • many users in order to avoid affecting the rest of family members when watching TV programs, many users usually tune the TV to a mute state or turn down the TV volume, which affects the experience of watching TV programs to a certain extent.
  • the embodiments of the present application provide a method for sending and playing audio and a smart TV.
  • the control device receives a first system time data packet sent by a smart TV, where the first system time data packet includes the smart TV when the smart TV sends the first system time data packet to the control device The first system time;
  • the control device calculates the difference between the second system time of the control device and the first system time as the first difference when the control device receives the first system time data packet;
  • the control device receives audio data sent by the smart TV
  • control device calculates a network delay for transmitting the audio data between the smart TV and the control device.
  • the control device discards the audio data.
  • the calculating the network delay between the smart TV and the control device according to the first difference includes:
  • the fourth system time is a system time at which the control device receives the audio data
  • the network delay for transmitting the audio data between the smart TV and the control device is: the difference between the second difference and the first difference.
  • the calculating the network delay between the smart TV and the control device according to the first difference includes the following:
  • the control device parses the third system time carried in the audio data, where the third system time is the system time of the smart TV when the smart TV sends the audio data;
  • the control device obtains a fourth system time, where the fourth system time is a system time at which the control device receives the audio data;
  • the control device calculates the difference between the fourth system time and the third system time as a second difference
  • the control device determines that the network delay for transmitting the audio data between the smart TV and the control device is: the difference between the second difference and the first difference.
  • control device sends an audio data transmission instruction to the smart TV
  • the audio data transmission instruction is used to instruct the smart TV to perform at least one of the following items: transmit the audio data of the broadcasted program to the control device, turn off the sound of the smart TV device, and transfer the The volume output of the smart TV is set to 0.
  • control device receives an instruction to start the remote control application of the smart TV
  • the control device presents a first user interface capable of controlling the smart TV in response to an instruction to start the remote control application of the smart TV;
  • the first user interface includes a main control area and a function selection area, wherein the main control area includes a volume control area and a program switching area; the function selection area includes at least one of the following items: a sound transmission tab, The home tab to return to the home page of the remote control and the keyboard tab to call the numeric keypad.
  • control device receives the selection of the transmission tab
  • the control device presents a second user interface in response to the selection of the sound transmission tab
  • the device for displaying sound playback on the second user interface is the prompt information of the smart TV, and the sound transmission switching option bar for switching the device for playing sound from the smart TV to the control device.
  • control device receives an instruction to select the transmission switching option bar
  • the control device switches the sound playback device from the smart TV to the control device and presents a third user interface in response to an instruction to select the sound transmission switching option bar;
  • the device for playing sound displayed on the third user interface is the prompt information of the control device.
  • the control device transmits the received audio data to the audio player of the control device.
  • control device receives an instruction to close the transmission switching option bar
  • the control device sends an instruction to stop recording to the smart TV in response to an instruction to close the transmission switching option bar.
  • the method before the receiving an instruction to start the remote control application of the smart TV, the method further includes:
  • the control device suggests a communication connection with the smart TV.
  • An audio sending method provided by an embodiment of the present application includes:
  • the smart TV sends a first system time data packet to the control device, where the first system time data packet includes the smart TV’s time when the smart TV sends the first system time data packet to the control device First system time
  • the smart TV receives the audio data transmission instruction sent by the control device
  • the smart TV sends audio data to the control device, wherein the audio data carries a second system time when the smart TV sends the audio data, so that the control device calculates the smart TV and the The network delay for transmitting the audio data between the control devices, and judging whether to play the audio data according to the network delay.
  • control device including:
  • Memory configured to store computer programs
  • the processor is configured to execute the computer program to control the control device to perform the following steps:
  • the audio data is discarded.
  • the processor is further configured to execute the computer program to implement:
  • the fourth system time is a system time at which the control device receives the audio data
  • the network delay for transmitting the audio data between the smart TV and the control device is: the difference between the second difference and the first difference.
  • the processor is further configured to execute the computer program to implement:
  • the audio data transmission instruction is used to instruct the smart TV to perform at least one of the following: transmit the audio data of the broadcasted program to the control device, turn off the device audio of the smart TV, and transfer all The volume output of the smart TV is set to 0.
  • the processor is further configured to execute the computer instructions to implement:
  • the first user interface includes a main control area and a function selection area, wherein the main control area includes a volume control area and a program switching area; the function selection area includes at least one of the following items: a sound transmission tab, The home tab to return to the home page of the remote control and the keyboard tab to call the numeric keypad.
  • the processor is further configured to execute the computer instructions to implement:
  • the device for displaying sound playback on the second user interface is the prompt information of the smart TV, and the sound transmission switching option bar for switching the device for playing sound from the smart TV to the control device.
  • the processor is further configured to execute the computer instructions to implement:
  • the device for playing sound displayed on the third user interface is the prompt information of the control device.
  • the processor is further configured to execute the computer instructions to implement:
  • the received audio data is transmitted to the audio player of the control device.
  • the processor is further configured to execute the computer instructions to implement:
  • an instruction to stop recording is sent to the smart TV.
  • the processor is further configured to execute the computer instructions to implement:
  • the embodiment of the present application also relates to a smart TV, including:
  • Memory configured to store computer programs
  • the processor is configured to execute the computer program to control the smart TV:
  • a machine-readable non-volatile storage medium provided by an embodiment of the present application has computer instructions stored thereon, which are implemented when the computer instructions are executed:
  • the audio data is discarded.
  • the first unit receives a first system time data packet sent by a smart TV, where the first system time data packet includes a location where the smart TV sends the first system time data packet to the audio playback device Describe the first system time of smart TV;
  • a second unit configured to calculate the difference between the second system time of the audio playback device and the first system time when the audio playback device receives the first system time data packet as the first difference
  • the third unit is used to send audio data transmission instructions to the smart TV
  • the fourth unit is used to receive the audio data sent by the smart TV.
  • the fifth unit is used to calculate the network delay for transmitting the audio data between the smart TV and the audio playback device according to the first difference;
  • a smart TV provided by an embodiment of the present application includes:
  • a first sending unit configured to send a first system time data packet to a control device, wherein the first system time data packet includes when the smart TV sends the first system time data packet to the control device The first system time of the smart TV;
  • the receiving unit is used to receive audio data transmission instructions sent by the control device
  • a second sending unit for sending audio data to the control device, wherein the audio data carries the second system time when the smart TV sends the audio data, so that the control device calculates the smart TV and the A network delay for transmitting the audio data between the control devices, and judging whether to play the audio data according to the network delay.
  • the embodiment of the present application also relates to an audio sending method, including:
  • Smart TV collects the currently playing audio data
  • the smart TV In response to the audio data being invalid audio data, the smart TV adds invalid audio data indication information to the audio data, and compresses the audio data to generate an invalid audio data package;
  • the smart TV sends the invalid audio data packet to the control device.
  • the method further includes: in response to the audio data not being invalid audio data, sending the audio data to the control device.
  • the determining whether the audio data is invalid audio data includes:
  • the collected audio data is a complete frame of audio data
  • the character of the frame of audio data is a preset character, it is determined that the audio data is invalid audio data.
  • the method further includes:
  • the smart TV Before collecting the audio data of the currently playing content, the smart TV is controlled to enter a recording state, and audio recording is performed on the smart TV.
  • the invalid audio data is discarded.
  • the audio data in response to the audio data not being invalid audio data, the audio data is played.
  • the determining whether the audio data is invalid audio data according to the parsed data includes:
  • the indication information is located in the first N bytes of the audio data, and N is an integer greater than 1.
  • the playing the audio data includes: storing the audio data in a playing buffer.
  • the embodiment of the present application also relates to a smart TV, including:
  • Memory configured to store computer programs
  • a processor configured to execute the computer instructions to cause the smart TV to execute:
  • the processor is further configured to execute the computer instructions to cause the smart TV to execute:
  • the audio data is sent to the control device.
  • the determining whether the audio data is invalid audio data includes:
  • the collected audio data is a complete frame of audio data
  • the character of the frame of audio data is a preset character, it is determined that the audio data is invalid audio data.
  • the processor is further configured to execute the computer instructions to cause the smart TV to execute:
  • control device including:
  • Memory configured to store computer instructions
  • a processor configured to execute the computer instructions to cause the control device to execute:
  • the invalid audio data is discarded.
  • the embodiment of the present application also relates to a smart TV, including:
  • the receiving unit is used to receive audio data transmission instructions sent by the control device
  • Collection unit used to collect the currently playing audio data
  • a determining unit configured to determine whether the audio data is invalid audio data
  • the processing unit is configured to add invalid audio data indication information to the audio data in response to the audio data being invalid audio data, and perform compression processing on the audio data to generate an invalid audio data packet;
  • the sending unit is configured to send the invalid audio data packet to the control device.
  • the receiving unit is used to receive the audio data sent by the smart TV
  • a parsing unit used to parse the audio data to obtain parsed data
  • a determining unit configured to determine whether the audio data is invalid audio data according to the parsed data
  • a processing unit configured to discard the invalid audio data in response to the audio data being invalid audio data
  • the playing unit is configured to play the audio data in response to the audio data being valid audio data.
  • Memory used to store computer-readable programs
  • the processor is configured to call the computer-readable program to implement at least one of the foregoing methods.
  • Embodiments of the present application provide a computer-readable non-volatile storage medium that stores a computer-readable program, and the computer-readable program is used by the computer to implement at least one of the foregoing methods .
  • FIG. 1 is a schematic diagram of an example of interaction between a smart TV and a control device according to an embodiment of the present application
  • FIG. 2 is a schematic diagram of a connection state between a mobile terminal and a smart TV according to an embodiment of the present application
  • FIG. 3 is a schematic diagram of a first user interface of a mobile terminal according to an embodiment of the present application.
  • FIG. 4 is a schematic diagram of a second user interface of a mobile terminal according to an embodiment of the present application.
  • FIG. 5 is a schematic diagram of a third user interface of a mobile terminal according to an embodiment of the present application.
  • FIG. 6 is a first schematic diagram of an audio playback method according to an embodiment of the present application.
  • FIG. 7 is a first schematic diagram of an audio sending method according to an embodiment of the present application.
  • FIG. 8 is a first schematic diagram of the overall process of delay optimization according to an embodiment of the present application.
  • FIG. 9 is a schematic diagram of an audio playback device according to an embodiment of the present application.
  • FIG. 10 is a first schematic diagram of a smart TV according to an embodiment of the present application.
  • FIG. 11 is a first schematic diagram of a control device according to an embodiment of the present application.
  • FIG. 12 is a second schematic diagram of a smart TV according to an embodiment of the present application.
  • FIG. 13 is a second schematic diagram of an audio sending method according to an embodiment of the present application.
  • FIG. 14 is a second schematic diagram of an audio playback method according to an embodiment of the present application.
  • 15 is a second schematic diagram of the overall process of delay optimization according to an embodiment of the present application.
  • FIG. 16 is a third schematic diagram of a smart TV according to an embodiment of the present application.
  • 17 is a second schematic diagram of a control device according to an embodiment of the present application.
  • FIG. 18 is a fourth schematic diagram of a smart TV according to an embodiment of the present application.
  • FIG. 19 is a third schematic diagram of a control device according to an embodiment of the present application.
  • the mobile device is used as a sound output device of the smart TV.
  • the smart TV captures the current sound in real time and transmits it to the mobile device for real-time playback.
  • the sound may be delayed due to network reasons and the mobile phone playback cache cannot be cleared in time.
  • the embodiments of the present application provide a method and device for audio playback and transmission.
  • the playback buffer is reduced, thereby reducing the playback delay and avoiding the larger network delay.
  • live TV refers to a TV production broadcast in real time or substantially synchronized with the time of occurrence of the event.
  • VOD Video On Demand
  • the term "Video On Demand (VOD)” as used in this application refers to a system and process that allows users to select and watch/listen to on-demand video or audio content.
  • the VOD system can distribute content to watch real-time content or download it to a storage medium for later viewing.
  • the embodiments of the present application provide an application scenario in which a smart TV interacts with a control device (such as a mobile terminal).
  • the smart TV 100 may be configured to have functions of entertainment, business applications, social interaction, content creation, and/or consumption, and one or more control devices are used to organize and control communication with the smart TV 100. Therefore, it can be understood that the smart TV and/or the control device can be used to enhance the user interactive experience, whether at home or in the workplace.
  • the smart TV 100 may be configured to receive and process various user and/or device inputs.
  • Mobile terminals such as mobile phones, tablet computers, computers, notebook computers, netbooks, and other smart devices can be used to control the smart TV 100.
  • an application program running on a smart device is used to control the smart TV 100.
  • the smart TV 100 may be configured to receive input through various input devices, including but not limited to video, audio, radio, light, haptics, and combinations thereof.
  • various input devices including but not limited to video, audio, radio, light, haptics, and combinations thereof.
  • the user can talk to the smart TV 100 in a conversational manner.
  • the smart TV 100 is like a smart personal assistant of a smart device and a voice-controlled navigation application (such as Apple's Siri, Android's Skyvi, Robin, Iris, and other applications) to receive and process voice commands.
  • a voice-controlled navigation application such as Apple's Siri, Android's Skyvi, Robin, Iris, and other applications
  • An operation step of a user interface of a mobile terminal for interacting with a user provided by an embodiment of the present application.
  • an embodiment of the present application provides a schematic diagram of a connection state between a mobile terminal and a smart TV.
  • the mobile terminal can establish a communication connection with the smart TV according to various connection protocols, and the mobile terminal can perform remote control operations on the smart TV, or can be installed on the smart TV.
  • Manipulated applications can be performed.
  • the mobile terminal when the mobile terminal receives an instruction to start the remote control application of the smart TV, the mobile terminal can start the remote control application and present a first user interface that can control the smart TV.
  • the first user interface includes the main A control area and a function selection area, wherein the main control area includes a volume control area and a program switching area; the function selection area includes at least one of the following items: a sound transmission tab, a home page for returning to the remote control home page Tabs and keyboard tabs for calling the numeric keypad.
  • the mobile terminal can realize interaction with the smart TV by operating on the first user interface. Exemplarily, when the user inputs an instruction in the volume control area, the volume output of the smart TV can be controlled.
  • the mobile terminal receives and responds to the selection of the sound transmission tab, and presents a second user interface, wherein the device displaying the currently playing sound on the second user interface is a prompt of the smart TV Information, and an audio transmission switching option bar for switching the sound playback terminal to the mobile terminal.
  • the mobile terminal receives and responds to an instruction to select the transmission switching option bar, switches the sound playing device to the mobile device, and presents a third user interface, wherein the third user interface
  • the device displaying the currently playing sound is the prompt information of the mobile terminal.
  • the smart TV adjusts the volume of its audio player to mute or 0, so that the audio data is still played in the audio player, but is not output from the speaker, and the audio data is collected Record and send to mobile terminal.
  • the smart TV After the smart TV receives the audio data transmission instruction of the mobile terminal, it encodes the collected audio data into a pulse code modulation (Pulse Code Modulation, PCM) data stream, which is transmitted to the mobile terminal through the network, and the mobile terminal plays the TV sound in real time.
  • PCM Pulse Code Modulation
  • the audio capture coding function provided by the main processing chip (CPU) of the smart TV machine is used to follow the sound of the current screen of the smart TV according to a certain sampling format (such as 8KHz sampling rate, 16Bit sampling accuracy, two-channel) to capture, provide audio data in PCM format, uncompressed and transmitted to the mobile terminal through the wired network or wireless network, the mobile terminal directly receives the audio data to the hardware decoder after receiving the audio data To decode and play.
  • a certain sampling format such as 8KHz sampling rate, 16Bit sampling accuracy, two-channel
  • an audio playback method provided by an embodiment of the present application is applied to a control device.
  • the method includes:
  • the first system time data packet includes the first system time of the smart TV when the smart TV sends the first system time data packet to the control device; for example, the control device is a mobile terminal.
  • the preset threshold can be 200 milliseconds.
  • the audio data is discarded and not sent to the audio player of the mobile terminal for playback (playing audio data with a large delay is meaningless);
  • the audio data is sent to the audio player of the mobile terminal for playback.
  • the mobile terminal may be a mobile phone. Before playing the audio data, the mobile phone puts the audio data received by the upper layer in a buffer queue, and the audio player of the mobile phone takes the audio data from the buffer queue when playing. The audio data will be deleted from the cache queue. If the upper layer application writes data to the cache queue too fast, and the player takes data from the cache queue relatively slowly, the cache data in this queue will increase.
  • the control device is a mobile phone
  • the first system time is the system time Time1_tv of the smart TV
  • the second system time is the system time Time1_phone of the mobile phone
  • the first system time is The second system time may be different.
  • the time on the TV has not been synchronized with the network (for example, the router is not connected to the external network), and the time on the mobile phone has been synchronized with the network; another reason is that the TV and the mobile phone The time has been synchronized with the network, but the time of the synchronized TV and mobile phone can not be exactly the same, there will be a few milliseconds to hundreds of milliseconds, or even an error of about 1 second.
  • step S104 further includes:
  • the fourth system time is a system time at which the control device receives the audio data
  • the network delay for transmitting the audio data between the smart TV and the control device is: the difference between the second difference and the first difference.
  • the third system time is the TV system time Time2_tv
  • the fourth system time is the mobile phone system time Time2_phone.
  • Time2_diff Time2_phone-Time2_tv.
  • Time_delay Time2_diff–Time1_diff.
  • (Time2_phone-Time2_tv) represents the network delay when transmitting audio data from the smart TV to the mobile phone plus the system time difference between the smart TV and the mobile phone
  • (Time1_phone-Time1_tv) represents the system time difference between the smart TV and the mobile phone (due to the The amount of data transmitted by the mobile phone is only one time, there is no audio data, so the network delay during the transmission process is very small and can be ignored)
  • (Time2_diff–Time1_diff) represents the network delay when transmitting audio data from the smart TV to the mobile phone.
  • the method further includes:
  • the control device sends an audio data transmission instruction to the smart TV
  • the audio data transmission instruction is used to instruct the smart TV to perform at least one of the following items: transmit the audio data of the broadcasted program to the control device, turn off the sound of the smart TV device, and transfer the The volume output of the smart TV is set to 0.
  • the method further includes:
  • the control device receives an instruction to start the remote control application of the smart TV
  • the control device presents a first user interface capable of controlling the smart TV in response to an instruction to start the remote control application of the smart TV;
  • the first user interface includes a main control area and a function selection area, wherein the main control area includes a volume control area and a program switching area; the function selection area includes at least one of the following items: a sound transmission tab, The home tab to return to the home page of the remote control and the keyboard tab to call the numeric keypad.
  • the method further includes:
  • the control device receives the selection of the sound transmission tab
  • the control device presents a second user interface in response to the selection of the sound transmission tab
  • the device for displaying sound playback on the second user interface is the prompt information of the smart TV, and the sound transmission switching option bar for switching the device for playing sound from the smart TV to the control device.
  • the method further includes:
  • the control device receives an instruction to select the transmission switching option bar
  • the control device switches the sound playback device from the smart TV to the control device and presents a third user interface in response to an instruction to select the sound transmission switching option bar;
  • the device for playing sound displayed on the third user interface is the prompt information of the control device.
  • the method further includes:
  • the control device After the device for playing sound is switched from the smart TV to the control device, the control device transmits the received audio data to the audio player of the control device.
  • the method further includes:
  • the control device receives an instruction to close the transmission switching option bar
  • the control device sends an instruction to stop recording to the smart TV in response to an instruction to close the transmission switching option bar.
  • the method before the receiving an instruction to start the remote control application of the smart TV, the method further includes:
  • the control device suggests a communication connection with the smart TV.
  • an audio transmission method which is applied to a smart TV, and includes:
  • S202 Receive audio data transmission instructions sent by the control device
  • a smart TV Before sending a PCM format audio data to a mobile phone, a smart TV will add (smart TV) current system time information in front of the PCM format audio data.
  • the overall specific processing flow of a smart TV and a mobile phone includes:
  • Time_systemDiff Time1_phone-Time1_tv
  • the smart TV obtains the current system time Time2_tv every time it captures the PCM format audio data, and then packages the PCM format audio data and the system time Time2_tv.
  • the packaging format is shown in Table 1:
  • the calculation method of audio data is:
  • the calculation formula (1) of the audio data size per frame is:
  • Time_delay Time_netDiff-Time_systemDiff.
  • the mobile phone When the network delay Time_delay is greater than the preset threshold K, for example K is 200ms, the mobile phone discards the audio data and does not send it to the audio player (playing audio data with a large delay, it does not make sense); when the network delay Time_delay is less than or When it is equal to the preset threshold K, the mobile phone sends the audio data to the audio player for playback.
  • K the preset threshold
  • Time_systemDiff Time1_phone-Time1_tv
  • Time_systemDiff It is 1 second and 10 milliseconds, that is to say, the system time of the mobile phone is 1 second and 10 milliseconds faster than the system time of the smart TV.
  • Time_netDiff Time2_phone-Time2_tv
  • Time_netDiff 1 second and 110 milliseconds.
  • Time_delay Time_netDiff-Time_systemDiff
  • Time_delay 100 milliseconds.
  • an audio playback device provided by an embodiment of the present application includes:
  • the first unit 11 receives a first system time data packet sent by a smart TV, where the first system time data packet includes when the smart TV sends the first system time data packet to the audio playback device The first system time of the smart TV;
  • the second unit 12 is configured to calculate the difference between the second system time of the audio playback device and the first system time as the first difference when the audio playback device receives the first system time data packet;
  • the third unit 13 is used to send audio data transmission instructions to the smart TV
  • the fourth unit 14 is used to receive audio data sent by a smart TV
  • the fifth unit 15 is used to calculate the network delay for transmitting the audio data between the smart TV and the audio playback device according to the first difference;
  • the sixth unit 16 is configured to discard the audio data when the network delay is greater than a preset threshold.
  • the smart TV provided by the embodiment of the present application includes:
  • the first sending unit 21 is configured to send a first system time data packet to a control device, where the first system time data packet includes the smart TV sending the first system time data packet to the control device The first system time of the smart TV;
  • the receiving unit 22 is used to receive audio data transmission instructions sent by the control device;
  • the second sending unit 23 is used to send audio data to the control device, wherein the audio data carries the second system time when the smart TV sends the audio data, so that the control device calculates the smart TV and The network delay for transmitting the audio data between the control devices, and judging whether to play the audio data according to the network delay.
  • an embodiment of the present application further provides a control device, including:
  • the memory 310 is configured to store a computer program
  • the processor 300 is configured to execute the computer program to control the control device to perform the following steps:
  • the audio data is discarded.
  • control discards the audio data, thereby reducing the playback delay of the control device.
  • the control device further includes a user interface 320 and a bus interface 330.
  • the processor 300 and the memory 310 are interactively connected through the bus interface 330, and the user interface 320 and the bus interface 330 are connected.
  • the processor 300 is further configured to execute the computer program to implement the above audio playback method.
  • a smart TV also provided by an embodiment of the present application includes:
  • the memory 410 is configured to store a computer program
  • the processor 400 is configured to execute the computer program to control the electronic device to perform the following steps:
  • the control device calculates a network delay for transmitting the audio data between the smart TV and the control device, and determines whether to play the audio data according to the network delay, thereby reducing the playback delay of the control device.
  • the smart TV further includes a user interface 420 and a bus interface 430.
  • the processor 400 and the memory 410 are interactively connected through the bus interface 430, and the user interface 420 and the bus interface 430 are connected.
  • the smart TV continuously records sound and sends it to the mobile terminal through the network. Before sending the recorded sound, the smart TV performs real-time "valid audio data frame” analysis on the recorded sound and sends the "valid audio data" in real time; if the smart TV does not play sound or video files, the smart TV records a The sound of the frame is continuous 0x00 (0x00 represents a hexadecimal number, 0x is a hexadecimal prefix). For this invalid data frame (null data) that is all 0x00, it also occupies a The amount of network data of the frame; after the mobile terminal receives it, add it to the player's cache, and also increase the mobile phone's playback cache.
  • an embodiment of the present application also provides an audio transmission method, which is applied to a smart TV.
  • the method includes:
  • the smart TV will continuously record the sound being played by the system, that is, continuously collect the audio data of the currently playing program. During the recording process, the smart TV does not detect whether there is an audio file output at this moment, but is always in the recording stage.
  • the user chooses to transmit instructions to the audio data through the control device during the playing of the football program, and the smart TV starts recording.
  • the user quits the football program due to misoperation or reselection or viewing of the viewing program.
  • the recording does not It will be terminated because of this. Therefore, during the period when no football program is playing, the audio data collected by the smart TV is actually empty, and such audio data is invalid audio data.
  • the valid audio data is sent to the control device.
  • the collected audio data is a complete frame of audio data
  • the character of the frame of audio data is a preset character, it is determined that the audio data is invalid audio data.
  • the preset character is 0x00.
  • the audio data of all frames of 0x00 is called invalid audio data.
  • a left and right byte of a frame of data can be cyclically detected. If one of the audio data is not 0x00, the audio data is determined to be Effective audio data; if each byte in the audio data is 0x00, it is determined that the audio data is invalid audio data.
  • step S302 further includes:
  • invalid audio data indication information is added to the invalid bytes, and the invalid audio data is compressed to generate an invalid audio data packet.
  • the audio capture encoding function provided by the main processing chip (CPU) of the smart TV machine is used to follow the sound of the current screen of the smart TV in a certain sampling format For example, 8KHz sampling rate, 16Bit sampling accuracy, two channels) to capture, provide audio data in PCM format.
  • a certain sampling format For example, 8KHz sampling rate, 16Bit sampling accuracy, two channels
  • the smart TV collects the audio data, it packs the audio data and sends it to the mobile phone.
  • the data package includes the collected audio data and the current system time packaging format as shown in Table 1 above.
  • the calculation method of the size of each frame of audio data is:
  • the calculation formula (2) of the audio data size per frame is:
  • the next frame of data occupies 640 bytes. If the audio data is valid audio data, the 640 bytes of valid audio data is sent to the control device. If the audio data is invalid data, for such invalid data frames that are all 0x00, a frame of network data is also occupied during transmission. After the mobile terminal receives this invalid data frame that is all 0x00, Writing to the player's cache will also increase the phone's playback cache.
  • these invalid audio data are compressed, and not all 640 bytes are sent, but the audio data whose entire section is 0x00 is compressed into M bytes, that is, the invalid section
  • the audio data is re-assigned as a data stream of M characters.
  • the compressed audio data is specifically expressed as 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00.
  • M is a positive integer less than 640. In some embodiments, 1 ⁇ M ⁇ 100.
  • the transmission of invalid audio data is also used to maintain the communication connection between the smart TV and the mobile terminal. If it is stipulated that the smart TV needs to send a data packet to the mobile terminal every 20ms to ensure the communication connection, when the smart TV does not send an audio data packet at 20ms, it will be disconnected. The mobile terminal cannot receive audio data in real time. Therefore, although there is no actual audio data, the smart TV will still send a data packet to the mobile terminal every 20ms.
  • an instruction message is added to indicate the data packet
  • the data in is invalid data.
  • the first N bytes are added before the M bytes, and N is an integer greater than 1.
  • the instruction information is located in the first 5 bytes of the audio data.
  • the instruction information can be specifically expressed as 0x01, 0x02, 0x03, 0x04, and 0x05. It can be agreed in advance with the control device which characters are used to indicate whether the audio data is valid.
  • the generated invalid data packet includes invalid audio data indication information and compressed M-bit invalid audio data.
  • the invalid audio data indication information is 0x01, 0x02, 0x03, 0x04, and 0x05.
  • the compressed audio data is specifically expressed as 0x01 0x02 0x03 0x04 0x05 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
  • step S301 before step S301, it further includes:
  • an audio playback method also provided by an embodiment of the present application includes:
  • S403 Determine whether the audio data is invalid audio data according to the parsed data
  • the playback buffer of the control device can be reduced, and the real-time playback delay of audio can be optimized.
  • the invalid audio data is discarded and not written into the cache of the mobile phone player.
  • the audio data is played.
  • the smart TV after receiving the audio data sent by the smart TV, according to the indication information in the audio data, it is determined whether the audio data is invalid audio data.
  • the indication information is located in the first N bytes of the audio data, and N is an integer greater than 1.
  • the indication information is located in the first 5 bytes of the audio data, and the indication information can be specifically expressed as 0x01, 0x02, 0x03, 0x04, and 0x05. It can be agreed in advance with the smart TV which characters are used to indicate whether the audio data is valid.
  • the valid audio data is stored in the playback buffer.
  • a mobile terminal Before playing audio data, a mobile terminal, such as a mobile phone, puts the audio data received by the upper layer in a buffer queue.
  • the audio player of the mobile phone plays the audio data from the buffer queue, after the playback is completed, the audio data that has been played will It is deleted from the cache queue. If the upper-layer application writes data to the cache queue too fast, and the player fetches data from the cache queue relatively slowly, the cache data in this queue will increase.
  • the overall specific processing flow of a smart TV and a mobile terminal includes:
  • the mobile terminal is, for example, a mobile phone. After receiving the connection message that the mobile phone is connected to the smart TV, the smart TV captures the sound of the whole machine (smart TV) and judges whether the captured sound is valid audio.
  • the specific judgment method is as follows:
  • this frame of data is all 0x00, that is to say, whether the 640 bytes of data collected by the smart TV at a time are all 0x00.
  • the data is invalid audio data; if the data of this frame is not all 0x00, it is composed of some data such as 0x12, it means that this data is valid audio data.
  • the valid audio data is sent to the mobile terminal frame by frame; when the data collected by the smart TV is invalid audio data, the invalid audio data is optimized first, that is 640 The invalid audio data whose byte data is all 0x00 is re-assigned to a 20-character data stream (or fake data).
  • the first five digits can be defined as 0x01, 0x02, 0x03, 0x04, 0x05 (that is, used for Indication information indicating invalid audio data, the indication information is not limited to 0x01 0x02 0x03 0x04 0x05, the first five digits can also be defined as 0x01 0x01 0x02 0x02 0x02, etc.), the last 15 digits are all 0x00, which can be expressed as 0x01 0x02 0x03 0x04 0x05 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
  • the mobile terminal When the mobile terminal receives the audio data packet stream sent by the smart TV, first determine whether the received audio data is valid audio data. If the mobile terminal receives the above data stream with 20 bytes: 0x01 0x02 0x03 0x04 0x05 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00, then by judging the first 5 digits of the data 0x01 0x02 0x03 0x04 0x05, it is determined that the received audio data is invalid audio Sound), will not write data into the cache of the mobile phone player, discard this invalid audio data (this can play the role of clearing the audio cache data of the mobile phone system); if the mobile terminal receives not the above 20 bytes Data,
  • a smart TV provided by an embodiment of the present application includes:
  • the receiving unit 31 is used to receive audio data transmission instructions sent by the control device;
  • the collection unit 32 is used to collect the currently playing audio data
  • the determining unit 33 is configured to determine whether the audio data is invalid audio data
  • the processing unit 34 is configured to add invalid audio data indication information to the audio data in response to the audio data being invalid audio data, and perform compression processing on the audio data to generate an invalid audio data packet;
  • the sending unit 35 is configured to send the invalid audio data packet to the control device.
  • a control device provided by an embodiment of the present application includes:
  • the receiving unit 41 is used to receive audio data sent by a smart TV
  • the parsing unit 42 is configured to parse the audio data to obtain parsed data
  • the determining unit 43 is configured to determine whether the audio data is invalid audio data according to the parsed data
  • the processing unit 44 is configured to discard the invalid audio data in response to the audio data being invalid audio data
  • the playing unit 45 is configured to play the audio data in response to the audio data being valid audio data.
  • an embodiment of the present application further provides a smart TV, including:
  • the processor 600 is configured to read the computer program in the memory 610 to control the smart TV to perform the following processes:
  • Using this smart TV can reduce the burden of network transmission and optimize the delay of real-time audio playback.
  • the control device further includes a user interface 620 and a bus interface 630.
  • the processor 600 and the memory 610 are interactively connected through the bus interface 630, and the user interface 620 and the bus interface 630 are connected.
  • the processor 600 is configured to read the computer program to control the smart TV: in response to the audio data not being invalid audio data, sending the audio data to the control device.
  • the processor 600 is configured to read the computer program to control the smart TV:
  • the collected audio data is a complete frame of audio data
  • the character of the frame of audio data is a preset character, it is determined that the audio data is invalid audio data.
  • the processor 600 is configured to read the computer program to control the smart TV: before the audio data of the currently playing content is collected, control the smart TV to enter a recording state, The TV records audio.
  • a control device also provided in an embodiment of the present application includes:
  • the processor 800 is used to read the program in the memory 810 and perform the following processes:
  • the invalid audio data is discarded.
  • the playback buffer of the control device can be reduced, and the real-time playback delay of audio can be optimized.
  • the smart TV further includes a user interface 820 and a bus interface 830.
  • the processor 800 and the memory 810 are interactively connected through the bus interface 830, and the user interface 820 and the bus interface 830 are connected.
  • the audio data in response to the audio data not being invalid audio data, the audio data is played.
  • the determining whether the audio data is invalid audio data according to the parsed data includes:
  • the indication information is located in the first N bytes of the audio data, and N is an integer greater than 1.
  • the playing the audio data includes: storing the audio data in a playing buffer.
  • the bus architecture may include any number of interconnected buses and bridges, specifically one or more processors and memories represented by processors 300, 400, 600, or 800
  • the various circuits of the memory represented by 310, 410, 610, or 810 are linked together.
  • the bus architecture can also link various other circuits such as peripheral devices, voltage regulators, and power management circuits, etc., which are well known in the art, and therefore, they will not be further described in this article.
  • the bus interface provides an interface.
  • the display terminal may specifically be a desktop computer, a portable computer, a smart phone, a tablet computer, a personal digital assistant (Personal Digital Assistant, PDA), or the like.
  • the display terminal may include a central processing unit (CPU), memory, input/output devices, etc.
  • the input device may include a keyboard, a mouse, a touch screen, etc.
  • the output device may include a display device, such as a liquid crystal display (Liquid Crystal Display, LCD), cathode ray tube (Cathode Ray Tube, CRT), etc.
  • the user interface 320, 420, 620, or 820 may be an interface that can be externally connected to a required device.
  • the connected devices include but are not limited to a keypad, a display, a speaker, a microphone, and a joystick Wait.
  • the processor is responsible for managing the bus architecture and normal processing, and the memory can store data used by the processor when performing operations.
  • the processor may be a CPU (Central Embedded Device), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), or CPLD (Complex Programmable Logic Device, complex programmable logic device).
  • CPU Central Embedded Device
  • ASIC Application Specific Integrated Circuit
  • FPGA Field-Programmable Gate Array
  • CPLD Complex Programmable Logic Device, complex programmable logic device
  • the memory may include a read-only memory (ROM) and a random access memory (RAM), and provide the processor with program instructions and data stored in the memory.
  • ROM read-only memory
  • RAM random access memory
  • the memory may be used to store the program of any of the methods provided in the embodiments of the present application.
  • the processor calls the program instructions stored in the memory, and the processor is configured to execute any of the methods provided in the embodiments of the present application according to the obtained program instructions.
  • An embodiment of the present application provides a computer storage medium for storing computer program instructions for the device provided by the embodiment of the present application, which includes a program for executing any method provided by the embodiment of the present application.
  • the computer storage medium may be any available medium or data storage device that can be accessed by the computer, including but not limited to magnetic storage (such as floppy disk, hard disk, magnetic tape, magneto-optical disk (MO), etc.), optical storage (such as CD, DVD, BD, HVD, etc.), and semiconductor memory (such as ROM, EPROM, EEPROM, non-volatile memory (NAND FLASH), solid-state hard disk (SSD)), etc.
  • magnetic storage such as floppy disk, hard disk, magnetic tape, magneto-optical disk (MO), etc.
  • optical storage such as CD, DVD, BD, HVD, etc.
  • semiconductor memory such as ROM, EPROM, EEPROM, non-volatile memory (NAND FLASH), solid-state hard disk (SSD)
  • the audio transmission and playback method and smart TV provided by the embodiments of the present application are used to reduce the network transmission burden and the playback buffer of the control device, thereby optimizing the real-time audio playback delay.
  • the phenomenon that the control device cannot synchronize the audio data with the smart TV in real time due to the large network delay is avoided.
  • a method and device for audio playback and transmission are used to reduce the burden of network transmission and reduce the playback buffer of the control device, thereby optimizing the real-time playback delay of audio.
  • the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Moreover, the present application may take the form of a computer program product implemented on one or more computer usable storage media (including but not limited to disk storage and optical storage, etc.) containing computer usable program code.
  • a computer usable storage media including but not limited to disk storage and optical storage, etc.
  • These computer program instructions may also be stored in a computer readable memory that can guide a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, the instructions
  • the device implements the functions specified in one block or multiple blocks of the flowchart one flow or multiple flows and/or block diagrams.
  • These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operating steps are performed on the computer or other programmable device to produce computer-implemented processing, which is executed on the computer or other programmable device
  • the instructions provide steps for implementing the functions specified in one block or multiple blocks of the flowchart one flow or multiple flows and/or block diagrams.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Human Computer Interaction (AREA)
  • Theoretical Computer Science (AREA)
  • General Health & Medical Sciences (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Health & Medical Sciences (AREA)
  • Databases & Information Systems (AREA)
  • Computer Security & Cryptography (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)

Abstract

本申请公开了一种音频发送、播放的方法及智能电视，用以减轻网络传输负担，优化音频的实时播放延迟。本申请实施例提供的一种音频发送方法，包括：智能电视采集当前播放的音频数据；确定所述音频数据是否为无效音频数据；响应于所述音频数据是无效音频数据，则所述智能电视在所述音频数据中添加无效音频数据指示信息，并对所述音频数据进行压缩处理，生成无效音频数据包；所述智能电视发送所述无效音频数据包至控制装置。

Description

音频发送、播放的方法及智能电视
本申请要求在2018年12月20日提交中国专利局、申请号为201811565526.9、发明名称为“一种音频播放、发送的方法及装置”的中国专利申请的优先权、在2018年12月20日提交中国专利局、申请号为201811565099.4、发明名称为“一种音频发送、播放的方法及智能终端”的中国专利申请的优先权、以及在2018年12月20日提交中国专利局、申请号为201811565963.0、发明名称为“一种音频播放方法及计算设备”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及智能设备互动技术领域，尤其涉及一种音频发送、播放的方法及智能电视。
背景技术
随着电视技术的不断发展，电视产品越来越受到广大用户的认可和使用。在一些场景中，许多用户在看电视节目时，为了避免影响家庭成员的休息，通常将电视调成静音状态或将电视音量调小，这在一定程度上影响到电视节目收看的体验。
发明内容
本申请实施例提供了一种音频发送、播放的方法及智能电视。
本申请实施例提供的一种音频播放方法，包括：
控制装置接收智能电视发送的第一系统时间数据包，其中，所述第一系统时间数据包中包括所述智能电视在向所述控制装置发送所述第一系统时间数据包时所述智能电视的第一系统时间；
所述控制装置计算所述控制装置接收到所述第一系统时间数据包时所述控制装置的第二系统时间与所述第一系统时间的差值作为第一差值；
所述控制装置接收所述智能电视发送的音频数据；
所述控制装置根据所述第一差值，计算所述智能电视与所述控制装置之间传输所述音频数据的网络延时；以及
当所述网络延时大于预设阈值时，所述控制装置丢弃所述音频数据。
在一些实施例中，所述根据所述第一差值，计算智能电视与控制装置之间传输所述音频数据的网络延时，包括：
解析所述音频数据中携带的第三系统时间，其中，所述第三系统时间是所述智能电视发送所述音频数据时的系统时间；
获取第四系统时间，其中，所述第四系统时间是所述控制装置接收到所述音频数据的 系统时间；
计算所述第四系统时间与所述第三系统时间的差值作为第二差值；
确定所述智能电视与控制装置之间传输所述音频数据的网络延时为：所述第二差值与所述第一差值的差值。
在一些实施例中，所述根据所述第一差值，计算所述智能电视与所述控制装置之间传输所述音频数据的网络延时，包括：
所述控制装置解析所述音频数据中携带的第三系统时间，其中，所述第三系统时间是所述智能电视发送所述音频数据时的所述智能电视的系统时间；
所述控制装置获取第四系统时间，其中，所述第四系统时间是所述控制装置接收到所述音频数据的系统时间；
所述控制装置计算所述第四系统时间与所述第三系统时间的差值作为第二差值；以及
所述控制装置确定所述智能电视与所述控制装置之间传输所述音频数据的网络延时为：所述第二差值与所述第一差值的差值。
在一些实施例中，所述控制装置向所述智能电视发送音频数据传输指令；
其中，所述音频数据传输指令用于指示所述智能电视执行以下项中的至少一者：将播放的节目的音频数据传输至所述控制装置，关闭所述智能电视设备音响，和将所述智能电视的音量输出设置为0。
在一些实施例中，所述控制装置接收启动所述智能电视的遥控应用的指令；
所述控制装置响应于启动所述智能电视的遥控应用的指令，呈现能对所述智能电视进行操控的第一用户界面；
其中该第一用户界面包括主控制区和功能选择区，其中，所述主控制区包括音量控制区和节目切换区；所述功能选择区包括以下项中的至少一者：传音选项卡、用于返回遥控器主页的主页选项卡和用于调用数字键盘的键盘选项卡。
在一些实施例中，所述控制装置接收对所述传音选项卡的选择；
所述控制装置响应于对所述传音选项卡的选择，呈现第二用户界面；
其中，所述第二用户界面上显示播放声音的设备为所述智能电视的提示信息，和用于将播放声音的设备从所述智能电视切换至所述控制装置的传音切换选项栏。
在一些实施例中，所述控制装置接收选中所述传音切换选项栏的指令；
所述控制装置响应于选中所述传音切换选项栏的指令，将播放声音的设备从所述智能电视切换为所述控制装置并呈现第三用户界面；
其中，所述第三用户界面上显示有播放声音的设备为所述控制装置的提示信息。
在一些实施例中，在播放声音的设备从所述智能电视切换为所述控制装置后，所述控制装置将接收到的所述音频数据传输至所述控制装置的音频播放器。
在一些实施例中，所述控制装置接收关闭所述传音切换选项栏的指令；
所述控制装置响应于关闭所述传音切换选项栏的指令，向所述智能电视发送停止录音的指令。
在一些实施例中，在所述接收启动所述智能电视的遥控应用的指令之前，所述方法还包括：
所述控制装置建议与所述智能电视之间的通信连接。
本申请实施例提供的一种音频发送方法，该方法包括：
智能电视向控制装置发送第一系统时间数据包，其中，所述第一系统时间数据包中包括所述智能电视在向所述控制装置发送所述第一系统时间数据包时所述智能电视的第一系统时间；
所述智能电视接收所述控制装置发送的音频数据传输指令；
所述智能电视发送音频数据至所述控制装置，其中，所述音频数据中携带所述智能电视发送所述音频数据时的第二系统时间，使得所述控制装置计算所述智能电视与所述控制装置之间传输所述音频数据的网络延时，并根据所述网络延时判断是否播放所述音频数据。
本申请实施例还涉及一种控制装置，包括：
存储器，被配置为存储计算机程序；
处理器，被配置为执行所述计算机程序以控制所述控制装置执行以下步骤：
接收智能电视发送的第一系统时间数据包，其中，所述第一系统时间数据包中包括所述智能电视在向所述控制装置发送所述第一系统时间数据包时所述智能电视的第一系统时间；
计算所述控制装置接收到所述第一系统时间数据包时所述控制装置的第二系统时间与所述第一系统时间的差值作为第一差值；
接收所述智能电视发送的音频数据；
根据所述第一差值，计算所述智能电视与所述控制装置之间传输所述音频数据的网络延时；以及
当所述网络延时大于预设阈值时，丢弃所述音频数据。
在一些实施例中，所述处理器，还被配置为执行所述计算机程序以实现：
解析所述音频数据中携带的第三系统时间，其中，所述第三系统时间是所述智能电视发送所述音频数据时的所述智能电视的系统时间；
获取第四系统时间，其中，所述第四系统时间是所述控制装置接收到所述音频数据的系统时间；
计算所述第四系统时间与所述第三系统时间的差值作为第二差值；以及
确定所述智能电视与所述控制装置之间传输所述音频数据的网络延时为：所述第二差 值与所述第一差值的差值。
在一些实施例中，所述处理器，还被配置为执行所述计算机程序以实现：
向所述智能电视发送音频数据传输指令；
其中，所述音频数据传输指令用于指示所述智能电视执行以下项中的至少一者：将播放的节目的音频数据传输至所述控制装置，关闭所述智能电视的设备音响，和将所述智能电视的音量输出设置为0。
在一些实施例中，所述处理器还配置为执行所述计算机指令以实现：
接收启动所述智能电视的遥控应用的指令；
响应于启动所述智能电视的遥控应用的指令，呈现能对所述智能电视进行操控的第一用户界面；
其中该第一用户界面包括主控制区和功能选择区，其中，所述主控制区包括音量控制区者节目切换区；所述功能选择区包括以下项中的至少一者：传音选项卡、用于返回遥控器主页的主页选项卡和用于调用数字键盘的键盘选项卡。
在一些实施例中，所述处理器还被配置为执行所述计算机指令以实现：
接收对所述传音选项卡的选择；
响应于对所述传音选项卡的选择，呈现第二用户界面；
其中，所述第二用户界面上显示播放声音的设备为所述智能电视的提示信息，和用于将播放声音的设备从所述智能电视切换至所述控制装置的传音切换选项栏。
在一些实施例中，所述处理器还被配置为执行所述计算机指令以实现：
接收选中所述传音切换选项栏的指令；
响应于选中所述传音切换选项栏的指令，将播放声音的设备从所述智能电视切换为所述控制装置并呈现第三用户界面；
其中，所述第三用户界面上显示有播放声音的设备为所述控制装置的提示信息。
在一些实施例中，所述处理器还被配置为执行所述计算机指令以实现：
在播放声音的设备从所述智能电视切换为所述控制装置后，将接收到的所述音频数据传输至所述控制装置的音频播放器。
在一些实施例中，所述处理器还被配置为执行所述计算机指令以实现：
接收关闭所述传音切换选项栏的指令；
响应于关闭所述传音切换选项栏的指令，向所述智能电视发送停止录音的指令。
在一些实施例中，所述处理器还被配置为执行所述计算机指令以实现：
在所述接收启动所述智能电视的遥控应用的指令之前，建立所述控制装置与所述智能电视之间的通信连接。
本申请实施例还涉及一种智能电视，包括：
存储器，被配置为存储计算机程序；
处理器，被配置为执行所述计算机程序以控制所述智能电视：
向控制装置发送第一系统时间数据包，其中，所述第一系统时间数据包中包括所述智能电视在向所述控制装置发送所述第一系统时间数据包时所述智能电视的第一系统时间；
接收所述控制装置发送的音频数据传输指令；
发送音频数据至所述控制装置，其中，所述音频数据中携带所述智能电视发送所述音频数据时的第二系统时间，使得所述控制装置计算所述智能电视与所述控制装置之间传输所述音频数据的网络延时，并根据所述网络延时判断是否播放所述音频数据。
本申请实施例提供的一种机器可读的非易失性存储介质，其上存储有计算机指令，所述计算机指令被执行时实现：
接收智能电视发送的第一系统时间数据包，其中，所述第一系统时间数据包中包括所述智能电视在向所述控制装置发送所述第一系统时间数据包时所述智能电视的第一系统时间；
计算所述控制装置接收到所述第一系统时间数据包时所述控制装置的第二系统时间与所述第一系统时间的差值作为第一差值；
接收所述智能电视发送的音频数据；
根据所述第一差值，计算所述智能电视与所述控制装置之间传输所述音频数据的网络延时；以及
当所述网络延时大于预设阈值时，丢弃所述音频数据。
本申请实施例提供的一种音频播放装置，包括：
第一单元，接收智能电视发送的第一系统时间数据包，其中，所述第一系统时间数据包中包括所述智能电视在向所述音频播放装置发送所述第一系统时间数据包时所述智能电视的第一系统时间；
第二单元，用于计算所述音频播放装置接收到所述第一系统时间数据包时所述音频播放装置的第二系统时间与所述第一系统时间的差值作为第一差值；
第三单元，用于向智能电视发送音频数据传输指令；
第四单元，用于接收智能电视发送的音频数据；
第五单元，用于根据所述第一差值，计算智能电视与音频播放装置之间传输所述音频数据的网络延时；
第六单元，用于当所述网络延时大于预设阈值时，丢弃所述音频数据。本申请实施例提供的一种智能电视，包括：
第一发送单元，用于向控制装置发送第一系统时间数据包，其中，所述第一系统时间数据包中包括所述智能电视在向所述控制装置发送所述第一系统时间数据包时所述智能 电视的第一系统时间；
接收单元，用于接收控制装置发送的音频数据传输指令；
第二发送单元，用于发送音频数据至控制装置，其中，所述音频数据中携带所述智能电视发送所述音频数据时的第二系统时间，使得所述控制装置计算所述智能电视与所述控制装置之间传输所述音频数据的网络延时，并根据所述网络延时判断是否播放所述音频数据。
本申请实施例还涉及一种音频发送方法，包括：
智能电视采集当前播放的音频数据；
确定所述音频数据是否为无效音频数据；
响应于所述音频数据是无效音频数据，则所述智能电视在所述音频数据中添加无效音频数据指示信息，并对所述音频数据进行压缩处理，生成无效音频数据包；
所述智能电视发送所述无效音频数据包至控制装置。
在一些实施例中，所述方法还包括：响应于所述音频数据不是无效音频数据，则将所述音频数据发送至所述控制装置。
在一些实施例中，所述确定所述音频数据是否为无效音频数据包括：
确定所述音频数据是否全部为无效字节。
在一些实施例中，所述采集到的音频数据为完整的一帧音频数据，并且该帧音频数据的字符为预设字符，则确定所述音频数据为无效音频数据。
在一些实施例中，所述方法还包括：
在所述采集当前播放内容的音频数据之前，控制所述智能电视进入录音状态，对所述智能电视进行音频录制。
本申请实施例提供的一种音频播放方法，包括：
接收智能电视发送的音频数据；
对所述音频数据进行解析得到解析数据；
根据所述解析数据确定所述音频数据是否为无效音频数据；
响应于所述音频数据为无效音频数据，则丢弃所述无效音频数据。
在一些实施例中，响应于所述音频数据不是无效音频数据，播放所述音频数据。
在一些实施例中，所述根据所述解析数据确定所述音频数据是否为无效音频数据包括：
根据所述解析数据中的指示信息，确定所述音频数据是否为无效音频数据；
在一些实施例中，所述指示信息位于所述音频数据的前N位字节，N为大于1的整数。
在一些实施例中，所述播放所述音频数据，包括：将所述音频数据存入播放缓存中。
本申请实施例还涉及一种智能电视，包括：
存储器，被配置为存储计算机程序；
处理器，配置为执行所述计算机指令以使得所述智能电视执行：
采集当前播放的音频数据；
确定所述音频数据是否为无效音频数据；
响应于所述音频数据是无效音频数据，则在所述音频数据中添加无效音频数据指示信息，并对所述音频数据进行压缩处理，生成无效音频数据包；
发送所述无效音频数据包至控制装置。
在一些实施例中，所述处理器还配置为执行所述计算机指令以使得所述智能电视执行：
响应于所述音频数据不是无效音频数据，则将所述音频数据发送至所述控制装置。
在一些实施例中，所述确定所述音频数据是否为无效音频数据包括：
确定所述音频数据是否全部为无效字节。
在一些实施例中，所述采集到的音频数据为完整的一帧音频数据，并且该帧音频数据的字符为预设字符，则确定所述音频数据为无效音频数据。
在一些实施例中，所述处理器还配置为执行所述计算机指令以使得所述智能电视执行：
控制所述智能电视进入录音状态，对所述智能电视进行音频录制。
本申请实施例还涉及一种控制装置，包括：
存储器，配置为存储计算机指令；
处理器，配置为执行所述计算机指令以使得所述控制装置执行：
接收智能电视发送的音频数据；
对所述音频数据进行解析得到解析数据；
根据所述音频数据确定所述音频是否为无效音频数据；
响应于所述音频数据为无效音频数据，则丢弃所述无效音频数据。
本申请实施例还涉及一种智能电视，包括：
接收单元，用于接收控制装置发送的音频数据传输指令；
采集单元，用于采集当前播放的音频数据；
确定单元，用于确定所述音频数据是否为无效音频数据；
处理单元，用于响应于所述音频数据是无效音频数据，则在所述音频数据中添加无效音频数据指示信息，并对所述音频数据进行压缩处理，生成无效音频数据包；
发送单元，用于发送所述无效音频数据包至所述控制装置。
本申请实施例提供的一种控制装置，该装置包括：
接收单元，用于接收智能电视发送的音频数据；
解析单元，用于对所述音频数据进行解析得到解析数据；
确定单元，用于根据所述解析数据确定所述音频数据是否为无效音频数据；
处理单元，用于响应于所述音频数据为无效音频数据，则丢弃所述无效音频数据；
播放单元，用于响应于所述音频数据为有效音频数据，播放所述音频数据。
本申请实施例还提供的一种计算设备，包括：
存储器，用于存储计算机可读程序；
处理器，用于调用所述计算机可读程序以实现上述方法中的至少一种。
本申请实施例提供了一种计算机可读的非易失性存储介质，所述计算机存储介质存储有计算机可读程序，所述计算机可读程序被所述计算机以实现上述方法中的至少一种。
附图说明
为了更清楚地说明本申请实施例中的技术方案，下面将对实施例描述中所需要使用的附图作简要介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域的普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1为根据本申请实施例的智能电视与控制装置交互实例的示意图；
图2为根据本申请实施例的移动终端与智能电视连接状态的示意图；
图3为根据本申请实施例的移动终端的第一用户界面示意图；
图4为根据本申请实施例的移动终端的第二用户界面示意图；
图5为根据本申请实施例的移动终端的第三用户界面示意图；
图6为根据本申请实施例的音频播放方法的第一示意图；
图7为根据本申请实施例的音频发送方法的第一示意图；
图8为根据本申请实施例的延时优化总体流程的第一示意图；
图9为根据本申请实施例的音频播放装置示意图；
图10为根据本申请实施例的智能电视的第一示意图；
图11为根据本申请实施例的控制装置的第一示意图；
图12为根据本申请实施例的智能电视的第二示意图；
图13为根据本申请实施例的音频发送方法的第二示意图；
图14为根据本申请实施例的音频播放方法的第二示意图；
图15为根据本申请实施例的延时优化总体流程的第二示意图；
图16为根据本申请实施例的智能电视的第三示意图；
图17为根据本申请实施例的控制装置的第二示意图；
图18为根据本申请实施例的智能电视的第四示意图；
图19为根据本申请实施例的控制装置的第三示意图。
具体实施方式
把移动设备作为智能电视的一个声音输出设备，智能电视实时抓取当前的声音，并传送到移动设备进行实时播放。但是电视与手机连接后，可能由于网络原因及手机播放缓存不能及时清理而造成声音延时。
下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本申请一部分实施例，并不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
本申请实施例提供了一种音频播放、发送的方法和装置，通过在控制装置丢弃延时时间过长的音频数据，以减小播放缓存，从而减少播放延迟，避免了由于网络延时较大而控制装置无法与智能电视实时同步音频数据的现象。
下面结合说明书附图对本申请各个实施例进行详细描述。需要说明的是，本申请实施例的展示顺序仅代表实施例的先后顺序，并不代表实施例所提供的技术方案的优劣。
在以下描述中，阐述了大量特定细节以提供对本发明实施例的更透彻说明。然而，对于本领域技术人员显而易见的是，在实现本发明实施例而可以不用这些特定细节。
本申请中使用的术语“直播电视”是指实时或与事件发生时间基本同步的电视制作广播。
本申请中使用的术语“视频点播”(Video On Demand，VOD)是指允许用户选择和观看/收听点播视频或音频内容的系统和过程。VOD系统可以将内容分流，从而观看实时内容或将其下载到储存介质以稍后再观看。
本申请中使用的术语“确定”、“计算”和“计算机计算”及其变通说法，可互换使用，并包括任何类型的方法论、过程、数学运算或技巧。
本申请实施例提供了一种智能电视与控制装置(例如移动终端)交互的应用场景。
参考图1所示的根据一些实施例的智能电视100与控制装置200。智能电视100可被配置为具有娱乐、商业应用、社交互动、内容创建和/或消费的功能，一个或多个控制装置用于组织和控制与智能电视100的通信。因此可以理解为，智能电视和/或控制装置可以用于增强用户互动体验，无论是在家中还是工作场所。
在一些实施例中，智能电视100可以被配置为接收和处理各种用户和/或设备输入。可以使用移动终端，如手机、平板电脑、计算机、笔记本电脑、上网本和其他智能设备以控制智能电视100。例如，使用在智能设备上运行的应用程序控制智能电视100。
在一些实施例中，智能电视100可被配置为通过各种输入设备接收输入，包括但绝不限于视频、音频、无线电、光、触觉及其组合。例如，用户可以以会话方式和智能电视100讲话。智能电视100就像是智能设备的智能私人助手和音控导航应用程序(如苹果的Siri、安卓的Skyvi、Robin、Iris和其他应用程序)一样接收和处理语音命令。
本申请实施例提供的一种移动终端的用于与用户交互的用户界面的操作步骤。
如图2所示，本申请实施例提供了移动终端与智能电视之间的连接状态的示意图。当智能电视与移动终端进行交互时，移动终端可以根据各种连接协议与智能电视建立通信连接，并且，该移动终端可以对所述智能电视进行遥控操作，或者安装有能对所述智能电视进行操控的应用程序。
如图3所示，当移动终端接收到启动智能电视的遥控应用的指令时，移动终端可以启动遥控应用，呈现能对所述智能电视进行操控的第一用户界面，该第一用户界面包括主控制区和功能选择区，其中，所述主控制区包括音量控制区和节目切换区；所述功能选择区包括以下项中的至少一者：传音选项卡、用于返回遥控器主页的主页选项卡和用于调用数字键盘的键盘选项卡。移动终端可以通过在第一用户界面上操作，实现与智能电视的交互。示例性的，当用户在音量控制区输入指令，可控制智能电视的音量输出。
如图4所示，移动终端接收并响应于对所述传音选项卡的选择，呈现第二用户界面，其中，所述第二用户界面上显示当前播放声音的设备为所述智能电视的提示信息，以及用于将声音播放端切换至所述移动终端的传音切换选项栏。
如图5所示，移动终端接收并响应于选中所述传音切换选项栏的指令，将播放声音的设备切换为所述移动设备，并呈现第三用户界面，其中，所述第三用户界面上显示有当前播放声音的设备为所述移动终端的提示信息。在传音切换的过程中，智能电视将其音频播放器的音量调节为静音或者0，使得所述音频数据仍在音频播放器中播放，但是不从喇叭出输出，将所述音频数据进行采集录制，下发至移动终端。
当智能电视接收到移动终端的音频数据传输指令后，将采集到的音频数据编码成脉冲编码调制(Pulse Code Modulation，PCM)数据流，通过网络传输到移动终端，移动终端对电视声音进行实时播放，由于音频数据流是原始的PCM数据，因此，移动终端每接收到一帧数据，不需要做任何解码处理，直接将音频数据送到硬件解码器中进行解码播放。
在基于Linux或者Android系统开发的智能电视系统上，利用智能电视整机主处理芯片(CPU)提供的音频抓取编码功能，将智能电视当前屏幕的声音按照一定的采样格式(例如8KHz采样率、16Bit采样精度，双声道)进行抓取，提供PCM格式的音频数据，无压缩的通过有线网络或者无线网络传送给移动终端，移动终端接收到音频数据后，直接将音频数据送到硬件解码器中进行解码播放。
参见图6，本申请实施例提供的一种音频播放方法，应用于控制装置，该方法包括：
S101、接收智能电视发送的第一系统时间数据包。其中，所述第一系统时间数据包中包括所述智能电视在向所述控制装置发送所述第一系统时间数据包时所述智能电视的第一系统时间；例如，控制装置为移动终端。
S102、计算所述控制装置接收到所述第一系统时间数据包时所述控制装置的第二系统 时间与所述第一系统时间的差值作为第一差值；
S103、接收所述智能电视发送的音频数据；
S104、根据所述第一差值，计算所述智能电视与所述控制装置之间传输所述音频数据的网络延时；
S105、当所述网络延时大于预设阈值时，丢弃所述音频数据。
通过该方法，可以减少控制装置的播放延迟。
例如，预设阈值可以为200毫秒，当网络延时大于200毫秒时，则丢弃音频数据，不送入移动终端的音频播放器进行播放(将延时较大的音频数据播放，没有意义)；当网络延时小于或等于预设阈值时，将音频数据送入移动终端的音频播放器进行播放。该移动终端可以是手机，手机在播放音频数据之前，会将上层接收到的音频数据放在一个缓存队列中，手机的音频播放器播放时从缓存队列中取音频数据，播放完毕后，已播放的音频数据会从缓存队列中被删除，如果上层应用向缓存队列中写数据过快，而播放器从缓存队列中取数据相对较慢，则这个队列中的缓存数据会越来越多。
例如，控制装置为手机，第一系统时间为智能电视的系统时间Time1_tv，第二系统时间为手机的系统时间Time1_phone，第一差值Time1_diff的计算公式为Time1_diff＝Time1_phone-Time1_tv；第一系统时间与第二系统时间可能是不相同的，一种原因是电视上的时间没有与网络同步过(例如路由器没有连接外网)，而手机上的时间与网络同步过；另一种原因是电视与手机都与网络同步过时间，但同步后的电视与手机的时间不可能完全一样，会存在几毫秒到几百毫秒，甚至1秒左右的误差。
在一些实施例中，步骤S104还包括：
解析所述音频数据中携带的第三系统时间，其中，所述第三系统时间是所述智能电视发送所述音频数据时的所述智能电视的系统时间；
获取第四系统时间，其中，所述第四系统时间是所述控制装置接收到所述音频数据的系统时间；
计算所述第四系统时间与所述第三系统时间的差值作为第二差值；以及
确定所述智能电视与所述控制装置之间传输所述音频数据的网络延时为：所述第二差值与所述第一差值的差值。
例如，第三系统时间为电视的系统时间Time2_tv，第四系统时间为手机的系统时间Time2_phone。
第二差值Time2_diff的计算公式为：
Time2_diff＝Time2_phone-Time2_tv。
网络延时的计算公式为：
Time_delay＝Time2_diff–Time1_diff。
其中，(Time2_phone-Time2_tv)表示从智能电视传输音频数据到手机时的网络延时加上智能电视与手机的系统时间差；(Time1_phone-Time1_tv)表示智能电视与手机的系统时间差(由于从智能电视到手机传输的数据量只是一个时间，没有音频数据，因此传输过程中的网络延时很小，可以忽略)；(Time2_diff–Time1_diff)表示从智能电视传输音频数据到手机时的网络延时。
在一些实施例中，所述方法还包括：
所述控制装置向所述智能电视发送音频数据传输指令；
其中，所述音频数据传输指令用于指示所述智能电视执行以下项中的至少一者：将播放的节目的音频数据传输至所述控制装置，关闭所述智能电视设备音响，和将所述智能电视的音量输出设置为0。
在一些实施例中，所述方法还包括：
所述控制装置接收启动所述智能电视的遥控应用的指令；
所述控制装置响应于启动所述智能电视的遥控应用的指令，呈现能对所述智能电视进行操控的第一用户界面；
其中该第一用户界面包括主控制区和功能选择区，其中，所述主控制区包括音量控制区和节目切换区；所述功能选择区包括以下项中的至少一者：传音选项卡、用于返回遥控器主页的主页选项卡和用于调用数字键盘的键盘选项卡。
在一些实施例中，所述方法还包括：
所述控制装置接收对所述传音选项卡的选择；
所述控制装置响应于对所述传音选项卡的选择，呈现第二用户界面；
其中，所述第二用户界面上显示播放声音的设备为所述智能电视的提示信息，和用于将播放声音的设备从所述智能电视切换至所述控制装置的传音切换选项栏。
在一些实施例中，所述方法还包括：
所述控制装置接收选中所述传音切换选项栏的指令；
所述控制装置响应于选中所述传音切换选项栏的指令，将播放声音的设备从所述智能电视切换为所述控制装置并呈现第三用户界面；
其中，所述第三用户界面上显示有播放声音的设备为所述控制装置的提示信息。
在一些实施例中，所述方法还包括：
在播放声音的设备从所述智能电视切换为所述控制装置后，所述控制装置将接收到的所述音频数据传输至所述控制装置的音频播放器。
在一些实施例中，所述方法还包括：
所述控制装置接收关闭所述传音切换选项栏的指令；
所述控制装置响应于关闭所述传音切换选项栏的指令，向所述智能电视发送停止录音 的指令。
在一些实施例中，在所述接收启动所述智能电视的遥控应用的指令之前，所述方法还包括：
所述控制装置建议与所述智能电视之间的通信连接。
参见图7，根据本申请实施例，提供了一种音频发送方法，应用于智能电视，包括：
S201、向控制装置发送第一系统时间数据包，其中，所述第一系统时间数据包中包括所述智能电视在向所述控制装置发送所述第一系统时间数据包时所述智能电视的第一系统时间；
S202、接收控制装置发送的音频数据传输指令；
S203、发送音频数据至所述控制装置，其中，所述音频数据中携带所述智能电视发送所述音频数据时的第二系统时间，使得所述控制装置计算所述智能电视与所述控制装置之间传输所述音频数据的网络延时，并根据所述网络延时判断是否播放所述音频数据。
通过该方法，减少控制装置的播放延迟。
例如，智能电视将PCM格式音频数据发送给手机之前，会在PCM格式音频数据的前面加上(智能电视)当前的系统时间信息。
参见图8，为本申请实施例提供的一种延时优化总体流程图，智能电视和手机总的具体处理流程包括：
智能电视和手机连接后，获取智能电视的当前系统时间Time1_tv，并把在当前系统时间将该系统时间Time1_tv发送到手机；
手机接收到该系统时间Time1_tv时，获取手机的当前系统时间Time1_phone；
计算手机和智能电视的系统时间差值Time_systemDiff(对于不同的设备，例如手机、电视，在同一时间获取到的系统时间可能不一样)，公式如下：
Time_systemDiff＝Time1_phone-Time1_tv
智能电视每次抓取到PCM格式音频数据的同时获取当前的系统时间Time2_tv，然后对PCM格式音频数据和系统时间Time2_tv进行打包处理，打包格式如表1所示：
表1 音频数据包
时间(长度8Byte) 音频数据(长度640Byte)
音频数据的计算方法为：
假设音频采样率为8000Hz，采样通道为2，位深度为16，采样间隔为20ms，则每帧音频数据大小的计算公式(1)为：
Figure PCTCN2019107265-appb-000001
手机接收到音频数据包后，解析出时间Time2_tv，并获取接收到音频数据包时本系统当前的时间Time2_phone，并计算手机和智能电视的系统时间差值Time_netDiff＝Time2_phone-Time2_tv；
计算真实的网络延时Time_delay，公式为：
Time_delay＝Time_netDiff-Time_systemDiff。
当网络延时Time_delay大于预设阈值K时，例如K为200ms，手机把音频数据丢弃，不送入音频播放器(播放延时较大的音频数据，没有意义)；当网络延时Time_delay小于或等于预设阈值K时，手机把音频数据送入音频播放器进行播放。
以下为本申请提供的一个具体实施例，以2018年12月1日这一天为例。
假设开始传输系统时间数据包时，智能电视的系统时间Time1_tv为11点11分11秒100毫秒，手机的系统时间Time1_phone为11点11分12秒110毫秒，根据计算公式Time_systemDiff＝Time1_phone-Time1_tv得到Time_systemDiff为1秒10毫秒，也就是说，手机的系统时间比智能电视的系统时间快了1秒10毫秒。
经过2秒后，智能电视开始向手机发送音频数据，此时智能电视的系统时间Time2_tv为11点11分13秒100毫秒；手机接收到音频数据的系统时间Time2_phone为11点11分14秒210毫秒，根据计算公式Time_netDiff＝Time2_phone-Time2_tv得到Time_netDiff为1秒110毫秒。
根据上述计算公式Time_delay＝Time_netDiff-Time_systemDiff得到Time_delay为100毫秒。
上述控制装置可以是音频播放装置。参见图9，本申请实施例提供的一种音频播放装置，包括：
第一单元11，接收智能电视发送的第一系统时间数据包，其中，所述第一系统时间数据包中包括所述智能电视在向所述音频播放装置发送所述第一系统时间数据包时所述智能电视的第一系统时间；
第二单元12，用于计算所述音频播放装置接收到所述第一系统时间数据包时所述音频播放装置的第二系统时间与所述第一系统时间的差值作为第一差值；
第三单元13，用于向智能电视发送音频数据传输指令；
第四单元14，用于接收智能电视发送的音频数据；
第五单元15，用于根据所述第一差值，计算智能电视与音频播放装置之间传输所述音频数据的网络延时；
第六单元16，用于当所述网络延时大于预设阈值时，丢弃所述音频数据。
参见图10，本申请实施例提供的智能电视，包括：
第一发送单元21，用于向控制装置发送第一系统时间数据包，其中，所述第一系统时 间数据包中包括所述智能电视在向所述控制装置发送所述第一系统时间数据包时所述智能电视的第一系统时间；
接收单元22，用于接收控制装置发送的音频数据传输指令；
第二发送单元23，用于发送音频数据至控制装置，其中，所述音频数据中携带所述智能电视发送所述音频数据时的第二系统时间，使得所述控制装置计算所述智能电视与所述控制装置之间传输所述音频数据的网络延时，并根据所述网络延时判断是否播放所述音频数据。
参见图11，本申请实施例还提供了一种控制装置，包括：
存储器310，被配置为存储计算机程序；
处理器300，用于为执行所述计算机程序以控制所述控制装置执行以下步骤：
接收智能电视发送的第一系统时间数据包，其中，所述第一系统时间数据包中包括所述智能电视在向所述控制装置发送所述第一系统时间数据包时所述智能电视的第一系统时间；
计算所述控制装置接收到所述第一系统时间数据包时所述控制装置的第二系统时间与所述第一系统时间的差值作为第一差值；
接收所述智能电视发送的音频数据；
根据所述第一差值，计算所述智能电视与所述控制装置之间传输所述音频数据的网络延时；以及
当所述网络延时大于预设阈值时，丢弃所述音频数据。
当所述网络延时大于预设阈值时，控制丢弃所述音频数据，从而减少控制装置的播放延迟。
该控制装置还包括用户接口320和总线接口330。所述处理器300和存储器310通过所述总线接口330交互连接，所述用户接口320和所述总线接口330连接。
在一些实施例中，所述处理器300，还被配置为执行所述计算机程序以实现上述音频播放方法。
参见图12，本申请实施例还提供的一种智能电视，包括：
存储器410，被配置为存储计算机程序；
处理器400，被配置为执行所述计算机程序以控制所述电子设备执行以下步骤：
向控制装置发送第一系统时间数据包，其中，所述第一系统时间数据包中包括所述智能电视在向所述控制装置发送所述第一系统时间数据包时所述智能电视的第一系统时间；
接收所述控制装置发送的音频数据传输指令；
发送音频数据至所述控制装置，其中，所述音频数据中携带所述智能电视发送所述音频数据时的第二系统时间，使得所述控制装置计算所述智能电视与所述控制装置之间传输 所述音频数据的网络延时，并根据所述网络延时判断是否播放所述音频数据。
所述控制装置计算所述智能电视与所述控制装置之间传输所述音频数据的网络延时，并根据所述网络延时判断是否播放所述音频数据，从而减少控制装置的播放延迟。
所述智能电视还包括用户接口420和总线接口430。所述处理器400和存储器410通过所述总线接口430交互连接，所述用户接口420和所述总线接口430连接。
在实时音频收听过程中，智能电视不断地录制声音并通过网络发送到移动终端。在发送录制的声音之前，智能电视对录制的声音进行实时“有效性音频数据帧”分析，对“有效的音频数据”进行实时发送；如果智能电视没有播放声音或视频文件时，智能电视录制一帧的声音都是连续的0x00(0x00表示一个16进制的数，0x是16进制的前缀)，对于这种全是0x00的无效数据帧(空数据)，在传输过程中也占用了一帧的网络数据量；移动终端接收到之后，向播放器的缓存中加入，也增加了手机的播放缓存，如果对这部分无效数据帧做特殊处理(对于全是0x00的数据，播放器播放出来也没有声音)再发送给移动终端，可以大大减轻网络传输负担，同时减小手机系统的播放缓存，从而有效地优化音频的实时播放延迟。
参见图13，本申请的实施例还提供了一种音频发送方法，应用于智能电视，该方法包括：
S301、采集当前播放的音频数据；
在实时音频收听过程中，智能电视会不断录制系统正在播放的声音，即不断采集当前播放节目的音频数据。智能电视在录制过程中，并不会检测此刻是否有音频文件的输出，而是一直处于录制阶段。
S302、确定所述音频数据是否为无效音频数据；
S303、响应于所述音频数据是无效音频数据，则在所述音频数据中添加无效音频数据指示信息，并对所述音频数据进行压缩处理，生成无效音频数据包；
S304、发送所述无效音频数据包至控制装置。或者，若智能电视采集到的音频数据为无效音频数据时，智能电视不将这段无效音频数据发送给控制装置。
通过该方法，可以减轻网络传输负担，优化音频的实时播放延迟。
示例性的，假设用户选择在足球节目播放过程中，通过控制装置向音频数据传输指令，智能电视开始录音。在录音过程中，用户由于误操作或者要对观看节目进行重新选择或浏览，而退出足球节目的播放，此时，音频播放器中其实并没有实际的音频数据的播放，但是，由于录音并不会因此而终止，因此，在没有足球节目播放的这段期间，智能电视所采集到的音频数据其实为空，这样的音频数据为无效音频数据。
在一些实施例中，若所述音频数据为有效音频数据，则将所述有效音频数据发送至所述控制装置。
在一些实施例中，所述采集到的音频数据为完整的一帧音频数据，并且该帧音频数据的字符为预设字符，则确定所述音频数据为无效音频数据。
示例的，预设字符为0x00。现将一帧全部为0x00音频数据称为无效音频数据。
具体的，判断一帧音频数据是否为无效数据，可以对一帧数据的左右字节做一个循环式的检测，若所述音频数据中，有一个字节不是0x00，则确定所述音频数据为有效音频数据；若所述音频数据中，每一个字节都为0x00时，则确定所述音频数据为无效音频数据。
在一些实施例中，步骤S302还包括：
判断所述音频数据是否全部为无效字节；
若所述音频数据全部为无效字节，则在所述无效字节中添加无效音频数据指示信息，并对所述无效音频数据进行压缩处理，生成无效音频数据包。
如前所述，在基于Linux或者Android系统开发的智能电视系统上，利用智能电视整机主处理芯片(CPU)提供的音频抓取编码功能，将智能电视当前屏幕的声音按照一定的采样格式(例如8KHz采样率、16Bit采样精度，双声道)进行抓取，提供PCM格式的音频数据。当智能电视采集在音频数据后，对该音频数据进行打包处理，并发送给手机，该数据包包括采集到的音频数据和当前系统时间打包格式如上述表1所示。
每帧音频数据大小的计算方法为：
假设音频采样率为8000Hz，采样通道为2，位深度为16，采样间隔为20ms，则每帧音频数据大小的计算公式(2)为：
Figure PCTCN2019107265-appb-000002
智能电视在采集音频数据时，正常情况下一帧数据占用640个字节，若所述音频数据为有效音频数据，则将所述640个字节的有效音频数据发送至所述控制装置。若所述音频数据为无效数据时，对于这种全是0x00的无效数据帧，在传输过程中也占用了一帧的网络数据量，移动终端接收到这帧全是0x00的无效数据帧之后，向播放器的缓存中写入，也会增加手机的播放缓存。
本申请实施例中，将这些无效的音频数据进行压缩处理，不全部发送640个字节，而是将这段全是0x00的音频数据压缩为M个字节，也就是说，将这段无效音频数据重新赋值为M个字符的数据流，压缩后的音频数据具体表示为0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00。其中，M为小于640的正整数。在一些实施例中，1≤M≤100。
本申请实施例中，无效音频数据的发送还用于保持智能电视与移动终端的通信连接。若规定智能电视每个20ms需向移动终端发送一个数据包以保证通信连接，当智能电视在20ms未发送音频数据包，将会断开连接，这样导致下一时刻还有有效音频数据发送时， 移动终端无法实时接收音频数据。因此，虽无实际音频数据，但是智能电视还是会每隔20ms发送一个数据包到移动终端。
在对无效数据进行压缩处理以后，为使移动终端在接收到无效数据以后进行识别以便无需传输至其播放器，所以，在该压缩后的数据之前，添加一个指示信息，用于指示该数据包中的数据为无效数据。示例性的，在所述M个字节前添加前N位字节，N为大于1的整数。指示信息位于音频数据的前5位字节，指示信息具体可以表示为0x01 0x02 0x03 0x04 0x05，可以预先与控制装置端约定好采用哪些字符用于指示音频数据是否有效。
生成的无效数据包中，包括无效音频数据指示信息和被压缩的M位无效音频数据。
例如，无效音频数据指示信息为0x01 0x02 0x03 0x04 0x05，将这段全是0x00的音频数据压缩为20个字节，也就是说，将这段无效音频数据重新赋值为20个字节的数据流，压缩后的音频数据具体表示为0x01 0x02 0x03 0x04 0x05 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00。
在一些实施例中，在步骤S301之前，还包括：
控制所述智能电视进入录音状态，对所述智能电视进行音频录制。
参见图14，本申请实施例还提供的一种音频播放方法，包括：
S401、接收智能电视发送的音频数据；
S402、对所述音频数据进行解析得到解析数据；
S403、根据所述解析数据确定所述音频数据是否为无效音频数据；
S404、响应于所述音频数据为无效音频数据，则丢弃所述无效音频数据。
通过该方法，可以减小控制装置的播放缓存，优化音频的实时播放延迟。
当移动终端接收的音频数据为无效音频数据时，丢弃这段无效音频数据，不向手机播放器的缓存中写入。当所述音频数据为有效音频数据时，播放所述音频数据。
在一些实施例中，在接收智能电视发送的音频数据后，根据所述音频数据中的指示信息，判断所述音频数据是否为无效音频数据。
在一些实施例中，所述指示信息位于所述音频数据的前N位字节，N为大于1的整数。
例如，指示信息位于音频数据的前5位字节，指示信息具体可以表示为0x01 0x02 0x03 0x04 0x05，可以预先与智能电视约定好采用哪些字符用于指示音频数据是否有效。
在一些实施例中，当确定所述音频数据为有效音频数据时，将所述有效音频数据存入播放缓存中。
移动终端例如手机在播放音频数据之前，会将上层接收到的音频数据放在一个缓存队列中，手机的音频播放器播放时从缓存队列中取音频数据，播放完毕后，已播放的音频数据会从缓存队列中被删除，如果上层应用向缓存队列中写数据过快，而播放器从缓存队列中取数据相对较慢，则这个队列中的缓存数据会越来越多。
参见图15，为本申请实施例还提供的一种延时优化总体流程图，智能电视和移动终端总的具体处理流程包括：
移动终端例如是手机。智能电视接收到手机连接到智能电视的连接消息后，抓取整机(智能电视)声音，并判断抓取的声音是否为有效性音频，具体地判断方法如下：
当智能电视采集到一帧音频数据后，判断这一帧数据是否全部为0x00，也就是说，判断智能电视一次采集的640个字节数据是否全是0x00，如果全是0x00，则定义这段数据为无效音频数据；如果这一帧数据不全是0x00，是由一些例如0x12等数据组成，则表示这段数据是有效音频数据。
当智能电视采集的数据为有效音频数据时，将该有效音频数据逐帧发送给移动终端；当智能电视采集的数据为无效音频数据时，先对这段无效音频数据做优化处理，即将640个字节数据全是0x00的无效音频数据重新赋值为20个字符的数据流(或者假数据)，例如这20个字符的数据流中，前五位可以定义为0x01 0x02 0x03 0x04 0x05(即用于指示无效音频数据的指示信息，该指示信息不限于0x01 0x02 0x03 0x04 0x05，也可以定义前五位为0x01 0x01 0x02 0x02 0x02等)，后面的15位全部为0x00，具体可以表示为0x01 0x02 0x03 0x04 0x05 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00，这样这段640个字节的无效音频数据被压缩成20个字节的数据；智能电视再将这20个字节的数据发送给移动终端。
当移动终端接收到智能电视发送的音频数据包流时，先判断所接收的音频数据是否为有效音频数据，如果移动终端接收到的是上述具有20个字节的数据流：0x01 0x02 0x03 0x04 0x05 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00，则通过判断数据的前5位0x01 0x02 0x03 0x04 0x05，确定所接收到的数据是无效音频数据(这段无效音频数据在智能电视没有声音)，不会向手机播放器的缓存中写入数据，丢弃这段无效音频数据(这样可以起到清空手机系统音频缓存数据的作用)；如果移动终端接收到的不是上述具有20个字节的数据，而是具有640个字节的数据流，则确定所接收到的数据是有效音频数据，将这段有效音频数据全部写入手机播放器的缓存中。
相应地，在装置侧，参见图16，本申请实施例提供的一种智能电视，包括：
接收单元31，用于接收控制装置发送的音频数据传输指令；
采集单元32，用于采集当前播放的音频数据；
确定单元33，用于确定所述音频数据是否为无效音频数据；
处理单元34，用于响应于所述音频数据是无效音频数据，则在所述音频数据中添加无效音频数据指示信息，并对所述音频数据进行压缩处理，生成无效音频数据包；
发送单元35，用于发送所述无效音频数据包至所述控制装置。
在装置侧，参见图17，本申请实施例提供的一种控制装置，包括：
接收单元41，用于接收智能电视发送的音频数据；
解析单元42，用于对所述音频数据进行解析得到解析数据；
确定单元43，用于根据所述解析数据确定所述音频数据是否为无效音频数据；
处理单元44，用于响应于所述音频数据为无效音频数据，则丢弃所述无效音频数据；
播放单元45，用于响应于所述音频数据为有效音频数据，播放所述音频数据。
参见图18，本申请实施例还提供了一种智能电视，包括：
处理器600，用于读取存储器610中的计算机程序以控制所述智能电视执行下列过程：
采集当前播放的音频数据；
确定所述音频数据是否为无效音频数据；
响应于所述音频数据是无效音频数据，则在所述音频数据中添加无效音频数据指示信息，并对所述音频数据进行压缩处理，生成无效音频数据包；
发送所述无效音频数据包至控制装置。
采用该智能电视，可以减轻网络传输负担，优化音频的实时播放延迟。
该控制装置还包括用户接口620和总线接口630。所述处理器600和存储器610通过所述总线接口630交互连接，所述用户接口620和所述总线接口630连接。
在一些实施例中，处理器600，用于读取所述计算机程序以控制所述智能电视：响应于所述音频数据不是无效音频数据，则将所述音频数据发送至所述控制装置。
在一些实施例中，处理器600，用于读取所述计算机程序以控制所述智能电视：
确定所述音频数据是否全部为无效字节。
在一些实施例中，所述采集到的音频数据为完整的一帧音频数据，并且该帧音频数据的字符为预设字符，则确定所述音频数据为无效音频数据。
在一些实施例中，处理器600，用于读取所述计算机程序以控制所述智能电视：在所述采集当前播放内容的音频数据之前，控制所述智能电视进入录音状态，对所述智能电视进行音频录制。
参见图19，本申请实施例还提供的一种控制装置，包括：
处理器800，用于读取存储器810中的程序，执行下列过程：
接收智能电视发送的音频数据；
对所述音频数据进行解析得到解析数据；
根据所述解析数据确定所述音频数据是否为无效音频数据；
响应于所述音频数据为无效音频数据，则丢弃所述无效音频数据。
通过该控制装置，可以减小控制装置的播放缓存，优化音频的实时播放延迟。
所述智能电视还包括用户接口820和总线接口830。所述处理器800和存储器810通过所述总线接口830交互连接，所述用户接口820和所述总线接口830连接。
在一些实施例中，响应于所述音频数据不是无效音频数据，播放所述音频数据。
在一些实施例中，所述根据所述解析数据确定所述音频数据是否为无效音频数据包括：
根据所述解析数据中的指示信息，确定所述音频数据是否为无效音频数据；
在一些实施例中，所述指示信息位于所述音频数据的前N位字节，N为大于1的整数。
在一些实施例中，所述播放所述音频数据，包括：将所述音频数据存入播放缓存中。
其中，在图11、图12、图18或图19中，总线架构可以包括任意数量的互联的总线和桥，具体由处理器300、400、600或800代表的一个或多个处理器和存储器310、410、610或810代表的存储器的各种电路链接在一起。总线架构还可以将诸如外围设备、稳压器和功率管理电路等之类的各种其他电路链接在一起，这些都是本领域所公知的，因此，本文不再对其进行进一步描述。总线接口提供接口。
本申请实施例提供了一种显示终端，该显示终端具体可以为桌面计算机、便携式计算机、智能手机、平板电脑、个人数字助理(Personal Digital Assistant，PDA)等。该显示终端可以包括中央处理器(Center Processing Unit，CPU)、存储器、输入/输出设备等，输入设备可以包括键盘、鼠标、触摸屏等，输出设备可以包括显示设备，如液晶显示器(Liquid Crystal Display，LCD)、阴极射线管(Cathode Ray Tube，CRT)等。
针对不同的显示终端，在一些实施例中，用户接口320、420、620或820可以是能够外接内接需要设备的接口，连接的设备包括但不限于小键盘、显示器、扬声器、麦克风、操纵杆等。
处理器负责管理总线架构和通常的处理，存储器可以存储处理器在执行操作时所使用的数据。
在一些实施例中，处理器可以是CPU(中央处埋器)、ASIC(Application Specific Integrated Circuit，专用集成电路)、FPGA(Field－Programmable Gate Array，现场可编程门阵列)或CPLD(Complex Programmable Logic Device，复杂可编程逻辑器件)。
存储器可以包括只读存储器(ROM)和随机存取存储器(RAM)，并向处理器提供存储器中存储的程序指令和数据。在本申请实施例中，存储器可以用于存储本申请实施例提供的任一所述方法的程序。
处理器通过调用存储器存储的程序指令，处理器用于按照获得的程序指令执行本申请实施例提供的任一所述方法。
本申请实施例提供了一种计算机存储介质，用于储存为上述本申请实施例提供的装置所用的计算机程序指令，其包含用于执行上述本申请实施例提供的任一方法的程序。
所述计算机存储介质可以是计算机能够存取的任何可用介质或数据存储设备，包括但不限于磁性存储器(例如软盘、硬盘、磁带、磁光盘(MO)等)、光学存储器(例如CD、DVD、BD、HVD等)、以及半导体存储器(例如ROM、EPROM、EEPROM、非易失性 存储器(NAND FLASH)、固态硬盘(SSD))等。
综上所述，本申请实施例提供的一种音频发送、播放的方法及智能电视，用以减轻网络传输负担，减小控制装置的播放缓存，从而优化音频的实时播放延迟。避免了由于网络延时较大而控制装置无法与智能电视实时同步音频数据的现象。
本申请实施例提供的一种音频播放、发送的方法及装置，用以减轻网络传输负担，减小控制装置的播放缓存，从而优化音频的实时播放延迟。
本领域内的技术人员应明白，本申请的实施例可提供为方法、系统、或计算机程序产品。因此，本申请可采用完全硬件实施例、完全软件实施例、或结合软件和硬件方面的实施例的形式。而且，本申请可采用在一个或多个其中包含有计算机可用程序代码的计算机可用存储介质(包括但不限于磁盘存储器和光学存储器等)上实施的计算机程序产品的形式。
本申请是参照根据本申请实施例的方法、设备(系统)、和计算机程序产品的流程图和/或方框图来描述的。应理解可由计算机程序指令实现流程图和/或方框图中的每一流程和/或方框、以及流程图和/或方框图中的流程和/或方框的结合。可提供这些计算机程序指令到通用计算机、专用计算机、嵌入式处理机或其他可编程数据处理设备的处理器以产生一个机器，使得通过计算机或其他可编程数据处理设备的处理器执行的指令产生用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的装置。
这些计算机程序指令也可存储在能引导计算机或其他可编程数据处理设备以特定方式工作的计算机可读存储器中，使得存储在该计算机可读存储器中的指令产生包括指令装置的制造品，该指令装置实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能。
这些计算机程序指令也可装载到计算机或其他可编程数据处理设备上，使得在计算机或其他可编程设备上执行一系列操作步骤以产生计算机实现的处理，从而在计算机或其他可编程设备上执行的指令提供用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的步骤。
显然，本领域的技术人员可以对本申请进行各种改动和变型而不脱离本申请的精神和范围。这样，倘若本申请的这些修改和变型属于本申请权利要求及其等同技术的范围之内，则本申请也意图包含这些改动和变型在内。

Claims (17)

  1. 一种音频发送方法，该方法包括：
    智能电视采集当前播放的音频数据；
    确定所述音频数据是否为无效音频数据；
    响应于所述音频数据是无效音频数据，则所述智能电视在所述音频数据中添加无效音频数据指示信息，并对所述音频数据进行压缩处理，生成无效音频数据包；
    所述智能电视发送所述无效音频数据包至控制装置。
  2. 根据权利要求1所述的方法，该方法还包括：
    响应于所述音频数据不是无效音频数据，则将所述音频数据发送至所述控制装置。
  3. 根据权利要求1所述的方法，所述确定所述音频数据是否为无效音频数据包括：确定所述音频数据是否全部为无效字节。
  4. 根据权利要求3所述的方法，所述采集到的音频数据为完整的一帧音频数据，并且该帧音频数据的字符为预设字符，则确定所述音频数据为无效音频数据。
  5. 根据权利要求3所述的方法，在所述采集当前播放内容的音频数据之前，还包括：
    控制所述智能电视进入录音状态，对所述智能电视进行音频录制。
  6. 一种音频播放方法，该方法包括：
    接收智能电视发送的音频数据；
    对所述音频数据进行解析得到解析数据；
    根据所述解析数据确定所述音频数据是否为无效音频数据；
    响应于所述音频数据为无效音频数据，则丢弃所述无效音频数据。
  7. 根据权利要求6所述的方法，该方法还包括：
    响应于所述音频数据不是无效音频数据，播放所述音频数据。
  8. 根据权利要求6所述的方法，所述根据所述解析数据确定所述音频数据是否为无效音频数据包括：
    根据所述解析数据中的指示信息，确定所述音频数据是否为无效音频数据。
  9. 根据权利要求8所述的方法，所述指示信息位于所述音频数据的前N位字节，N为大于1的整数。
  10. 根据权利要求6所述的方法，所述播放所述音频数据，包括：
    将所述音频数据存入播放缓存中。
  11. 一种智能电视，包括：
    存储器，配置为存储计算机指令；
    处理器，配置为执行所述计算机指令以使得所述智能电视执行：
    采集当前播放的音频数据；
    确定所述音频数据是否为无效音频数据；
    响应于所述音频数据是无效音频数据，则在所述音频数据中添加无效音频数据指示信息，并对所述音频数据进行压缩处理，生成无效音频数据包；
    发送所述无效音频数据包至控制装置。
  12. 根据权利要求11所述的智能电视，所述处理器还配置为执行所述计算机指令以使得所述智能电视执行：
    响应于所述音频数据不是无效音频数据，则将所述音频数据发送至所述控制装置。
  13. 根据权利要求11所述的智能电视，所述确定所述音频数据是否为无效音频数据包括：
    确定所述音频数据是否全部为无效字节。
  14. 根据权利要求13所述的智能电视，所述采集到的音频数据为完整的一帧音频数据，并且该帧音频数据的字符为预设字符，则确定所述音频数据为无效音频数据。
  15. 根据权利要求13所述的智能电视，所述处理器还配置为执行所述计算机指令以使得所述智能电视执行：
    控制所述智能电视进入录音状态，对所述智能电视进行音频录制。
  16. 一种控制装置，包括：
    存储器，配置为存储计算机指令；
    处理器，配置为执行所述计算机指令以使得所述控制装置执行：
    接收智能电视发送的音频数据；
    对所述音频数据进行解析得到解析数据；
    根据所述音频数据确定所述音频是否为无效音频数据；
    响应于所述音频数据为无效音频数据，则丢弃所述无效音频数据。
  17. 一种机器可读的非易失性存储介质，其上存储有计算机指令，所述计算机指令被执行时实现：
    采集当前播放的音频数据；
    确定所述音频数据是否为无效音频数据；
    响应于所述音频数据是无效音频数据，则在所述音频数据中添加无效音频数据指示信息，并对所述音频数据进行压缩处理，生成无效音频数据包；
    发送所述无效音频数据包至控制装置。
PCT/CN2019/107265 2018-12-20 2019-09-23 音频发送、播放的方法及智能电视 Ceased WO2020125110A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201980079748.6A CN113261300B (zh) 2018-12-20 2019-09-23 音频发送、播放的方法及智能电视

Applications Claiming Priority (6)

Application Number Priority Date Filing Date Title
CN201811565526.9 2018-12-20
CN201811565099.4 2018-12-20
CN201811565099.4A CN109495776B (zh) 2018-12-20 2018-12-20 一种音频发送、播放的方法及智能终端
CN201811565963.0 2018-12-20
CN201811565963.0A CN111355996A (zh) 2018-12-20 2018-12-20 一种音频播放方法及计算设备
CN201811565526.9A CN111352605A (zh) 2018-12-20 2018-12-20 一种音频播放、发送的方法及装置

Publications (1)

Publication Number Publication Date
WO2020125110A1 true WO2020125110A1 (zh) 2020-06-25

Family

ID=71100403

Family Applications (2)

Application Number Title Priority Date Filing Date
PCT/CN2019/104971 Ceased WO2020125076A1 (zh) 2018-12-20 2019-09-09 音频播放、发送方法和装置
PCT/CN2019/107265 Ceased WO2020125110A1 (zh) 2018-12-20 2019-09-23 音频发送、播放的方法及智能电视

Family Applications Before (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/104971 Ceased WO2020125076A1 (zh) 2018-12-20 2019-09-09 音频播放、发送方法和装置

Country Status (4)

Country Link
US (2) US11190836B2 (zh)
EP (1) EP3902271A4 (zh)
CN (1) CN113261300B (zh)
WO (2) WO2020125076A1 (zh)

Families Citing this family (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2020125076A1 (zh) * 2018-12-20 2020-06-25 青岛海信电器股份有限公司 音频播放、发送方法和装置
TWI811692B (zh) * 2020-06-12 2023-08-11 中央研究院 用於場景音轉換的方法與裝置及電話系統
CN114530148B (zh) * 2020-10-30 2025-10-24 华为终端有限公司 一种控制方法、装置及电子设备
CN112437339A (zh) * 2020-11-10 2021-03-02 深圳Tcl新技术有限公司 音频输出控制方法及装置、智能电视机、存储介质
CN115776698A (zh) * 2021-09-07 2023-03-10 炬芯科技股份有限公司 一种延时调整方法及相关装置
CN114143297B (zh) * 2021-11-25 2024-04-26 北京蔚领时代科技有限公司 适用于云游戏场景的超低延迟音频处理方法及装置
CN115297364B (zh) * 2022-07-27 2024-01-23 北京奇艺世纪科技有限公司 一种云游戏视频传输时延的确定方法、系统及电子设备
CN115691516B (zh) * 2022-11-02 2023-09-05 广东保伦电子股份有限公司 一种低延迟音频矩阵配置方法及服务器
CN117032620B (zh) * 2023-06-30 2024-07-30 荣耀终端有限公司 音频焦点控制方法及装置

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106935253A (zh) * 2017-03-10 2017-07-07 北京奇虎科技有限公司 音频文件的裁剪方法、装置及终端设备
CN106936847A (zh) * 2017-04-11 2017-07-07 深圳市米尔声学科技发展有限公司 音频数据的处理方法及处理器
US10037761B1 (en) * 2017-06-05 2018-07-31 Intel IP Corporation Audio decoder state update for packet loss concealment
CN109495776A (zh) * 2018-12-20 2019-03-19 青岛海信电器股份有限公司 一种音频发送、播放的方法及智能终端

Family Cites Families (33)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2005184383A (ja) 2003-12-18 2005-07-07 Sony Corp リアルタイムデータ通信システム、リアルタイムデータ通信装置及びリアルタイムデータ通信方法
EP2043372B1 (en) * 2007-09-26 2019-07-17 Vodafone GmbH Method for audio and video synchronization, receiving and transmitting device
WO2009098878A1 (ja) * 2008-02-06 2009-08-13 Panasonic Corporation 映像音声データ同期方法、映像出力機器、音声出力機器及び映像音声出力システム
EP2120224A3 (en) * 2008-05-14 2012-05-30 Samsung Electronics Co., Ltd. A method and communication system for controlling appliance device using a mobile device
US10080061B1 (en) * 2009-12-18 2018-09-18 Joseph F. Kirley Distributing audio signals for an audio/video presentation
EP3499503B1 (en) 2010-04-13 2024-07-03 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Decoding sample-accurate representation of an audio signal
TWI501673B (zh) 2011-02-16 2015-09-21 Amtran Technology Co Ltd 無線音訊與視訊同步播放的方法及其播放系統
JP5060649B1 (ja) * 2011-09-30 2012-10-31 株式会社東芝 情報再生装置及び情報再生方法
US9398344B2 (en) 2012-06-08 2016-07-19 Lg Electronics Inc. Image display apparatus, mobile terminal and method for operating the same
CN102800324A (zh) * 2012-07-30 2012-11-28 东莞宇龙通信科技有限公司 用于移动终端的音频处理系统和方法
CN102867402A (zh) 2012-08-17 2013-01-09 邓振强 移动通信终端遥控电器的方法
US9521449B2 (en) 2012-12-24 2016-12-13 Intel Corporation Techniques for audio synchronization
US9307508B2 (en) * 2013-04-29 2016-04-05 Google Technology Holdings LLC Systems and methods for syncronizing multiple electronic devices
CN103297824A (zh) * 2013-05-29 2013-09-11 华为技术有限公司 视频处理方法、电视棒、控制终端和系统
CN103546794A (zh) * 2013-10-30 2014-01-29 乐视致新电子科技（天津）有限公司 一种遥控装置和智能电视
CN103905876A (zh) * 2014-03-13 2014-07-02 北京奇艺世纪科技有限公司 一种视频数据和音频数据同步播放的方法、装置和设备
CN103905880A (zh) * 2014-03-13 2014-07-02 北京奇艺世纪科技有限公司 音视频数据的播放方法、智能电视和移动设备
CN103888818B (zh) * 2014-03-24 2017-12-19 青岛海信电器股份有限公司 一种电视节目播放方法、设备和系统
CN104254007B (zh) * 2014-09-03 2017-11-03 海信集团有限公司 一种音频处理方法及装置
CN105992040A (zh) * 2015-02-15 2016-10-05 深圳市民展科技开发有限公司 多声道音频数据发送方法、音频数据同步播放方法及装置
CN105992025B (zh) 2015-02-15 2019-09-27 湖南汇德电子有限公司 基于音频同步播放的系统时间校准方法、音频同步播放方法及装置
CN104902311B (zh) * 2015-05-28 2019-03-01 青岛海信宽带多媒体技术有限公司 一种音视频资源共享的方法、共享网关及系统
US20170006331A1 (en) * 2015-06-30 2017-01-05 Stmicroelectronics International N.V. Synchronized rendering of split multimedia content on network clients
CN105430436A (zh) 2015-10-30 2016-03-23 青岛海尔智能家电科技有限公司 一种移动终端共享电视音频的方法及装置
US10394518B2 (en) 2016-03-10 2019-08-27 Mediatek Inc. Audio synchronization method and associated electronic device
US10158905B2 (en) * 2016-09-14 2018-12-18 Dts, Inc. Systems and methods for wirelessly transmitting audio synchronously with rendering of video
CN106454453A (zh) 2016-11-15 2017-02-22 努比亚技术有限公司 电视控制装置、移动终端及方法
CN106789886A (zh) * 2016-11-16 2017-05-31 深圳市异度信息产业有限公司 数据的传输方法及装置
CN106792073B (zh) * 2016-12-29 2019-09-17 北京奇艺世纪科技有限公司 跨设备的音视频数据同步播放的方法、播放设备及系统
US9979998B1 (en) * 2017-05-02 2018-05-22 Amazon Technologies, Inc. System for time synchronization of audio devices
CN108184164B (zh) 2017-12-28 2021-06-11 深圳Tcl新技术有限公司 音频播放方法、智能电视及计算机可读存储介质
CN108449630B (zh) 2018-04-09 2020-05-22 歌尔股份有限公司 音频同步方法及系统
WO2020125076A1 (zh) * 2018-12-20 2020-06-25 青岛海信电器股份有限公司 音频播放、发送方法和装置

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106935253A (zh) * 2017-03-10 2017-07-07 北京奇虎科技有限公司 音频文件的裁剪方法、装置及终端设备
CN106936847A (zh) * 2017-04-11 2017-07-07 深圳市米尔声学科技发展有限公司 音频数据的处理方法及处理器
US10037761B1 (en) * 2017-06-05 2018-07-31 Intel IP Corporation Audio decoder state update for packet loss concealment
CN109495776A (zh) * 2018-12-20 2019-03-19 青岛海信电器股份有限公司 一种音频发送、播放的方法及智能终端

Also Published As

Publication number Publication date
CN113261300A (zh) 2021-08-13
EP3902271A1 (en) 2021-10-27
CN113261300B (zh) 2022-08-30
WO2020125076A1 (zh) 2020-06-25
US20200204854A1 (en) 2020-06-25
US11871075B2 (en) 2024-01-09
US11190836B2 (en) 2021-11-30
US20220053231A1 (en) 2022-02-17
EP3902271A4 (en) 2022-10-26

Similar Documents

Publication Publication Date Title
WO2020125076A1 (zh) 音频播放、发送方法和装置
US20210235153A1 (en) Video processing method, electronic device, and computer-readable medium
CN110662017B (zh) 一种视频播放质量检测方法和装置
EP3697088A1 (en) Video sending and receiving method, device, and terminal
CN111352605A (zh) 一种音频播放、发送的方法及装置
US20230345089A1 (en) Audio and Video Data Processing Method, Live Streaming Apparatus, Electronic Device, and Storage Medium
WO2018171548A1 (zh) 一种解码方法、终端以及计算机可读存储介质
US11908481B2 (en) Method for encoding live-streaming data and encoding device
CN110248233A (zh) 一种音视频播放方法、装置、设备及存储介质
EP3402212A1 (en) Audio data pushing method and system
CN109495776B (zh) 一种音频发送、播放的方法及智能终端
CN116762344A (zh) 音视频数据的处理方法、直播装置、电子设备和存储介质
CN112995730A (zh) 音画同步调整方法、装置、电子设备及介质
CN115208864B (zh) 数据传输方法、装置、设备、车辆及存储介质
WO2020035036A1 (zh) 一种节目录制、播放方法及装置
CN111355996A (zh) 一种音频播放方法及计算设备
US20240340507A1 (en) Methods, systems, and media for streaming video content using adaptive buffers
CN115357813A (zh) 采样方法、装置及电子设备
US20240153522A1 (en) Audio playback method, electronic device, and computer-readable storage medium
CN118250486A (zh) 视频卡顿的检测方法、装置、终端及存储介质
CN118828715A (zh) 播放音频的方法及装置、音频设备以及芯片
CN115942000A (zh) 基于h.264格式的视频流转码方法及装置、设备及介质
CN118055277A (zh) 视频处理方法、装置、设备与计算机存储介质
CN105828131A (zh) 一种视频分享方法及通信终端
WO2023015436A1 (zh) 流媒体数据传输方法、装置及终端设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19899712

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19899712

Country of ref document: EP

Kind code of ref document: A1