WO2020073777A1 - Procédé et appareil de traitement multimédia - Google Patents

Procédé et appareil de traitement multimédia Download PDF

Info

Publication number
WO2020073777A1
WO2020073777A1 PCT/CN2019/106327 CN2019106327W WO2020073777A1 WO 2020073777 A1 WO2020073777 A1 WO 2020073777A1 CN 2019106327 W CN2019106327 W CN 2019106327W WO 2020073777 A1 WO2020073777 A1 WO 2020073777A1
Authority
WO
WIPO (PCT)
Prior art keywords
terminal device
data
media
server
streaming
Prior art date
Application number
PCT/CN2019/106327
Other languages
English (en)
Chinese (zh)
Inventor
靳丽
吴亮
折小强
杜衡
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Publication of WO2020073777A1 publication Critical patent/WO2020073777A1/fr

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/239—Interfacing the upstream path of the transmission network, e.g. prioritizing client content requests
    • H04N21/2393—Interfacing the upstream path of the transmission network, e.g. prioritizing client content requests involving handling client requests
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/239—Interfacing the upstream path of the transmission network, e.g. prioritizing client content requests
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/438—Interfacing the downstream path of the transmission network originating from a server, e.g. retrieving encoded video stream packets from an IP network
    • H04N21/4385—Multiplex stream processing, e.g. multiplex stream decrypting
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/45—Management operations performed by the client for facilitating the reception of or the interaction with the content or administrating data related to the end-user or to the client device itself, e.g. learning user preferences for recommending movies, resolving scheduling conflicts
    • H04N21/462—Content or additional data management, e.g. creating a master electronic program guide from data received from the Internet and a Head-end, controlling the complexity of a video stream by scaling the resolution or bit-rate based on the client capabilities
    • H04N21/4627—Rights management associated to the content

Definitions

  • the embodiments of the present application relate to the communication field, and in particular, to a media processing method and device.
  • Cloud video has become one of the mainstream of the video industry.
  • Cloud video can be implemented based on cloud technology.
  • Cloud video based on cloud technology can load video pages through the browser of the cloud server to obtain video images.
  • Cloud The server then streams the video picture and transmits it to the terminal device, and the terminal device plays the video picture.
  • the cloud server performs video decoding on each frame of the rendered video, and then merges the decoded rendered image with the page information, and re-encodes the merged video information with the page information to output a support terminal.
  • the encoded video stream played by the device is sent to the terminal device for playback.
  • the cloud server decodes every frame of the video frame rendering, which consumes a lot of resources on the cloud server, and the cloud server recodes the video stream that combines the decoded video information and page information to increase the broadcast control delay. Time, thereby affecting the user experience.
  • An embodiment of the present application provides a media processing method and apparatus for a server to respectively send streaming data of a media playback page and streaming media data of a media requested by a terminal device to a terminal device, and the terminal device displays the media playback page and Play streaming media data to reduce server resource consumption and broadcast control delay.
  • a first aspect of an embodiment of the present application provides a media processing method, including:
  • the server obtains the media playback request from the terminal device; the server obtains the streaming data of the media playback page according to the media playback request, and the media information and streaming media data of the media requested by the terminal device to play, where Is the configuration information of the streaming media data; the server sends the streaming data and the media information to the terminal device, and the streaming data is used by the terminal device to display the media on the media playback page
  • the information is used by the terminal device to create a media player, and the streaming data is used by the terminal device to display the media playback page; the server sends the streaming media data to the terminal device to enable the terminal
  • the device uses the media player to play the streaming media data on the media playing page.
  • the server sends the streaming data of the media playback page and the streaming media data of the media requested by the terminal device to the terminal device, and the terminal device displays the media playback page and plays the streaming media data, so that the server does not need to Decode and merge the streaming data and streaming media data to reduce server resource consumption and broadcast control delay.
  • the server obtains streaming data of a media playback page according to the media playback request, and media requested by the terminal device to play
  • the media information and streaming media data include: the server loads the media playback page corresponding to the media playback request, and streams the media playback page to obtain streaming data of the media playback page;
  • the media playback page obtains the media information and the streaming media data.
  • the streaming data, the media information, and the streaming media data can be obtained to provide a
  • the specific method of acquiring the streaming data, the media information, and the streaming media data improves the feasibility of the solution.
  • the streaming media data includes an audio data stream and a video data stream.
  • the server sending the streaming media data to the terminal device includes the server sending the audio data stream and the video data stream to the terminal device, respectively. It can be seen from the second implementation manner of the first aspect that the audio data stream and the video data stream of the streaming media data are separately sent to the terminal device, which improves the reliability of the audio data stream and the video data stream.
  • the method further includes: the server acquiring decoder information of the terminal device from the terminal device, the decoder information including decoding supported by the terminal device Format; the server sending the streaming media data to the terminal device includes the server sending the streaming media data to the terminal device in an encoding format of a target encoding format, the target encoding format and the decoder Information matching. It can be seen from the third implementation manner of the first aspect that the server sends the streaming media data in the encoding format of the target encoding format to the terminal device, so that the terminal device can decode and play the streaming media data using a matching decoder.
  • the streaming media data is an encrypted stream
  • the method further includes the server sending a digital rights management DRM certificate to the terminal device, where the DRM certificate is used by the terminal device to decrypt the encrypted streaming media data.
  • the method further includes: the server obtains a connection request from the terminal device, the connection request includes device information of the terminal device; the server Triggering a device management platform to verify the terminal device according to the device information; when the device management platform passes the verification of the terminal device, the server establishes a connection relationship with the terminal device. It can be seen from the fifth implementation of the first aspect that by triggering the device management platform to verify the terminal device, only the terminal device that passes the verification will maintain the connection relationship with the server, which improves security.
  • the method further includes: the server caches the streaming media data in a first cache area; when the streaming media is cached in the first cache area When the data amount of the data reaches the first preset threshold, the server sends a reception instruction to the terminal device, where the reception instruction is used to instruct the terminal device to receive the streaming media data. It can be seen from the sixth implementation manner of the first aspect that the streaming media data is cached in the first buffer area, which improves the integrity and fluency of the streaming media data.
  • the server sending the streaming media data to the terminal device includes: the server receiving a data acquisition request sent by the terminal device, the data acquisition request includes a data type of the requested data;
  • the terminal device sends the buffered streaming media data. It can be seen from the seventh implementation of the first aspect that the server sends streaming media data to the terminal device according to the data type of the data request, which improves the accuracy of sending streaming media data.
  • a second aspect of an embodiment of the present application provides a media processing method, including:
  • the terminal device When the terminal device obtains the media playback request input by the user, the terminal device sends a media playback request to the server; the terminal device receives the media information sent by the server and the streaming data of the media playback page, and the media information is Configuration information of the streaming media data; the terminal device creates a media player based on the media information; the terminal device displays the media playback page based on the streaming data; the terminal device receives streaming media data sent by the server , The terminal device uses the media player to play the streaming media data on the media playing page. It can be seen from the second aspect that the terminal device separately decodes and displays the streaming data of the media playback page and decodes and plays the streaming media data, thereby reducing the broadcast control delay.
  • the streaming media data includes an audio data stream and a video data stream
  • the terminal device uses the media playback page
  • the playing of the streaming media data by the media player includes: the terminal device using the media player to play the audio data stream and the video data stream on the media playing page; the terminal device according to the audio and video
  • the frame display time PTS synchronizes the audio data stream and the video data. It can be seen from the first implementation of the second aspect that the terminal device synchronizes the audio data stream and the video data according to the PTS, so that the audio data stream and the video data can be played synchronously in the media player .
  • the terminal device uses the media playback on the media playback page Before the player plays the streaming media data, the method further includes: the terminal device receives a digital rights management DRM certificate sent by the server; and the terminal device decrypts the streaming media data according to the DRM certificate. It can be seen from the second implementation manner of the second aspect that the terminal device decrypts the streaming media data by using a DRM certificate, thereby improving the security of the streaming media data.
  • the terminal Before the device receives the streaming media data sent by the server, the method further includes: the terminal device receives a receiving instruction sent by the server; the terminal device sends a data acquisition request to the server according to the receiving instruction
  • the data acquisition request includes a data type of the requested data, and the data type is used to instruct the server to send streaming media data corresponding to the data type.
  • the terminal After the device receives the streaming media data sent by the server, the method further includes: the terminal device caches the streaming media data in a second cache area; when the streaming media data is cached in the second cache area When the amount of data reaches the second preset threshold, the terminal device stops receiving the streaming media data. It can be seen from the fourth implementation of the second aspect that the terminal device caches the streaming media in the second buffer area, which improves the integrity and fluency of the streaming media data.
  • the method further includes: the terminal device sends decoder information of the terminal device to the server, and the decoder information is used to instruct the server to send the encoding format as The streaming media data in a target encoding format, the target encoding format matches the decoder information. It can be seen from the fifth implementation manner of the second aspect that the terminal device sends decoder information to the server so that the encoding format of the requested data matches the decoder information of the terminal device.
  • the method further includes: the terminal device sends a connection request to the server, and the connection request includes device information of the terminal device.
  • the device information is used by the server to verify the terminal device. It can be seen from the sixth implementation manner of the second aspect that the terminal device sends device information to the server, so that the server can trigger verification of the terminal device based on the device information, thereby improving security.
  • all The streaming media data includes the audio data stream and the video data stream
  • the terminal device buffering the audio and video data in the second buffer area includes: the terminal device buffering the audio data stream and the video data, respectively Flow in the second buffer area.
  • all The terminal device using the media player to play the streaming media data on the media playback page includes: the terminal device reading the first data from the second cache area and caching the first data In the third cache area of the media player; the terminal device reads the second data from the third cache area and uses the media player to play the second data on the media playback page data. It can be seen from the eighth implementation of the second aspect that the terminal device caches the first data in the third cache area of the media player, so that the media player can directly read the second data from the third cache area for playback, Improve the integrity and fluency of streaming media data.
  • a third aspect of an embodiment of the present application provides a server.
  • the server includes: a memory, a transceiver, and at least one processor, and the memory stores instructions; the memory, the transceiver, and the at least one processor Connected by line;
  • the at least one processor calls the instruction to perform the message processing or control operation performed on the server side in the first aspect.
  • a fourth aspect of an embodiment of the present application provides a terminal device.
  • the terminal device includes: a memory, a transceiver, and at least one processor, and the memory stores instructions; the memory, the transceiver, and the at least one The processor is connected by line;
  • the at least one processor calls the instruction to perform the message processing or control operation performed on the terminal device side in the second aspect.
  • a fifth aspect of an embodiment of the present application provides a computer-readable storage medium, which includes instructions, which when executed on a computer, causes the computer to execute the method of the first aspect or any possible implementation manner of the first aspect .
  • a sixth aspect of an embodiment of the present application provides a computer-readable storage medium, which includes instructions, which when executed on a computer, causes the computer to execute the method of the second aspect or any possible implementation manner of the second aspect .
  • a seventh aspect of the embodiments of the present application provides a computer program product containing instructions, characterized in that, when it is run on a computer, the computer is caused to execute the method of the first aspect or any possible implementation manner of the first aspect.
  • An eighth aspect of an embodiment of the present application provides a computer program product containing instructions, which, when it runs on a computer, causes the computer to execute the method of the second aspect or any possible implementation manner of the second aspect.
  • the server obtains the streaming data and streaming media data of the media playback page according to the media playback request sent by the terminal device, and sends the streaming data and streaming media data to the terminal device separately, and the terminal device according to the streaming data Display the media playback page, and decode and play the streaming media data by the terminal device. Therefore, in this embodiment, the terminal device decodes and plays the streaming media data, which reduces the consumption of server resources. At the same time, the streaming data and the streaming media data are separately sent to the terminal device without re-encoding the streaming data and the streaming media data. Reduced broadcast control delay and improved user experience.
  • FIG. 1 is a schematic diagram of an application scenario provided by an embodiment of the present application
  • FIG. 3 is a schematic diagram of an embodiment of a media processing method provided by an embodiment of the present application.
  • FIG. 4 is a schematic diagram of another embodiment of a media processing method provided by an embodiment of this application.
  • FIG. 5 is a schematic diagram of a streaming media data cache provided by an embodiment of this application.
  • FIG. 6 is a schematic diagram of another streaming media data cache provided by an embodiment of the present application.
  • FIG. 7 is a schematic block diagram of a server provided by an embodiment of this application.
  • FIG. 8 is a schematic block diagram of a terminal device provided by an embodiment of this application.
  • FIG. 9 is another schematic block diagram of a server provided by an embodiment of this application.
  • FIG. 10 is another schematic block diagram of a terminal device provided by an embodiment of this application.
  • FIG. 11 is a schematic diagram of a hardware structure of a server provided by an embodiment of the present application.
  • FIG. 12 is a schematic diagram of a hardware structure of a terminal device provided by an embodiment of the present application.
  • An embodiment of the present application provides a media processing method and apparatus for a server to respectively send streaming data of a media playback page and streaming media data of a media requested by a terminal device to a terminal device, and the terminal device displays the media playback page and Play streaming media data to reduce server resource consumption and broadcast control delay.
  • FIG. 1 is a schematic diagram of an application scenario provided by an embodiment of the present application.
  • the schematic diagram of the application scenario may include a server 101 and a terminal device 102.
  • the server 101 may serve as a media delivery party, and the terminal device 102 may serve as a media delivery party.
  • the terminal device 102 sends a media playback request to the server 101.
  • the server 101 sends the streaming data of the media playback page and the media information and streaming media data of the media requested by the terminal device 102 to the terminal device 102 according to the media playback request, and then the terminal
  • the device 102 displays a media playback page, and creates a media player according to the media information, and causes the media player to play streaming media data.
  • the terminal device 102 may include various handheld devices with wireless communication functions, vehicle-mounted devices, wearable devices, computing devices, or other processing devices connected to a wireless modem.
  • the terminal device 102 may be a set-top box, a smart TV, a mobile station (MS), a subscriber unit (subscriber unit), a cellular phone (cellular phone), a smart phone (smart phone), a wireless data card, and a personal digital assistant ( Personal digital assistant (PDA) computer, tablet computer, wireless modem (modem), handheld device (handset), laptop computer (laptop computer), machine type communication (machine type communication (MTC) terminal, etc.
  • the server 101 provided in the embodiment of the present application may be a cloud server, and the cloud server has the characteristics of being highly distributed and highly virtualized.
  • the cloud server may send the streaming data and streaming media data of the media playback page obtained by the media playback request to the terminal device 102 in a remote delivery mode.
  • the media playback request may be a Web video broadcast request, where the video requested by the Web video broadcast request is Web video. That is to say, the cloud server can send the streaming data corresponding to the video playback page of the Web video broadcast request and the streaming media data of the Web video to the terminal device 102 by remote drop.
  • FIG. 2 is a system framework diagram provided by an embodiment of the present application.
  • the system framework diagram includes a server and a terminal device.
  • the server may serve as a media delivery party
  • the server may include a browser 201 and a delivery module 202
  • the terminal device may serve as a media recipient
  • the terminal device includes a delivery receiving module 203 and a media player 204.
  • the browser 201 may be used to load a media playback page corresponding to the media playback request and detect media information in the media playback page.
  • the browser 201 loads the media playback page corresponding to the media broadcast request, and can obtain page element images, and media information and streaming media data of the media requested to be played by the receiving module, where the media information is configuration information of the streaming media data.
  • the browser 201 outputs page element images, media information, and streaming media data to the delivery module 202.
  • the delivery module 202 may be used to receive and process the interactive instruction of the delivery receiving module 203, and deliver media information, streaming data and media data of the media playback page to the delivery receiving module 203.
  • the delivery module 202 has pixel processing capability and streaming media processing capability.
  • the pixel processing capability is to stream the page element image obtained from the browser 201 to obtain streaming data of the media playback page
  • the streaming media processing capability is to use the streaming media protocol Obtain the encapsulated streaming media data from the browser 201, and decapsulate the streaming media protocol, and cache the decapsulated streaming media data in the cache area.
  • the delivery receiving module 203 may be used for interaction with the delivery module 202, as well as receiving media information, streaming data and streaming media data of media playback pages.
  • the media player 204 can use the local multimedia processing capabilities to decode the streaming media data, and synchronize the decoded audio and video information and other processes to restore the playback process performed on the server, thereby achieving media playback.
  • a media processing method provided by an embodiment of the present application may include the following steps:
  • the terminal device sends a connection request to the server.
  • the terminal device Before playing the media, the terminal device initiates a connection request to the server, and the connection request carries the device information of the terminal device.
  • the device information may include information such as the physical address (MAC Addr) of the terminal device, the factory identifier (UID) of the terminal device, and the user name of the user who logs in to the terminal device.
  • the device information may be as shown in Table 3-1.
  • the server triggers the device management platform to verify the terminal device.
  • the server acquires the device information of the terminal device from the connection request sent by the terminal device, and the server triggers the device management platform to verify the terminal device according to the device information. It should be noted that the server can also trigger the operator to verify the terminal device based on the device information, which is not limited here.
  • the server When the device management platform passes the verification of the terminal device, the server establishes a connection relationship with the terminal device, allowing the terminal device to access the server.
  • the server may directly allow the terminal device to access the server without verifying the terminal device.
  • the terminal device sends decoder information to the server.
  • the terminal device After the terminal device is connected to the server, the terminal device uploads the decoder information of the terminal device to the server, where the decoder information may include information such as audio encoding format, video encoding format, and subtitle format supported by the terminal device. Specifically, the decoder information may be as shown in Table 3-2.
  • the server initializes the system.
  • the server After the server obtains the decoder information from the terminal device, the server confirms the decoding capability of the terminal device and initializes the system according to the decoder information.
  • the terminal device sends a media playback request.
  • the terminal device sends a media playback request to the server, where the media playback request includes key instruction information input by the user.
  • the key instruction information may be input by a user through a remote controller, a keyboard, a mouse, and other devices.
  • the key instruction information may be as shown in Table 3-3.
  • the server obtains streaming data, media information, and streaming media data.
  • the server determines the key instruction information from the media playback request and performs logic processing on the key instruction information, so that the media playback page corresponding to the media playback request can be determined, and the server loads the media playback page and streams the media playback page to Obtain the streaming data of the media playback page, and obtain the media information and streaming media data of the media requested to be played by the terminal device from the media playback page.
  • the streaming media data is the media data of the media requested by the media playback request
  • the streaming data is the page data of the media playback page corresponding to the media playback request.
  • the streaming data can be a media playback operation button, a fast forward icon, and a fast backward Information such as icons or pause icons. It should be noted that the streaming data may be a picture or a video stream.
  • the media information includes configuration information such as the video resolution, streaming media protocol, audio and video encoding format and duration of the streaming media data.
  • the server caches the streaming media data in the first cache area.
  • the server caches the streaming media data acquired from the media playback page in the first cache area, and monitors the data volume of the streaming media data cached in the first cache area.
  • the server caches the audio data stream and the video data stream of the streaming media data in the first buffer area, respectively.
  • the server sends streaming data and media information.
  • the server sends the streaming data and media information of the media playback page to the terminal device, where the media information is used by the terminal device to create a media player.
  • the terminal device creates a media player and displays a media playback page.
  • the terminal device obtains the media information from the server, and determines the configuration information such as the video resolution, streaming media protocol, audio and video encoding format and duration of the streaming media data from the media information, and then creates a media player according to the configuration information of the streaming media data.
  • the media player plays the audio and video information corresponding to the streaming media data.
  • the terminal device obtains the streaming data of the media playback page from the server, and displays the media playback page according to the streaming data, where the media playback page can be used to operate the media. For example, when the media playback page includes a fast-forward icon for media, the terminal device displays a fast-forward icon on the media playback page, and the user can perform fast-forward operations on the media through the fast-forward icon.
  • the media player created by the terminal device may be embedded in the media playback page.
  • the server sends a reception instruction to the terminal device.
  • the server sends a receiving instruction to the terminal device, and the receiving instruction is used to instruct the terminal device to receive the streaming media data.
  • the terminal device sends a data acquisition request to the server.
  • the terminal device After acquiring the receiving instruction from the server, the terminal device sends a data acquisition request to the server, where the data acquisition request includes the data type of the requested data.
  • the data type may be as shown in Table 3-4.
  • the server sends streaming media data to the terminal device.
  • the server acquires the data acquisition request sent by the terminal device, and determines the data type of the data requested by the terminal device from the data acquisition request.
  • the server sends streaming media data whose encoding format is the target encoding format to the terminal device according to the decoder information of the terminal device, where the target encoding format matches the decoder information.
  • the server when the streaming media data is an ES stream, the server separately sends the audio data stream and the video data stream of the streaming media data to the terminal device.
  • the server may also send a digital rights management (digital rights management (DRM) certificate to the terminal device.
  • DRM digital rights management
  • the DRM certificate is used by the terminal device to decrypt the encrypted streaming media data .
  • the terminal device caches the streaming media data in the second cache area.
  • the terminal device receives streaming media data from the server, and caches the streaming media data in the second buffer area of the terminal device. During the process of receiving the streaming media data, the terminal device can monitor the data volume of the streaming media data cached in the second cache area, and when the data volume of the streaming media data cached in the second cache area reaches the second pre- When the threshold is set, the terminal device stops requesting streaming media data.
  • the terminal device may resume receiving streaming media data.
  • the terminal device when the streaming media data is an ES stream, the terminal device separately caches the audio data stream and the video data stream of the streaming media data in the second buffer area.
  • the terminal device caches the streaming media data in the third buffer area.
  • the terminal device When the data volume of the streaming media data buffered in the second buffer area reaches the second preset threshold, the terminal device reads the first data from the second buffer area, and buffers the read first data in the media player The third cache area. It should be noted that the first data is included in the streaming media data cached in the second cache area, and the first data may be part or all of the streaming media data cached in the second cache area. In the process of reading the first data, the terminal device monitors the data amount of the first data buffered in the third buffer area, and when the data amount of the first data in the third buffer area reaches the third preset threshold, the terminal The device stops reading the first data.
  • the terminal device when the streaming media data is an ES stream, after the terminal device reads the audio data stream and the video data stream of the buffered streaming media data from the second buffer area, the terminal device The presentation time (PTS) of the frame synchronizes the audio data stream and the video data stream, and buffers the synchronized audio data stream and video data stream in the third buffer area.
  • PTS presentation time
  • the streaming media data is an ES stream
  • the first data includes an audio data stream and a video data stream
  • the terminal device reads the audio data stream and the video data stream from the second buffer area, Cache in the third cache area respectively.
  • the streaming media data passes through the three-level cache of the first cache area, the second cache area, and the third cache area, which ensures the smoothness and integrity of the streaming media data.
  • the terminal device uses a media player to play streaming media data.
  • the terminal device reads the second data from the third buffer area, where the second data is streaming media data.
  • the terminal device uses a media player to decode and play the streaming media data.
  • the second data is included in the first data cached in the third cache area, and the second data may be part or all of the first data cached in the third cache area, that is, the second data is streaming media Part or all of the data.
  • the terminal device when the streaming media data is an encrypted stream, before the terminal device decodes and plays the second data, the terminal device may also obtain a DRM certificate sent by the server and use the DRM certificate to perform encryption on the streaming media data Decrypt.
  • the server obtains the streaming data and streaming media data of the media playback page according to the media playback request sent by the terminal device, and sends the streaming data and streaming media data to the terminal device separately, and the terminal device displays according to the streaming data Media playback page, and the terminal device decodes and plays streaming media data. Therefore, in this embodiment, the terminal device decodes and plays the streaming media data, which reduces the consumption of server resources. At the same time, the streaming data and the streaming media data are separately sent to the terminal device without re-encoding the streaming data and the streaming media data. Reduced broadcast control delay and improved user experience.
  • another media processing method provided by an embodiment of the present application may include the following steps:
  • the delivery receiving module sends a connection request to the delivery module.
  • the drop receiving module Before playing the media, the drop receiving module initiates a connection request to the drop module, the connection request carrying device information of the terminal device.
  • the device information may include information such as the physical address (MAC Addr) of the terminal device, the factory identifier (UID) of the terminal device, and the user name of the user who logs in to the terminal device.
  • the device information may be as shown in Table 3-1.
  • the drop module triggers the device management platform to verify the terminal device.
  • the drop module obtains the device information of the terminal device from the connection request sent by the drop receiving module, and the drop module triggers the device management platform to verify the terminal device according to the device information of the terminal device. It should be noted that the delivery module can also trigger the operator to verify the terminal device based on the device information, which is not limited here.
  • the drop module establishes a connection relationship with the drop receiving module, thereby allowing the terminal device to access the server.
  • the drop module may directly allow the terminal device to access the server without verifying the terminal device.
  • the delivery receiving module sends decoder information to the delivery module.
  • the drop receiving module uploads the decoder information of the terminal device to the drop module.
  • the decoder information may include information such as audio encoding format, video encoding format, and subtitle format supported by the terminal device. Specifically, the decoder information may be as shown in Table 3-2.
  • the delivery module initializes the system.
  • the delivery module After the delivery module obtains the decoder information from the delivery receiving module, the delivery module confirms the decoding capability of the terminal device according to the decoder information, and creates a thread with the browser, where the thread is used for data interaction between the browser and the delivery module.
  • the drop receiving module sends a media playback request.
  • the delivery receiving module sends a media playback request to the delivery module, where the media playback request includes key instruction information input by the user.
  • the key instruction information may be input by a user through a remote controller, a keyboard, a mouse, and other devices.
  • the key instruction information may be as shown in Table 3-3.
  • the delivery module transmits a media playback request to the browser.
  • the delivery module transmits the media playback request obtained from the delivery receiving module to the browser through a thread.
  • the browser obtains page element images, media information, and streaming media data.
  • the browser After receiving the media playback request, the browser determines the key instruction information from the media playback request and performs logic processing on the key instruction information, so that the media playback page corresponding to the media playback request can be determined, and the browser loads the media playback page, In order to obtain the page element image, and the media information and streaming media data of the media that the receiving module requests to play.
  • the streaming media data is the media data of the media requested by the media playback request
  • the page element image is the page image of the media playback page corresponding to the media playback request
  • the media information includes the video resolution of the streaming media data, streaming media protocol, audio and video Configuration information such as encoding format and duration.
  • the browser sends page element images and media information to the delivery module.
  • the browser sends page element images and media information to the drop module through threads.
  • the drop module performs pixel processing on the page element image.
  • the delivery module After the delivery module obtains the page element image from the browser, it uses its pixel processing capability to stream the page element image to obtain the streaming data of the media playback page.
  • the streaming data may be information such as a media play operation button, a fast forward icon, a fast backward icon, or a pause icon. It should be noted that the streaming data may be a picture or a video stream.
  • the delivery module sends streaming data and media information to the delivery receiving module.
  • the delivery module sends the streaming data and media information of the media playback page to the delivery receiving module, where the media information is used by the delivery receiving module to create a media player.
  • the delivery and reception module creates a media player and displays a media playback page.
  • the delivery receiving module obtains media information from the delivery module, and determines the configuration information such as the video resolution, streaming media protocol, audio and video encoding format and duration of the streaming media data from the media information, and then creates a media player according to the configuration information of the streaming media data .
  • the delivery receiving module obtains the streaming data of the media playback page from the delivery module, and displays the media playback page according to the streaming data, where the media playback page can be used to operate the media. For example, when the media playback page includes a fast-forward icon for media, the terminal device displays a fast-forward icon on the media playback page, and the user can perform fast-forward operations on the media through the fast-forward icon.
  • the media player created by the delivery and receiving module may be embedded in the media playback page.
  • the delivery module receives streaming media data.
  • the delivery module After the delivery module receives the media information, it determines the streaming media protocol from the media information, and receives the streaming media data from the browser according to the streaming media protocol. After the delivery module obtains the streaming media data, the delivery module decapsulates the streaming media data, and The unpacked streaming media data is cached in the first cache area. It should be noted that, during the process of receiving streaming media data, the delivery module may monitor the data volume of the streaming media data buffered in the first buffer area.
  • the drop module separately caches the audio data stream and the video data stream of the streaming media data in the first buffer area.
  • the delivery module sends a receiving instruction to the delivery receiving module.
  • the delivery module sends a receiving instruction to the delivery receiving module, and the receiving instruction is used to instruct the delivery receiving module to receive the streaming media data.
  • the delivery receiving module sends a data acquisition request to the delivery module.
  • the delivery receiving module After receiving the receiving instruction from the delivery module, the delivery receiving module sends a data acquisition request to the delivery module, where the data acquisition request includes the data type of the requested data.
  • the data type may be as shown in Table 3-4.
  • the delivery module sends streaming media data to the delivery receiving module.
  • the placing module obtains the data obtaining request sent by the placing receiving module, and determines the data type of the data requested by the placing receiving module from the data obtaining request.
  • the drop module sends the streaming media data whose coding format is the target coding format to the drop receiving module according to the decoder information of the terminal device, where the target coding format matches the decoder information.
  • the delivery module separately sends the audio data stream and the video data stream of the streaming media data to the delivery receiving module.
  • the placement module may also send a DRM certificate to the placement receiving module, and the DRM certificate is used by the placement receiving module to decrypt the encrypted streaming media data.
  • the drop receiving module caches the streaming media data in the second cache area.
  • the drop receiving module receives streaming media data from the drop module, and caches the streaming media data in the second buffer area of the drop receiving module.
  • the placing and receiving module may monitor the data volume of the streaming media data buffered in the second buffer area. When the data volume of the streaming media data buffered in the second buffer area reaches the second preset threshold, the drop receiving module stops requesting the streaming media data.
  • the drop receiving module separately buffers the audio data stream and the video data stream of the streaming media data in the second buffer area.
  • the drop receiving module sends a data reading instruction to the media player.
  • the drop receiving module stops requesting the streaming media data and sends a data reading instruction to the media player.
  • the delivery receiving module may resume receiving streaming media data from the delivery module.
  • the media player caches the streaming media data in the third cache area.
  • the media player obtains the data reading instruction sent by the delivery receiving module.
  • the media player reads the first data from the second buffer area of the delivery receiving module according to the data reading instruction, and buffers the read first data in the third buffer area. It should be noted that the first data is included in the streaming media data cached in the second cache area, and the first data may be part or all of the streaming media data cached in the second cache area.
  • the media player may provide an interface to the delivery receiving module, and the delivery receiving module may perform the data amount of the first data buffered in the third buffer area through the interface Monitoring, when the data amount of the first data in the third buffer area reaches the third preset threshold, the drop receiving module stops the media player from reading the first data.
  • the media player when the streaming media data is an ES stream, the media player reads the audio data stream and the video data stream of the buffered streaming media data from the second buffer area respectively, and reads them separately according to the PTS pair
  • the audio data stream and the video data stream are synchronized, and the synchronized audio data stream and video data stream are cached in the third buffer area.
  • the streaming media data when the streaming media data is an ES stream, the streaming media data includes an audio data stream and a video data stream, and the media player reads the audio data stream and the video data stream from the second buffer area, And cached in the third cache area respectively.
  • the streaming media data passes through the three-level cache of the first cache area, the second cache area, and the third cache area, which ensures the smoothness and integrity of the streaming media data.
  • the media player plays the streaming media data.
  • the media player reads the second data from the third buffer area, where the second data is streaming media data.
  • the media player decodes and plays the streaming media data.
  • the second data is included in the first data cached in the third cache area, and the second data may be part or all of the first data cached in the third cache area, that is, the second data is streaming media Part or all of the data.
  • the media player when the streaming media data is an encrypted stream, before the media player decodes and plays the streaming media data, the media player may also obtain a DRM certificate, and use the DRM certificate to decrypt the encrypted streaming media data.
  • the media player when the streaming media data is an ES stream, the media player reads the audio data stream and the video data stream from the third buffer area respectively, and performs decoding and playback respectively.
  • the server's delivery and reception module sends the streaming data and streaming media data of the media playback page to the delivery and reception module, respectively, the delivery and reception module of the terminal device displays the media playback page, and the media player of the terminal device performs convection Media data is decoded and played. Therefore, in this embodiment, the media player of the terminal device decodes and plays the streaming media data, which reduces the consumption of server resources. At the same time, the streaming data and the streaming media data are separately sent to the drop receiving module of the terminal device, and the server does not need to stream Data and streaming media data are re-encoded, reducing broadcast control delay and improving user experience.
  • FIG. 5 is a schematic diagram of a streaming media data cache provided by an embodiment of the present application. As shown in FIG. ⁇ 502 ⁇ ⁇ Buffer area 503.
  • the first buffer area 501 is a buffer area of the placement module, and is used to buffer the streaming media data after the decapsulation of the placement module.
  • the drop module creates a thread and a first buffer area 501 according to the streaming protocol in the media information, the drop module decapsulates the received streaming media data, and then decapsulates the The streaming media data is cached in the first cache area 501.
  • the delivery module monitors the data volume of the streaming media data cached in the first cache area 501. When the data volume of the streaming media data cached in the first cache area 501 is greater than At a preset threshold, a receiving instruction is sent to the delivery receiving module.
  • the drop module may read the streaming media data from the first buffer area 501 and send the streaming media data to the drop receiving module through the transmission channel.
  • the second buffer area 502 is a buffer area of the placement receiving module, and is used to buffer the streaming media data received from the placement module.
  • the delivery receiving module determines the encoding format of the streaming media data, and creates a second buffer area 502 according to the encoding format.
  • the delivery receiving module may monitor the data volume of the streaming media data buffered in the second buffer area 502, when the streaming media data buffered in the second buffer area 502 When the amount of data is greater than the second preset threshold, the drop receiving module stops requesting streaming media data.
  • the third buffer area 503 is a buffer area of the media player, and is used to buffer the streaming media data read from the second buffer area 502.
  • the streaming media data read from the second buffer area 502 is buffered to the third buffer area 503 to fetch the data.
  • the streaming media data passes through the three-level cache, which effectively guarantees the fluency and integrity of the streaming media data, so that the audio data and video data of the streaming media data are played synchronously.
  • FIG. 6 is a schematic diagram of another streaming media data cache provided by an embodiment of the present application.
  • the schematic diagram of the streaming media data cache may be used to cache ES streams.
  • Three levels of cache are performed, including a first cache area 601, a second cache area 602, and a third cache area 603.
  • the first buffer area 601 includes a video buffer area 6011 and an audio buffer area 6012
  • the second buffer area 602 includes a video buffer area 6021 and an audio buffer area 6022
  • the third buffer area 603 includes a video buffer area 6031 and an audio buffer area 6032.
  • the video buffer area 6011 and the audio buffer area 6012 are buffer areas of the delivery module, and the video buffer area 6021 and the audio buffer area 6012 are used to buffer the video data stream and the audio data stream received from the browser, respectively.
  • the delivery module creates a video buffer area 6011 and an audio buffer area 6012 according to the streaming media protocol in the media information.
  • the delivery module requests the audio data stream and the video data stream from the browser respectively.
  • the delivery module will The received video data stream and audio data stream are decapsulated respectively, and then the decapsulated video data stream is buffered in the video buffer area 6011, and the unpacked audio data stream is buffered in the audio buffer area 6012.
  • the delivery module monitors the respective amounts of data corresponding to the video data stream buffered in the video buffer area 6011 and the audio data stream buffered in the audio buffer area 6012.
  • the drop module stops requesting the video data stream.
  • the drop The module stops requesting audio data streams.
  • the video buffer area 6021 and the audio buffer area 6022 are buffer areas of the drop receiving module, and the video buffer area 6021 and the audio buffer area 6022 are used to buffer the video data stream and audio data received from the drop module, respectively.
  • the delivery receiving module determines the encoding format corresponding to the video data stream and audio data, respectively, and creates an audio buffer area and a video buffer area according to the encoding format.
  • the dropping and receiving module may cache according to the type of streaming media data.
  • the audio data stream is cached in the audio buffer area 6022.
  • the video data stream is buffered in the video buffer area 6021.
  • the delivery and receiving module monitors the data amounts corresponding to the video data stream buffered in the video buffer area 6021 and the audio data stream in the audio buffer area 6022 respectively.
  • the video data stream of the stop request is placed, and when the data volume of the audio data stream buffered in the audio buffer area 6022 is greater than the second preset threshold At this time, the drop receiving module stops requesting the audio data stream.
  • the video buffer area 6031 and the audio buffer area 6032 are buffer areas of the media player, wherein the video buffer area 6031 is used to buffer the video data stream read from the video buffer area 6021, and the audio buffer area 6032 is used to buffer the video buffer area 6022 Read audio data stream.
  • the media player reads the video data stream and the audio data stream from the video buffer area 6021 and the audio buffer area 6022 of the second buffer area 602, respectively, and controls The data stream and the video data stream are synchronized, and the synchronized audio data stream and video data stream are buffered in the video buffer area 6031 and the audio buffer area 6032, respectively.
  • the video data stream and the audio data respectively pass through the three-level cache, which effectively guarantees the smoothness and integrity of the video data stream and the audio data, so that the video data stream and the audio data of the streaming media data are played synchronously.
  • FIG. 7 is a schematic block diagram of a server provided by an embodiment of the present application.
  • the server includes:
  • the obtaining unit 701 is used to obtain a media playback request
  • the processing unit 702 is configured to obtain streaming data of a media playback page, media information and streaming media data of the media requested to be played by the terminal device according to the media playback request, where the media information is configuration information of the streaming media data ;
  • the sending unit 703 is configured to send the streaming data and the media information to the terminal device, the media information is used by the terminal device to create a media player, and the streaming data is used by the terminal device to display The media playback page;
  • the sending unit 703 is further configured to send the streaming media data to the terminal device.
  • processing unit 702 is specifically configured to:
  • the streaming media data includes an audio data stream and a video data stream
  • the sending unit 703 is specifically configured to send the audio data stream and the video data stream to the terminal device, respectively.
  • the obtaining unit 701 is further configured to obtain decoder information of the terminal device from a terminal device, where the decoder information includes a decoding format supported by the terminal device;
  • the sending unit 703 is specifically configured to send the streaming media data whose encoding format is a target encoding format to the terminal device, where the target encoding format matches the decoder information.
  • the sending unit 703 is further configured to send a digital rights management DRM certificate to the terminal device, and the DRM certificate is used by the terminal device to decrypt the encrypted streaming media data.
  • the obtaining unit 701 is further configured to obtain a connection request from the terminal device, where the connection request includes device information of the terminal device;
  • the server further includes a triggering unit 704 and a establishing unit 705, the triggering unit 704 is used to trigger a device management platform to verify the terminal device according to the device information; the establishing unit 705 is used to act as the device management platform When verification of the terminal device is passed, a connection relationship is established with the terminal device.
  • the server further includes a first cache unit 706, and the first cache unit 706 is configured to cache the streaming media data in the first cache area;
  • the sending unit 703 is further configured to send a receiving instruction to the terminal device, and the receiving instruction is used to indicate The terminal device receives the streaming media data.
  • the acquisition unit 701 is further configured to receive a data acquisition request sent by the terminal device, where the data acquisition request includes the data type of the requested data;
  • the sending unit 703 is specifically configured to send the buffered streaming media data to the terminal device according to the data type.
  • the processing unit 702 obtains the streaming data and streaming media data of the media playback page according to the media playback request sent by the terminal device, and the sending unit 703 sends the streaming data and streaming media data to the terminal device respectively, and the terminal device
  • the streaming data displays the media playback page, and the terminal device decodes and plays the streaming media data. Therefore, in this embodiment, the terminal device decodes and plays the streaming media data to reduce the consumption of server resources.
  • the sending unit 703 sends the streaming data and the streaming media data to the terminal device separately, without re-streaming the streaming data and the streaming media data. Encoding reduces broadcast control delay and improves user experience.
  • the server provided by the embodiment of the present application is described above, and the terminal device provided by the embodiment of the present application is described below.
  • FIG. 8 is a schematic block diagram of a terminal device provided by an embodiment of the present application. As shown in FIG. 8, the terminal device includes:
  • the sending unit 801 is used to send a media playback request to the server;
  • the receiving unit 802 is configured to receive media information, streaming media data, and streaming data of a media playback page sent by the server, where the media information is configuration information of the streaming media data;
  • a creating unit 803, configured to create a media player based on the media information
  • a display unit 804 configured to display the media playback page based on the streaming data
  • the playing unit 805 is configured to play the streaming media data on the media playing page using the media player.
  • the streaming media data includes an audio data stream and a video data stream
  • the playback unit 805 is specifically configured to play the audio data stream using the media player on the media playback page And the video data stream;
  • the terminal device further includes a synchronization unit 806, which is specifically configured to synchronize the audio data stream and the video data according to the display time PTS of the audio and video frames.
  • the receiving unit 802 is further configured to receive a digital rights management DRM certificate sent by the server;
  • the terminal device further includes a decryption unit 807 for decrypting the streaming media data according to the DRM certificate.
  • the receiving unit 802 is further configured to receive a receiving instruction sent by the server;
  • the sending unit 801 is further configured to send a data acquisition request to the server according to the receiving instruction, the data acquisition request includes a data type of the requested data, and the data type is used to instruct the server to send the data type corresponding to Streaming data.
  • the terminal device further includes a second cache unit 808, and the second cache unit 808 is configured to cache the streaming media data in the second cache area;
  • the receiving unit 802 is further configured to stop receiving the streaming media data when the data volume of the streaming media data buffered in the second buffer area reaches a second preset threshold.
  • the sending unit 801 is further configured to send decoder information of the terminal device to the server, where the decoder information is used to instruct the server to send the encoding format to the target encoding format
  • the target encoding format matches the decoder information.
  • the sending unit 801 is further configured to send a connection request to the server, where the connection request includes device information of the terminal device, and the device information is used by the server to End device verification.
  • the receiving unit 802 receives the streaming media data and the streaming data of the media playback page sent by the server respectively, and the display unit 804 displays the media playback page according to the streaming data, and the playback unit 805 decodes the streaming media data Play, reduce the broadcast control delay of playing media.
  • FIG. 9 is another schematic block diagram of a server provided by an embodiment of the present application.
  • the server includes a browser 201 and a delivery module 202.
  • the actions performed by the browser 201 are similar to the actions performed by the browser involved in the embodiment corresponding to FIG. 4 described above, and details are not described here again.
  • the actions performed by the delivery module 202 are similar to the actions performed by the delivery module involved in the embodiment corresponding to FIG. 4 described above, and details are not described herein again.
  • FIG. 4 refer to FIG. 4.
  • the functional structures of the browser 201 and the delivery module 202 provided in this embodiment are similar to those described in the corresponding embodiment of FIG. 2 described above, and details are not described herein again.
  • FIG. 10 is another schematic block diagram of a terminal device provided by an embodiment of the present application.
  • the terminal device includes a drop receiving module 203 and a media player 204.
  • the action performed by the delivery and receiving module 203 is similar to the action performed by the delivery and receiving module involved in the embodiment corresponding to FIG. 4 described above, which will not be repeated here.
  • the actions performed by the media player 204 are similar to the actions performed by the media player involved in the embodiment corresponding to FIG. 4 described above, and details are not described here again.
  • FIG. 4 refer to FIG. 4.
  • the functional structure of the delivery receiving module 203 and the media player 204 provided in this embodiment is similar to that described in the corresponding embodiment of FIG. 2 described above, and details are not described herein again.
  • FIG. 11 is a schematic diagram of a hardware structure of a server provided by an embodiment of the present application.
  • the server includes:
  • the transceiver may include a receiver and a transmitter.
  • the memory 1150 may include a read-only memory and / or a random access memory, and provide operation instructions and data to the processor 1110.
  • a portion of the memory 1150 may also include non-volatile random access memory (NVRAM).
  • NVRAM non-volatile random access memory
  • the memory and the processor may be independently connected through a bus or an interface, or may be integrated together.
  • the memory 1150 stores the following elements, executable modules or data structures, or their subsets, or their extensions.
  • the corresponding operation is performed by calling the operation instruction stored in the memory 1150 (the operation instruction may be stored in the operating system).
  • the processor 1110 controls the operation of the server, and the processor 1110 may also be called a CPU (Central Processing Unit, central processing unit).
  • the memory 1150 may include a read-only memory and a random access memory, and provide instructions and data to the processor 1110. A portion of the memory 1150 may also include non-volatile random access memory (NVRAM).
  • NVRAM non-volatile random access memory
  • the various components of the server are coupled together through a bus system 1120.
  • the bus system 1120 may also include a power bus, a control bus, and a status signal bus. However, for clarity, various buses are marked as the bus system 1120 in the figure.
  • the method disclosed in the above embodiments of the present application may be applied to the processor 1110 or implemented by the processor 1110.
  • the processor 1110 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method may be completed by an integrated logic circuit of hardware in the processor 1110 or an instruction in the form of software.
  • the aforementioned processor 1110 may be a general-purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware Components.
  • DSP digital signal processor
  • ASIC application specific integrated circuit
  • FPGA off-the-shelf programmable gate array
  • the methods, steps, and logical block diagrams disclosed in the embodiments of the present application may be implemented or executed.
  • the general-purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
  • the steps of the method disclosed in conjunction with the embodiments of the present application may be directly embodied and executed by a hardware decoding processor, or may be executed and completed by a combination of hardware and software modules in the decoding processor.
  • the software module may be located in a mature storage medium in the art, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, an electrically erasable programmable memory, and a register.
  • the storage medium is located in the memory 1150.
  • the memory 1150 may be a physically independent unit, or may be integrated with the processor 1110.
  • the processor 1110 reads the information in the memory 1150 and completes the steps of the above method in combination with its hardware.
  • the transceiver 1130 may be used to execute the embodiment corresponding to FIG. 3 and the embodiment corresponding to FIG. 4 on the server side related to the receiving and sending operation steps, respectively. Or used to perform the steps of sending and receiving data on the server side in other optional embodiments.
  • the processor 1110 may be used to execute the steps of server-side data processing in the embodiment corresponding to FIG. 3 and the embodiment corresponding to FIG. 4, respectively. Or it is used to perform the steps of server-side data processing in other optional embodiments.
  • FIG. 12 is a schematic diagram of a hardware structure of a terminal device provided by an embodiment of the present application.
  • FIG. 12 shows a simplified schematic diagram of a hardware structure of a terminal device. It is easy to understand and convenient to illustrate.
  • the terminal device uses a mobile phone as an example.
  • the terminal device includes a processor, a memory, a radio frequency circuit, an antenna, and input and output devices.
  • the processor is mainly used for processing communication protocols and communication data, as well as controlling terminal devices, executing software programs, and processing data of software programs.
  • the memory is mainly used to store software programs and data.
  • the radio frequency circuit is mainly used for the conversion of the baseband signal and the radio frequency signal and the processing of the radio frequency signal.
  • the antenna is mainly used to send and receive radio frequency signals in the form of electromagnetic waves.
  • Input and output devices such as touch screens, display screens, and keyboards, are mainly used to receive data input by users and output data to users. It should be noted that some types of terminal devices may not have input / output devices.
  • the processor When data needs to be sent, the processor performs baseband processing on the data to be sent, and then outputs the baseband signal to the radio frequency circuit.
  • the radio frequency circuit processes the baseband signal after radio frequency processing and sends the radio frequency signal to the outside in the form of electromagnetic waves through the antenna.
  • the radio frequency circuit receives the radio frequency signal through the antenna, converts the radio frequency signal into a baseband signal, and outputs the baseband signal to the processor.
  • the processor converts the baseband signal into data and processes the data.
  • FIG. 12 only one memory and processor are shown in FIG. 12. In actual terminal equipment products, there may be one or more processors and one or more memories.
  • the memory may also be referred to as a storage medium or storage device.
  • the memory may be set independently of the processor, or may be integrated with the processor, which is not limited in the embodiments of the present application.
  • an antenna and a radio frequency circuit with a transceiver function can be regarded as a transceiver unit of a terminal device, and a processor with a processing function can be regarded as a processing unit of the terminal device. among them.
  • the transceiver unit may also be called a transceiver, a transceiver, a transceiver device, or the like.
  • the processing unit may also be called a processor, a processing board, a processing module, a processing device, and the like.
  • the device used to implement the receiving function in the transceiver unit can be regarded as the receiving unit
  • the device used to implement the sending function in the transceiver unit can be regarded as the sending unit
  • the transceiver unit includes a receiving unit and a sending unit.
  • the transceiver unit may sometimes be called a transceiver, transceiver, or transceiver circuit.
  • the receiving unit may sometimes be called a receiver, a receiver, or a receiving circuit.
  • the sending unit may sometimes be called a transmitter, a transmitter, or a transmitting circuit.
  • transceiving unit is used to perform the sending operation and the receiving operation on the terminal device side in the above method embodiment
  • processing unit is used to perform other processing operations on the terminal device in addition to the transceiving operation in the above method embodiment.
  • the disclosed system, device, and method may be implemented in other ways.
  • the device embodiments described above are only schematic.
  • the division of the units is only a division of logical functions.
  • there may be other divisions for example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored, or not implemented.
  • the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may be in electrical, mechanical or other forms.
  • the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
  • each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.
  • the above integrated unit can be implemented in the form of hardware or software function unit.
  • the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it may be stored in a computer-readable storage medium.
  • the technical solution of the present application may be essentially or part of the contribution to the existing technology or all or part of the technical solution may be embodied in the form of a software product, and the computer software product is stored in a storage medium , Including several instructions to enable a computer device (which may be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in the embodiments of the present application.
  • the aforementioned storage media include: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disk and other media that can store program code .

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Databases & Information Systems (AREA)
  • Computer Security & Cryptography (AREA)
  • Information Transfer Between Computers (AREA)

Abstract

L'invention concerne un procédé de traitement multimédia, qui est utilisé pour permettre à un serveur d'envoyer respectivement des données de diffusion en continu d'une page de lecture multimédia et pour diffuser en continu des données multimédias d'un support dont la lecture est demandée au dispositif terminal par un dispositif terminal. Le procédé selon des modes de réalisation de la présente invention comprend les étapes suivantes : le serveur obtient une requête de lecture multimédia à partir du dispositif terminal ; le serveur obtient les données de diffusion en continu de la page de lecture multimédia et les informations multimédias et les données multimédias de flux du support dont la lecture est demandée par un dispositif terminal selon la requête de lecture multimédia, les informations multimédias étant des informations de configuration des données multimédias de flux ; le serveur envoie les données de diffusion en continu et les informations multimédias au dispositif terminal, les informations multimédias étant utilisées pour permettre au dispositif terminal de créer un lecteur multimédia, et les données de diffusion en continu étant utilisées pour permettre au dispositif terminal d'afficher la page de lecture multimédia ; le serveur envoie les données multimédias du flux au dispositif terminal. Par conséquent, selon les modes de réalisation de la présente invention, le serveur envoie respectivement les données de diffusion en continu et les données multimédias du flux au dispositif terminal, et le dispositif terminal affiche la page de lecture multimédia et lit les données multimédias du flux de façon à réduire la consommation de ressources de serveur et le retard de commande de lecture.
PCT/CN2019/106327 2018-10-12 2019-09-18 Procédé et appareil de traitement multimédia WO2020073777A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201811191179.8A CN111050192A (zh) 2018-10-12 2018-10-12 媒体处理方法及装置
CN201811191179.8 2018-10-12

Publications (1)

Publication Number Publication Date
WO2020073777A1 true WO2020073777A1 (fr) 2020-04-16

Family

ID=70164235

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/106327 WO2020073777A1 (fr) 2018-10-12 2019-09-18 Procédé et appareil de traitement multimédia

Country Status (2)

Country Link
CN (1) CN111050192A (fr)
WO (1) WO2020073777A1 (fr)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113595976B (zh) * 2021-06-15 2024-07-30 阿里巴巴创新公司 多媒体播放方法、云服务器、系统及存储介质
CN115695843B (zh) * 2022-10-27 2024-06-18 中国联合网络通信集团有限公司 预制视频播放方法、服务器、终端、介质及系统

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102215424A (zh) * 2010-04-12 2011-10-12 腾讯科技（深圳）有限公司 一种网络电视直播的方法、装置和系统
CN103561072A (zh) * 2013-10-25 2014-02-05 李俊松 一种基于互联网的云数据处理方法及云数码相框
CN104159150A (zh) * 2014-08-19 2014-11-19 龙芯中科技术有限公司 云终端、云服务器、媒体数据流播放系统及方法
US20170006314A1 (en) * 2014-05-06 2017-01-05 Tivo Inc. Managing media content upload groups

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102281473A (zh) * 2011-09-09 2011-12-14 青岛海信电器股份有限公司 电视终端的信息处理方法及电视终端
CN103179431B (zh) * 2013-03-21 2016-06-01 福建升腾资讯有限公司 Vdi环境下音视频重定向转码分离方法
CN104168486A (zh) * 2013-05-20 2014-11-26 上海范氏科技投资发展集团有限公司 基于云计算的虚拟机与客户端间视频重定向方法
CN104301742A (zh) * 2013-07-16 2015-01-21 上海国富光启云计算科技有限公司 虚拟机和客户端间视频重定向装置及其使用方法
CN105611394B (zh) * 2015-12-21 2018-11-06 福建时迅信息科技有限公司 Vdi环境下的视频重定向方法及系统
CN107360443B (zh) * 2016-05-09 2019-10-01 南京中兴新软件有限责任公司 一种云桌面图像处理方法、云桌面服务器及客户端
CN108174214A (zh) * 2017-12-08 2018-06-15 重庆邮电大学 一种基于屏幕内容视频编码的远程桌面共享方法

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102215424A (zh) * 2010-04-12 2011-10-12 腾讯科技（深圳）有限公司 一种网络电视直播的方法、装置和系统
CN103561072A (zh) * 2013-10-25 2014-02-05 李俊松 一种基于互联网的云数据处理方法及云数码相框
US20170006314A1 (en) * 2014-05-06 2017-01-05 Tivo Inc. Managing media content upload groups
CN104159150A (zh) * 2014-08-19 2014-11-19 龙芯中科技术有限公司 云终端、云服务器、媒体数据流播放系统及方法

Also Published As

Publication number Publication date
CN111050192A (zh) 2020-04-21

Similar Documents

Publication Publication Date Title
US20220263885A1 (en) Adaptive media streaming method and apparatus according to decoding performance
US20240283834A1 (en) Systems and methods for multi-device media broadcasting or recording with low-latency active control
US11368731B2 (en) Method and apparatus for segmenting data
JP2021514167A (ja) メディアコンテンツの配信および再生
CN107211186B (zh) 用于提供多视图流媒体服务的方法和设备
US8902967B2 (en) Systems and methods for distributed media stream transcoding and sharing
EP3267331B1 (fr) Procédé et appareil pour service de diffusion en continu en nuage
WO2021031739A1 (fr) Procédé de lecture vidéo de bureau en nuage, serveur, terminal, et support de stockage
US10171530B2 (en) Devices and methods for transmitting adaptively adjusted documents
EP3937434B1 (fr) Procédé de distribution de données et dispositif de réseau
CN102387187A (zh) 服务器、客户端及利用其远程播放视频文件的方法和系统
US20140325024A1 (en) Maximizing throughput of streaming media by simultaneously connecting to streaming media server over multiple independent network connections
WO2020073777A1 (fr) Procédé et appareil de traitement multimédia
US20150086184A1 (en) Reception apparatus and reception method
KR102299615B1 (ko) 컨텐트 분산 네트워크들에서 엠펙 미디어 전송 통합을 위한 방법 및 장치
US20160269757A1 (en) Apparatus, systems and methods for remote storage of media content events
CN113709493A (zh) 一种kvm系统的视频流数据加密装置、方法及设备
CN106210867A (zh) 一种数据分享的方法和装置
WO2024060719A1 (fr) Procédés de transmission de données, appareil, dispositif électronique et support de stockage
WO2024087208A1 (fr) Procédé et système de lecture de vidéo, et support de stockage
EP3785443A1 (fr) Système et procédé pour restituer un flux multimédia vidéo assemblé, sur un dispositif d'affichage
US20150156237A1 (en) Video Streaming System, Computer Device, and Video Streaming Method

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19870798

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19870798

Country of ref document: EP

Kind code of ref document: A1