WO2018157743A1

WO2018157743A1 - Media data processing method, device, system and storage medium

Info

Publication number: WO2018157743A1
Application number: PCT/CN2018/076676
Authority: WO
Inventors: 江庆海
Original assignee: 腾讯科技（深圳）有限公司
Priority date: 2017-02-28
Filing date: 2018-02-13
Publication date: 2018-09-07
Also published as: CN108512814A; CN108512814B

Abstract

Disclosed in the embodiments of the present application is a media data processing method, the method comprising: performing, for any client of a current service, the following processing: receiving first media data uploaded by the client; processing the first media data according to pre-configured media data processing requirements corresponding to the current service and obtaining second media; storing the second media data at a network side and obtaining a link address of the second media data; and in response to a link address acquisition request from a service server, sending the obtained one or more link addresses of the current service to the service server so that the service server provides one or more of the link addresses available to any client of the current service. Also disclosed in the embodiments of the present application are a corresponding device and a system.

Description

Media data processing method, device, system and storage medium

The present application claims the priority of the Chinese Patent Application, which is filed on Jan. 28, 2017, filed on Jan. 28,,,,,,,,,,,,,,,,,,,,,,,, in.

Technical field

The present application relates to the field of Internet technologies, and in particular, to a media data processing method, apparatus, system, and storage medium.

background

With the development of computer technology and Internet technology, more and more users need to process audio and video recorded by their own personal recording or work needs in more occasions, such as transposition, background music, audio conversion, editing video material. Playback, etc., there are many software and technologies for processing audio and video on the market, such as ffmpeg technology, ffmpeg is an open source computer program that can be used to record, convert digital audio, video, and convert it into a stream. Use LGPL or GPL license. It provides a complete solution for recording, converting, and streaming audio and video. It contains a very advanced audio/video codec library libavcodec. Ffmpeg is widely used in various playback software.

Technical content

The embodiment of the present application provides a media data processing method, which is applied to a data processing server. The method includes: performing, for any client of the current service, processing: receiving first media data uploaded by the client; according to pre-configured The media data processing requirement corresponding to the current service, processing the first media data and obtaining the second media data; and storing the second media data on the network side and obtaining a link address of the second media data; Transmitting one or more of the obtained link addresses of the current service to the service server in response to a link address acquisition request from the service server, so that the service server provides the client to any client of the current service One or more of the link addresses available at the end.

The embodiment of the present application further provides a media data processing method, which is applied to a client, and the method includes: uploading, to a data processing server, the first media data generated by the current service, where the client is uploaded for any client of the current service. The first media data, the data processing server processes the first media data according to a pre-configured media data processing requirement corresponding to the current service, and obtains the second media data, and stores the second media data in a network. And obtaining a link address of the second media data; sending a link address acquisition request to the service server; wherein the service server responds to the link address acquisition request, one or a current service obtained from the data processing server Determining, among the plurality of link addresses, each of the link addresses available to the client; receiving, from the service server, each of the link addresses available to the client; and responding to the user for each of the received link addresses a link address selection operation, according to the selected link address from the network Acquiring the second media data; and the second media player acquired data.

The embodiment of the present application further provides a media data processing device, the device comprising a processor and a memory, the memory storing a computer readable instruction module configured to be executed by the processor; wherein the instruction module comprises: The upload processing module receives the first media data uploaded by any client of the current service; the data processing module processes the first media data and obtains the second media according to the pre-configured media data processing requirements corresponding to the current service. Data; storing the second media data on a network side and obtaining a link address of the second media data, and saving the link address in an information providing module; and the information providing module responsive to the service server And a link address obtaining request, sending the obtained one or more of the link addresses of the current service to the service server, so that the service server provides one or more of the link addresses to any client of the current service. .

The embodiment of the present application further provides a media data processing system, including: a service server, an upload server, a processing server, and an information providing server, where the information providing server is integrated in the processing server or independently configured; Uploading the server, receiving the first media data uploaded by any client of the current service, and providing the first media data to the processing server; the processing server, according to the pre-configured media data processing demand corresponding to the current service Processing the first media data and obtaining second media data; storing the second media data on a network side and obtaining a link address of the second media data, and saving the link address in the An information providing server that sends a link address acquisition request to the information providing server in response to a request for uploading the first media data from any client of the current service; the information providing server is responsive to The link address acquisition request of the service server, Obtaining current traffic link to one or more of said address to the service server; the service server is provided to the current client service according to any one or more of the links to one of the addresses available to the client.

The embodiment of the present application further provides a computer readable storage medium storing computer readable instructions, which may cause a processor to execute the methods of the embodiments.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings used in the description of the examples will be briefly described below. It is obvious that the drawings in the following description are only some embodiments of the present application, Those skilled in the art can also obtain other drawings based on these drawings without paying for creative labor.

1A is a schematic structural diagram of a system according to an embodiment of the present application;

FIG. 1B is a schematic structural diagram of another system according to an embodiment of the present application;

2 is a flowchart of a method according to an embodiment of the present application;

3 is a diagram of a media data processing system according to an embodiment of the present application;

4 is a distributed process according to an embodiment of the present application;

FIG. 5 is a flowchart of a method according to another embodiment of the present application; FIG.

6 is a message interaction diagram of a media data processing process in an embodiment of the present application;

Figure 7 is a structural diagram of a device according to an embodiment of the present application;

FIG. 8 is a structural diagram of a device according to another embodiment of the present application; and

FIG. 9 is a structural diagram of a device according to an embodiment of the present application.

Implementation

The technical solutions in the embodiments of the present application are clearly and completely described in the following with reference to the accompanying drawings in the embodiments of the present application. It is obvious that the described examples are only a part of the examples of the present application, and not all examples. All other examples obtained by a person of ordinary skill in the art based on the embodiments of the present application without creative efforts are within the scope of the present application.

In some instances, the user directly in the electronic device (the product refers to an application or platform with audio and video processing functions, such as: dubbing software, media player or WeChat, or can display some kind of media interaction activities) The web page browser) performs media data processing, such as audio and video processing. For example, if the user needs to perform audio and video processing when using the dubbing software (for example, after the dubbing is completed, the audio and video data needs to be processed), the audio and video processing is performed by the dubbing software, and other functions of the dubbing software are performed during the processing. unavailable. It can be seen that the current media data processing and product coupling degree is high, media data processing will affect the use of other functions of the product, and the user waits for a long time, which makes the user experience less effective. At the same time, the module used for media data processing in this product is complex in design, so that the product is difficult to implement and the product testing time is longer, so that the development cost of the product will be higher. Moreover, due to the limited processing capability of the user-side electronic device, the product may have a long processing time of the media data, and the long-term processing may bring potential risks, such as damage to the audio and video source files. Further, if the product changes the audio and video requirements, such as improving the audio and video processing effect or increasing the audio and video processing functions (such as adding English subtitles in audio and video), the product is also difficult to quickly achieve audio and video processing requirements. Changes. In addition, if the product is a browser for displaying a web page, the product cannot implement complicated audio and video processing, and it is difficult to achieve the audio and video processing effect desired by the user.

Based on the above technical problem, the embodiment of the present application provides a media data processing method, which can be applied to the system architecture shown in FIG. 1A. As shown in FIG. 1A, the system architecture includes a client 101 and a background 102. The background 102 includes a service server 103 (which may be a website server or an application server), a data processing server 104, and a storage server 105. The client 101 The backend 102 can communicate over the Internet 106. Here, the client 101 refers to the aforementioned product, and may be a browser for displaying a page of a media interactive activity (such as an interactive advertising event, the activity may be implemented in the form of a web page) or an application (APP) (such as a dubbing software, etc.).

The user using the client 101 can generate media data processing requirements such as transposition, background music, audio conversion, and clip playback of the video material. In some examples, data processing server 104 encapsulates these processing requirements into processing processes. For a plurality of clients 101 corresponding to multiple products (ie, multiple services), the processing requirements of media data of different types of clients 101 (ie, processing requirements of different services) may be encapsulated into different processing processes, and each type The service corresponds to a service identifier (such as a project ID). Each service identifier corresponds to a processing process, and the name of the processing process can be associated with the service identifier (for example, the name of the processing process can include the project ID).

The media data processing process for a client 101 is as follows:

After the client 101 successfully uploads the media data to the data processing server 104, the data processing server 104 returns the ID of the media data to the client 101. After the predetermined time, the client 101 queries the data processing server 104 for the processing of the media data via the service server 103. If the data processing server 104 has processed the media data, the second media data is uploaded to the storage server 105 on the network side, and the link address (URL) of the second media data is obtained and returned to the service server 103. The second media data is media data after the media data is processed. The client 101 acquires the second media data from the storage server 105 based on the above URL acquired from the service server 103.

The media data processing method in the embodiment of the present application is also applicable to the system architecture shown in FIG. 1B. As shown in FIG. 1B, the system architecture includes: a client 101 and a background 102, and the background 102 includes a service server 103. The business server may be a web server or an application server, a data processing server 104, a storage server 105, and an information providing server 106, wherein the data processing server 104 includes an upload server 107, a processing server 108, and an authentication server 109, a client 101 and a background 102 can communicate over the Internet 110.

The user can generate media data processing requirements using the client 101, and in some instances, the processing server 108 encapsulates the above processing requirements into a processing process.

The media data processing process for a client 101 is as follows:

After the client 101 successfully uploads the media data to the upload server 107, in some examples, the media data includes an encrypted verification ticket, and the verification server 109 responds to the upload of the client 101 before the client 101 uploads the media data. The media data request returns a ticket to the client 101 as a credential for the user identity security. The uploading server 107 returns the ID of the media data to the client 101. After a predetermined time, the client 101 queries the processing server 108 via the service server 103 for the processing status of the media data. If the processing server 108 has processed the media data, it will The second media data is uploaded to the storage server 105 on the network side, and the link address (URL) of the second media data is obtained and returned to the service server 103. The client 101 acquires the second media data from the storage server 105 based on the above URL acquired from the service server 103.

For example, the above system architecture can be used to process media data generated in an ad interaction. In an advertisement interaction activity, each user participating in the activity may upload media data to the upload server 107 through the client 101, wherein the media data includes an audio and video source file and a ticket returned from the verification server 109, and the processing server 108 The media data is subjected to audio and video processing. After the processing is successful, the processing server 108 uploads the second media data to the storage server 105, and the storage server 105 returns the URL of the second media data to the processing server 108, and the processing server 108 saves the URL to Information providing server 106. The service server 103 corresponding to the client 101 queries the information providing server 106 for the URL of the second media data, and returns the URL of the second media data to the client 101 according to the request of the client 101, for the user to directly use in the interactive advertising campaign. If the user can click on the URL, the created audio and video can be played.

For example, the above system architecture can also be used to process media data generated during a live broadcast. During the live broadcast process, each user participating in the activity may upload media data to the upload server 107 through the client 101, wherein the media data includes an audio and video source file and a ticket returned from the verification server 109, and the processing server 108 The media data is subjected to audio and video editing and various function additions. After the processing is successful, the processing server 108 uploads the second media data to the storage server 105, and the storage server 105 returns the URL of the second media data to the processing server 108, and the processing server 108 This URL is saved to the information providing server 106. The service server 103 corresponding to the client 101 queries the information providing server 106 for the URL of the second media data, and returns the link address of the second media data to the client 101 according to the request of the client 101, for the user to play the live video.

The media data processing method proposed by the embodiment of the present application is applied to the data processing server 104. As shown in FIG. 2, the method includes the following steps:

Step 200: Perform any of the following steps 201-203 for any client of the current service:

Step 201: Receive first media data uploaded by the client.

The media data may be data such as audio and video, so that the first media data may be an audio and video source file. In some examples, the data processing server may receive and temporarily store the media data source file, and may be an upload server in the data processing server that temporarily stores the media data source file for the bypass processing server pair in the data processing server. Upload the media data source file stored by the server for processing.

In some examples, transmitting a verification ticket to the service server in response to a request from the service server to obtain a verification ticket;

Receiving the first media data, further receiving the verification ticket corresponding to the first media data;

Checking whether the first media data is legal according to the verification ticket;

When the verification result is that the first media data is legal, the process of processing the first media data and obtaining the second media data is performed.

In some examples, in response to the upload source file request of the client of the current service, the ticket server in the data processing server returns an encrypted verification ticket of the client as a credential for the user identity security to ensure uploading. The server is not attacked by illegal users.

In some examples, after receiving the ticket returned by the verification server, the client further uses a media data source file (such as an audio and video source file), the ticket, and user identification information (ie, a user ID registered in the corresponding application or activity, Data such as QQ number, micro-signal code and other instant messaging numbers are uploaded to the upload server. The uploading server parses the ticket to verify whether the request of the client is legal, and if the request is legal, receiving data such as the media data source file. When the upload is successful, the uploading server returns a file ID of the media data source file to the client, and the file ID may be part of the ticket data, and the ticket may be composed of, for example, a project ID, a user identification information, a file ID, and the sound. The information such as the upload date of the video source file. If the request is not valid, the upload server requests the client to resend the upload source file request.

After receiving the file ID, the client sends the file ID to the service server. The service server can be a web server or an application server, and the service server saves the file ID.

Step 202: Process the first media data and obtain second media data according to a pre-configured media data processing requirement corresponding to the current service.

In some examples, the data processing server separately encapsulates the media data processing requirements of the plurality of services into a plurality of processing processes, and associates the plurality of processing processes with the service identifiers of the plurality of services. The processing of the first media data includes: calling an associated processing process according to the service identifier carried by the first media data, to process the first media data by the processing process.

Specifically, the service identifier used by the data processing server may be a project ID, so that each service corresponds to a project ID, and the name of the processing process and the project ID may be associated, for example, the name of the process includes the project ID.

The data processing server may adopt a powerful audio and video processing technology such as ffmpeg, and the data processing server adopts a multi-process mode, and invokes a corresponding processing process according to the item ID corresponding to the current service. Here, the first media data uploaded by the client received by the data processing server may carry the service identifier of the current service (eg, the audio and video source file carries the item ID). During the processing, the data processing server saves processing information such as processing status and file ID of the process in real time, and the processing information may be stored in a local cache or stored in a dedicated cache server or other storage system ( As in redis db). The processing status can be divided into five types: 1. processing, 2, success, 3. parsing error (refers to protocol error for parsing audio and video processing request, such as requesting audio and video processing protocol parameters are incorrect), 4 Processing error (refers to audio and video processing failure, such as audio format failure caused by audio format mismatch), 5, upload storage server error.

Step 203: Store the second media data on the network side and obtain a link address of the second media data.

In some examples, the data processing server uploads the second media data to a storage server, wherein the storage server stores the second media data and generates the link address; receiving the link address from the storage server .

Specifically, after the processing of the processing is completed, the data processing server uploads the processed media data file, that is, the second media data, to the storage server, where the storage server may be a cloud storage server or a content distribution network ( a CDN) server, the cloud storage server or CDN server returns a download address (URL) of the second media data to the data processing server, wherein the second media data is the media processed by the data processing server by the data processing server data.

In some examples, after processing the first media data and obtaining the second media data, recording a processing state, wherein the processing state indicates that the processing is successful; wherein, when the processing is successful, The second media data is stored on the network side and obtains the link address; the method further includes: when the processing of the first media data uploaded by a client fails, providing the processing status to the a service server to cause the service server to provide the processing status to the client.

Step 204: Send one or more of the obtained link addresses of the obtained current service to the service server in response to the link address acquisition request from the service server, so that the service server is to any client of the current service. One or more of the link addresses available to the client are provided.

In some examples, after receiving the first media data, sending the identifier of the first media data to the service server; and obtaining the obtained link address of the second media data and the first The identifier of the media data is correspondingly saved;

The providing the link address to the service server includes:

Responding to the query request from the service server, searching for the corresponding link address according to the identifier of the first media data carried by the query request; and

Sending the found link address to the service server.

Here, after the service server learns that the product uploads the first media data successfully, it queries the bypass processing server in the data processing server for the first media data processing state after a predetermined time, and when the bypass processing server finishes processing, the second media data The download address (ie, the link address) is available, and the download address is returned to the service server.

In some examples, after the audio and video source file is successfully uploaded, the client may query the service server for the processing status of the audio and video source file after a predetermined time. After the bypass processing server successfully processes the audio and video source file, the link address of the corresponding audio and video file (ie, the second media data) can be obtained, and the link address can be returned to the service server, and the service server receives the received address. Each link address and the file ID of the corresponding audio and video source file are saved. The bypass processing server can also send the processing status of the audio and video source files (eg, 1, processing, 2, success, 3, parsing error, 4, processing error, 5, uploading storage server error, etc.) to the service server. . For a client, the service server may provide the processing status of the audio and video source file uploaded by the client to the client, or provide the link address of the one or more second media data available to the client to the client. The client obtains the second media data corresponding to the link address from the storage server by using any link address.

Here, the link address of the one or more second media data available to a certain client may include at least one of the following: 1. uploading by another user who participates in the same interactive activity (such as an advertisement interaction activity) with the current user of the client. a link address of each second media data corresponding to each of the first media data, and a link address of each second media data corresponding to each of the first media data uploaded by the user of the current user of the client for a certain media content 3. The link address of each second media data corresponding to each first media data uploaded by each user that is currently concerned by the client, and the first corresponding to each first media data uploaded by the friend of the current user of the client a link address of the second media data, a link address of each second media data corresponding to each first media data uploaded by all users of a certain media content, and the like.

For example, for the first item mentioned above, in an advertisement interaction activity, each user participating in the activity may upload an audio and video source file through a respective client, and obtain a link of an audio and video file (ie, second media data) by the background processing. The address, the link address corresponding to the audio and video files of each user generated in the advertisement interaction activity can be provided to the client of a certain user, so that the user can play the audio and video files of each user.

For example, for the second item mentioned above, the client is a dubbing software, and an audio/video source file (a dubbing work source file) uploaded by the current user of the client through the dubbing software is processed in the background to obtain a link address of a dubbed work and is published in In the platform of the dubbing software, the clients of other users who pay attention to this user can access the link address of the dubbed work and play it, and can also send comments or multimedia interactions for the dubbed work, and then upload the respective audio and video source files (for The voice comment or audio and video interactive information published by the dubbed work) to the background, the background can process the link address of each audio and video file, and provide the link address of each audio and video file to the client that publishes the dubbed work. The background address can also be provided to the clients of each user who pays attention to the dubbed work (ie, item 5 above).

For another example, for the third item, the client is a dubbing software, and an audio/video source file (a dubbing work source file) uploaded by the current user of the client through the dubbing software is processed in the background to obtain a link address of a dubbed work and is published in In the platform of the dubbing software, the clients of other users who are concerned by the user can also upload their respective audio and video source files (the dubbing work source files) to the background, and the background can process the link addresses of the respective audio and video files, and the respective sounds. The link address of the video file is provided to the client so that the user can access the dubbed work published by each other user of interest. For item 4 above, the user's client can obtain audio and video works or audio and video comments published by each friend.

In the embodiment of the present application, the link address of the one or more second media data that is available to the client is not limited to the above five situations, and according to different media interaction functions provided by the current client page and the user's permission settings, The link address of the second media data available to the client may be different. In summary, the service server may determine the link address of the second media data available in the current page of the client according to the business logic.

Here, since the data processing server is in the background, when the processing demand of the client for the media data is changed, only the processing process corresponding to the processing requirement needs to be adjusted in the background to implement the change quickly, without having to do the program to the client. Any changes, and the format conversion of media data and other processing will not occupy the processing resources of the user side, and can improve the performance of the entire system.

The processing process may be referred to as a bypass processing process of the client (such as a dubbing software, a media player, or a WeChat, etc.), and the processing process does not affect the normal operation of the client, thereby eliminating user use. The client waits to improve the user experience.

In some examples, the data processing server 104 may include an upload processing module, a data processing module, and an information providing module. The modules may be integrated in the same server device or distributed in different server devices. For example, the upload processing module may be located in the upload server, the data processing module may be located in the data processing server, and the information providing module may be located in a separate information providing server, or may be located on the same server (such as a data processing server) as the data processing module. As such, in some examples, data processing server 104 can include an upload server, a processing server (integrated with a data processing module, and an information providing module). In other examples, data processing server 104 can include an upload server, a processing server, and an information providing server.

The embodiment of the present application proposes a media data processing system. In some examples, the system 300 includes a service server 301, an upload server 302, a processing server 303, and an information providing server 304, and the information providing server 304 is integrated in the processing server 303 or independently configured; The uploading server 302 receives the first media data uploaded by any client of the current service, and provides the first media data to the processing server 303. The processing server 303 is configured according to the pre-configured media corresponding to the current service. Data processing requirements, processing the first media data and obtaining second media data; storing the second media data on a network side and obtaining a link address of the second media data, and saving the link address The information providing server 304; the service server 301, in response to a request for uploading the first media data from any client of the current service, transmitting a link address acquisition request to the information providing server 304; the information a server 304 is provided in response to the link address from the service server And sending, to the service server 301, one or more of the obtained link addresses of the current service; the service server 301 provides one or more available to the client for any client of the current service. The link address.

In some examples, the system 300 further includes: an authentication server 305; wherein the service server 301 further transmits a verification verification ticket to the verification server 305 in response to the request to upload the first media data The verification server 305 sends a verification ticket to the service server 301 in response to the request for obtaining the verification ticket from the service server 301; the service server 301 further sends the verification ticket to the server When the first server data is received, the uploading server 302 further receives the verification ticket corresponding to the first media data, and checks whether the first media data is legal according to the verification ticket; As a result, when the first media data is legal, the first media data is provided to the processing server 303.

In some instances, when multiple clients have more processing requirements and a single server may have insufficient processing power, a distributed processing scheme may be employed, and multiple data processing servers may be used for distributed processing, and the processed result information may be uniformly stored. Provides business server query processing status usage. In this case, the system 300 may include multiple sets of the upload server 302 and the processing server 303, wherein each group includes one of the upload server 302 and one of the processing servers 303, and the information providing server 304 Independently setting; the service server 301 further receives the first media data from the client, and selects one for the client, in response to a request for uploading the first media data from any client of the current service. The uploading server and the processing server 303, the first media data is sent to the selected upload server 302; the uploading server 302 provides the first media data to the same group of the processing Server 303.

In some examples, processing server 303 may be referred to as a bypass processing server, and information providing server 304 may be referred to as a processing information storage module.

As shown in FIG. 4, the distributed processing process involves: N clients (client 1, client 2, ... client N), a storage server storing second media data, and processing information (including URLs and/or processing). Status) Unified storage processing information storage module. The distributed processing platform further includes N groups of servers for media data processing, each group including an upload server and a bypass processing server, as shown in FIG. 4, which may include: an upload server 1 and a bypass processing server 1. uploading The server 2 and the bypass processing server 2, the upload server N, and the bypass processing server N. Because the embodiment of the present application mainly relates to distributed processing of media data, the service server and the verification server are not shown in the platform. In actual applications, the platform may also include one or more service servers and verification servers, where Any client may send a request to upload a media data source file to a service server corresponding to the current service, or obtain a ticket from the corresponding authentication server.

Then, the client sends data such as a ticket, a media data source file, and a project ID to the corresponding service server, and the service server selects an upload server and uploads data such as a ticket, a media data source file, and a project ID to the selected upload server. The selection rule may be as follows: the IP address and the port number list of each upload server are configured in the service server, and when the service server receives the request from the client, the method modulates the total number of the project ID and the upload server. Select one of the upload servers. For example: there are 7 project IDs, 1000, 1001, ..., 1006, and a total of 5 upload servers, the IP addresses are {"10.127.0.1", "10.127.0.2", "10.127.0.3", "10.127.0.4", "10.127.0.5"}, the upload server corresponding to the client 1 is: modulo the project ID of the service currently used by the client 1 and the total number of the upload server, 1000% 5 = 0, then determine The ID of the upload server corresponding to client 1 is 0, and the IP address of the upload server is server[0]=10.127.0.1. Similarly, the upload server corresponding to client 2 is: 1001%5=1, then the client is determined. The upload server ID corresponding to terminal 2 is 1, and the IP address of the upload server is server[1]=10.127.0.2. Similarly, the upload server corresponding to client 3 is: 1002%5=2, then client 3 is determined. The corresponding upload server ID is 2, and the IP address of the upload server is server[2]=10.127.0.3. Similarly, the upload server corresponding to client 4 is: 1003%5=3, then the client 4 is determined. The upload server ID is 3, and the IP address of the upload server is server[3]=10.127.0.4; similarly, the guest The upload server corresponding to the terminal 5 is: 1004% 5=4, and it is determined that the upload server identifier corresponding to the client 5 is 4, and the IP address of the upload server is server[4]=10.127.0.5; similarly, the client 6 The corresponding upload server is: 1005%5=0, it is determined that the upload server identifier corresponding to the client 6 is 0, and the IP address of the upload server is server[0]=10.127.0.1; similarly, the client 7 corresponds to The upload server is: 1006%5=1, it is determined that the upload server identifier corresponding to client 7 is 1, and the IP address of the upload server is server[1]=10.127.0.2.

After the data of the audio and video source file is successfully uploaded, the uploading server receiving the data notifies the corresponding bypass processing server to perform processing, and the notification carries the item ID, and the bypass processing server calls the item ID according to the item ID. The corresponding processing process is processed, and the processed information is uniformly placed in the processing information storage module, so that the service server queries the processing state for use. After the bypass processing server successfully processes the media data source file, the second media data is uploaded to the storage server, and the storage server returns the link address (URL) (processing information) of the second media data to the bypass processing server. The bypass processing server saves the URL to the processing information storage module.

The above processing technology solution can realize media data processing and product decoupling, and the data processing server processes asynchronously in the background, can process the processing requirements of the completed media data in a short time, and does not block during the processing. Other user actions, which makes the user experience better. The solution can quickly realize the needs of various complex media data processing and improve the user experience.

In some instances, for interactive advertising campaigns (such as for users to click on an ad to enter an event page, that is, a page that interacts with the user (the user can perform voice interactions or voice comments on the page), deepen the user’s awareness of the ad. There are often scenes that need to be processed using media data such as video and audio material. In general, editing audio and video, adding background music, subtitles, cutting audio and video into a required length of time, etc., requires a large time cost, and must be After the audio and video is completed, it can be uploaded to the storage server for storage, so that it can be called directly when it is used next time. With the technical solution of the embodiment of the present application, the client selects an uploading server in the distributed processing platform (as shown in FIG. 4) through the service server to upload the audio and video source file, and then the bypass processing server performs audio and video processing. After the processing is successful, the bypass processing server uploads the second media data to the storage server, and the storage server returns the URL of the second media data to the bypass processing server, and the bypass processing server saves the URL to the processing information storage module. The service server corresponding to the client queries the processing information storage module for the link address of the second media data, and returns the link address of the second media data to the client according to the request of the client, for the user to directly use in the interactive advertising campaign, such as Users can click on the URL to play the produced audio and video. The process of interacting between the client and the distributed processing platform (in the background in FIG. 1B) can be seen in FIG. 1B. At the same time, because the interactive advertising campaign has a relatively short time from development to launch, and the demand changes more frequently, the above technical solution of the embodiment of the present application can quickly meet the requirements of development online, function update, and demand change.

The foregoing technical solutions of the embodiments of the present application may also be used in combination with an existing live broadcast platform or a live broadcast event. After the live broadcast, there is a need for the clip playback of the video material. The embodiment of the present application uploads the video source file to the service server by providing a packaged distributed processing platform (as shown in FIG. 4), the live broadcast platform or the live broadcast event. The service server selects an upload server in the distributed processing platform, and then the audio and video clips and various functions are added by the bypass processing server, and finally the storage server provides the URL of the processed audio and video file to the bypass processing server. The road processing server saves the URL to the processing information storage module. The service server corresponding to the client queries the processing information storage module for the URL of the second media data, and returns the URL of the second media data to the client according to the request of the client, for the user to play the live video. The process of interacting between the client and the distributed processing platform (in the background in FIG. 1B) can be seen in FIG. 1B.

By adopting the above technical solution, various complex media data processing requirements can be quickly realized, the user's other operations are not blocked, and the media data processing requirements are quickly changed, and the client development and function update are saved. lot of time. Efficient and fast distributed asynchronous processing improves the user experience of products with media data processing requirements. At the same time, by opening the encapsulated platform portal to a live broadcast platform or a live broadcast event, the audio and video source can be quickly imported, and the live audio and video can be quickly produced, reused, and propagated.

The embodiment of the present application further provides a media data processing method, which can be applied to the client 101. As shown in FIG. 5, the method includes the following steps:

Step 501: Upload the first media data generated by the current service to the data processing server, where the data processing server is configured according to the pre-configured current service, for the first media data uploaded by any client of the current service. The media data processing requirement processes the first media data and obtains the second media data, and stores the second media data on a network side and obtains a link address of the second media data.

Step 502: Send a link address obtaining request to the service server, where the service server determines, according to the link address obtaining request, one or more link addresses of the current service obtained by the data processing server for the client. Each of the linked addresses available.

Step 503: Receive, from the service server, each of the link addresses available to the client.

Step 504: Acquire the second media data from the network side according to the selected link address in response to the user selecting operation of any one of the received link addresses.

Step 505: Play the acquired second media data.

In some examples, an upload request is sent to the service server; wherein the service server obtains an authentication ticket from the data processing server in response to the upload request; receiving the verification ticket from the service server; And the verification ticket is uploaded to the data processing server together with the first media data, so that the first media data is valid according to the verification ticket, and when the verification result is that the first media data is legal And performing the processing of processing the first media data and obtaining the second media data.

In some examples, receiving a processing status from the service server, wherein the processing status is used to indicate whether the data processing server performs the processing on the first media data uploaded by the client is successful; Processing status.

FIG. 6 is a diagram showing an interaction diagram of media data processing provided by an embodiment of the present application. The client is a browser for displaying a webpage of a media interaction activity (such as a live event), such that the corresponding service server is a web server; the data processing server 104 is in the figure by a dotted line. The server consists of a ticket server, an upload server, and a bypass processing server. Here, the media data may be audio and video data.

As shown in FIG. 6, the media data processing includes the following steps:

Step 601: The bypass processing server encapsulates the audio and video processing requirements of the activity that the client can participate in into a processing process corresponding to the processing requirement, where each activity corresponds to one project ID, and each processing process may correspond to one project ID. save.

Step 602: The client sends a request for uploading an audio and video source file to the website server through the web page of the activity. Here, when the user participates in the above activities, the user may generate audio and video processing requirements, for example, recording an audio and video and uploading the audio and video source files through the web page. At this time, the client sends an audio and video source to the website server. Request for a file.

Step 603: After receiving the above request, the website server obtains, from the verification server, a ticket for verifying whether the user request is legal.

Step 604: Correspondingly, the verification server returns the ticket to the website server, and the website server returns it to the client.

Step 605: The client uploads data such as the ticket, the item ID, the audio and video source file, and the user identification information to the upload server. Here, when the background uses a distributed processing platform, the data may be first sent to the service server, and the service server selects the upload server, and then sends the data to the selected upload server.

Here, the audio and video source file may include a voice interaction or a voice comment recorded by the user in an interactive advertising campaign, the voice needs to be processed, or the video recorded in the live event, the video needs to be edited and various functions. Add and so on.

Step 606: The uploading server parses the ticket. If the verification is legal, step 608 is performed; otherwise, step 607 is performed.

Step 607: The upload server feeds back the result of the upload failure to the client. After receiving the information, the client may resend the request for uploading the audio and video source file again, and continue to perform the above steps, that is, perform steps 601-606 again, or end the current process.

Step 608: The uploading server receives data such as audio and video source files uploaded by the client, and temporarily stores the audio and video source files locally for use by the bypass processing server.

Step 609: If the above data upload is successful, the upload server returns the file ID of the audio and video source file of the client.

Step 610: The client sends a request to save the file ID to the website server.

Step 611: The website server saves the file ID in response to the above request.

Step 612: The uploading server notifies the bypass processing server to process the audio and video source file, and the notification carries data such as an item ID, a file ID, and an audio and video source file.

Step 613: The bypass processing server calls the already packaged audio and video processing process according to the item ID, and performs audio and video processing, such as transposition, adding background music, audio conversion, or performing clip playback on the video material.

Step 614: The bypass processing server stores its processing information of the audio and video source file, such as the file ID and the processing status, into the local cache in real time for the client to query. The bypass processing server uniformly stores the processing information to the processing information storage module when the distributed processing platform is used in the background.

Step 615: After the bypass processing server successfully processes the audio and video source files, the successfully processed audio and video files, that is, the second media data, are uploaded to a storage server, such as a cloud storage platform. When the bypass processing server does not successfully process the audio/video source file, the audio and video files cannot be uploaded to the storage server, and steps 615-624 are not performed.

Step 616: The storage server stores the audio and video files.

Step 617: The storage server returns a link address (URL) of the audio and video file to the bypass processing server. Here, when the background uses a distributed processing platform, the storage server returns the URL to the bypass processing server, and the bypass processing server saves the URL to the processing information storage module.

Step 618: The bypass processing server stores the URL in the local cache and saves it corresponding to the file ID and the item ID. Similarly, when the background uses a distributed processing platform, the bypass processing server saves the URL corresponding to the file ID and the item ID to the processing information storage module.

Step 619: After the web server learns that the audio and video source file is successfully uploaded, the web server queries the bypass processing server for the processing status of the audio and video source file and the URL of the audio and video file according to the file ID. When the background adopts the distributed processing platform, the website server queries the processing information storage module for the processing status of the audio and video source file and the URL of the audio and video file according to the file ID.

Step 620: When the processing status is 2, that is, "success", the URL is available, the bypass processing server returns the URL to the website server; when the processing status is other, the URL is not returned. Further, the bypass processing server can feed back the processing status to the website server. Among them, the file ID is returned to the web server at the same time when returning the URL and processing status.

Step 621: After obtaining the URL, the website server saves the file ID and its corresponding URL, and obtains a series of file IDs and URLs, that is, a file ID and a list of URL lists. Further, the item ID of the current service and the file ID and the URL list are correspondingly saved to distinguish the respective audio and video files generated in different services and their URLs.

Step 622: The client requests a URL from the website server, and the request may be sent to the website server after the client uploads the audio and video source file (such as automatically refreshing the current page every 10s), or may click the client for the user. Sent by the corresponding control (such as the user clicks the refresh button in the page).

Step 623: The website server responds to the client's link address acquisition request, and returns the client-related URL (such as all URLs of the current page) to the client.

Step 624: The client acquires the second media data from the storage server according to any of the foregoing URLs, and plays the second media data, so that the user can view the processed audio and video files. Here, after obtaining a series of URLs, the client may be displayed in the page. Further, in response to the user selecting a URL, the second media data may be acquired from the storage server according to the selected URL.

Based on the method provided by the foregoing example, the embodiment of the present application further provides a media data processing apparatus, where the apparatus is applicable to a data processing server, where the data processing server may be a server cluster composed of several servers, or a cloud. Computing Service Center. As shown in Figure 7, the device includes:

The upload processing module 701 receives the first media data uploaded by any client of the current service.

The data processing module 702 processes the first media data and obtains the second media data according to the pre-configured media data processing requirement corresponding to the current service; and stores the second media data on the network side and obtains the first The link address of the second media data, and saves the link address in the information providing module.

The information providing module 703, in response to the link address obtaining request from the service server, transmitting one or more of the obtained link addresses of the current service to the service server, so that the service server is to the current service. Any client provides one or more of the linked addresses.

In some examples, the apparatus further includes: a verification module 704, responsive to a request from the service server to obtain a verification ticket, transmitting an authentication ticket to the service server; wherein the upload processing module receives the first Receiving, by the media data, the verification ticket corresponding to the first media data; verifying whether the first media data is legal according to the verification ticket; and when the verification result is that the first media data is legal, the data The processing module performs the processing of processing the first media data and obtaining the second media data.

In some examples, the data processing module 702 further pre-packages the media data processing requirements of the multiple services into multiple processing processes, and respectively associates the multiple processing processes with the service identifiers of the multiple services. The processing of the first media data by the data processing module 702 includes: calling an associated processing process according to the service identifier carried by the first media data, to the first process by the processing process Media data is processed.

In some examples, the upload processing module 701 further sends an identifier of the first media data to the client after receiving the first media data, so that the client An identifier of the media data is sent to the service server; the information providing module 703 further saves the link address of the second media data obtained by the data processing module 702 and the identifier of the first media data; The information providing module 703, when the one or more of the link addresses of the current service obtained by the data processing module 702 are sent to the service server, includes: acquiring one or And identifying, by the identifiers of the plurality of first media data, one or more of the link addresses; and sending the one or more of the link addresses that are found to the service server.

In some examples, the data processing module 702 further records a processing status in the information providing module after processing the first media data and obtaining the second media data, wherein the processing status indication is The processing is successful; wherein, the data processing module 702, when the processing is successful, storing the second media data on the network side and obtaining the link address; the information providing module 703, when the data processing And when the module fails to perform the processing on the first media data uploaded by any client, further providing the processing status to the service server, so that the service server provides the processing status to the client .

In some examples, the apparatus includes a plurality of sets of upload processing modules 701 and a data processing module 702, wherein each set includes an upload processing module 701 and a data processing module 702.

The embodiment of the present application further provides a media data processing device, which can be applied to a client, where the client can be a mobile phone, a PAD (tablet), a laptop portable computer, a desktop computer, or the like. As shown in FIG. 8, the apparatus 800 includes:

The uploading module 801 is configured to upload the first media data generated by the current service to the data processing server, where the data processing server is configured according to the pre-configured current service for the first media data uploaded by any client of the current service. The media data processing requirement processes the first media data and obtains second media data, and stores the second media data on a network side and obtains a link address of the second media data.

The first sending module 802 sends a link address obtaining request to the service server, where the service server determines, according to the link address obtaining request, one or more link addresses of the current service obtained by the data processing server Each of the linked addresses available to the client.

The first receiving module 803 receives, from the service server, each of the link addresses available to the client.

The obtaining module 804 is configured to acquire the second media data from the network side according to the selected link address in response to the user selecting a connection address of any one of the received link addresses.

The playing module 805 plays the acquired second media data.

In some examples, the apparatus further includes: a second sending module 806, sending an upload request to the service server; wherein the service server acquires the verification ticket from the data processing server in response to the upload request; Module 807, receiving the verification ticket from the service server; the uploading module uploads the verification ticket together with the first media data to the data processing server to make it validate according to the verification ticket Whether the first media data is legal. When the verification result is that the first media data is legal, the process of processing the first media data and obtaining the second media data is performed.

In some examples, the apparatus further includes: a third receiving module 808, receiving a processing status from the service server, wherein the processing status is used to indicate the first media data uploaded by the data processing server to the client Whether the processing is successful; and displaying the processing status.

The specific methods and principles for implementing various functions in each entity in the above system are described in the foregoing, and are not described here.

In addition, the devices and modules in the various embodiments of the present application may be integrated into one processing unit, or each module may exist physically separately, or two or more devices or modules may be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

In an embodiment, the media data processing apparatus described above can be executed in any computing device for media data processing and loaded into the memory of the computing device. As shown in FIG. 9, the computing device includes one or more processors (CPUs) 902, communication modules 904, memory 906, user interface 910, and a communication bus 908 for interconnecting these components.

The processor 902 can receive and transmit data through the communication module 904 to enable network communication and/or local communication.

User interface 910 includes one or more output devices 912 that include one or more speakers and/or one or more visual displays. User interface 910 also includes one or more input devices 914 including, for example, a keyboard, a mouse, a voice command input unit or loudspeaker, a touch screen display, a touch sensitive tablet, a gesture capture camera or other input button or control, and the like.

The memory 906 can be a high speed random access memory such as DRAM, SRAM, DDRRAM, or other random access solid state storage device; or a non-volatile memory such as one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, or Other non-volatile solid-state storage devices.

The memory 906 stores a set of instructions executable by the processor 902, including:

Operating system 916, including programs for processing various basic system services and for performing hardware related tasks;

The application 918 can include various applications for implementing media data processing, such an application being capable of implementing the processing flow in each of the above examples, such as the device 700 shown in FIG. 7 and/or the device shown in FIG. 800.

In some examples, the apparatus 700 may include the modules 701-703 of the modules 701-704 shown in FIG. 7, the apparatus 700 may further include a module 704, and each of the modules 701-704 may be stored in the memory 906. Machine executable instructions. The processor 902 can implement the functions of the modules 701 to 704 described above by executing the machine executable instructions in the modules 701 to 704 in the memory 906.

In some examples, device 800 can include modules 801-805 of modules 801-808 shown in FIG. 8, and device 800 can further include modules 806-808, and each module 801-808 can be in memory 906. Stored machine executable instructions. The processor 902 can implement the functions of the modules 801 to 808 described above by executing the machine executable instructions in the modules 801 to 808 in the memory 906.

Additionally, each embodiment of the present application can be implemented by a data processing program executed by a data processing device such as a computer. Obviously, the data processing program constitutes an embodiment of the present application. Further, a data processing program that is usually stored in one storage medium is executed by directly reading the program out of the storage medium or by installing or copying the program to a storage device (such as a hard disk and/or a memory) of the data processing device. Therefore, such a storage medium also constitutes an embodiment of the present application. The storage medium can use any type of recording method, such as paper storage medium (such as paper tape, etc.), magnetic storage medium (such as floppy disk, hard disk, flash memory, etc.), optical storage medium (such as CD-ROM, etc.), magneto-optical storage medium ( Such as MO, etc.).

Therefore, the embodiment of the present application further provides a non-volatile storage medium in which a data processing program is stored, and the data processing program is used to execute any one of the foregoing methods of the present application.

The above description is only the preferred embodiment of the present application, and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc., which are included in the spirit and principles of the embodiments of the present application, should be included in Within the scope of protection of this application.

Claims

A media data processing method, wherein the method is applied to a data processing server, the method comprising:

For any client of the current service, perform the following processing:

Receiving first media data uploaded by the client;

Processing the first media data and obtaining second media data according to a pre-configured media data processing requirement corresponding to the current service; and

Storing the second media data on a network side and obtaining a link address of the second media data;

And transmitting, in response to the link address obtaining request from the service server, one or more of the obtained link addresses of the obtained current service to the service server, so that the service server provides the client to any client of the current service. One or more of the linked addresses available to the client.
The method of claim 1 further comprising:

Before receiving the first media data, sending a verification ticket to the service server in response to a request from the service server to obtain a verification ticket;

Receiving the first media data, further receiving the verification ticket corresponding to the first media data;

Checking whether the first media data is legal according to the verification ticket;

When the verification result is that the first media data is legal, the process of processing the first media data and obtaining the second media data is performed.
The method according to claim 1, further comprising: separately pre-packaging media data processing requirements of the plurality of services into a plurality of processing processes, and associating the plurality of processing processes with the service identifiers of the plurality of services respectively ;

The processing of the first media data includes: calling an associated processing process according to the service identifier carried by the first media data, to process the first media data by the processing process.
The method of claim 1 further comprising:

After receiving the first media data, sending the identifier of the first media data to the client, so that the client sends the identifier of the first media data to the service server; and

And saving the obtained link address of the second media data and the identifier of the first media data;

The sending, to the service server, one or more of the link addresses of the obtained current service, includes:

And searching for the corresponding one or more of the link addresses according to the identifier of one or more of the first media data carried by the link address obtaining request; and

Sending the one or more of the found link addresses to the service server.
The method of claim 1, wherein the storing the second media data on a network side and obtaining a link address of the second media data comprises:

Uploading the second media data to a storage server, where the storage server stores the second media data and generates the link address;

The link address is received from the storage server.
The method of claim 1, further comprising: after processing the first media data and obtaining the second media data, recording a processing state, wherein the processing state indicates that the processing is successful;

Wherein, when the processing is successful, storing the second media data on a network side and obtaining the link address;

The method further includes:

When the processing of the first media data uploaded by any client fails, the processing status is provided to the service server, so that the service server provides the processing status to the client.
A media data processing method, wherein the method is applied to a client, and the method includes:

And uploading the first media data generated by the current service to the data processing server, where the data processing server processes the media data corresponding to the pre-configured current service according to the first media data uploaded by any client of the current service. Relating to processing the first media data and obtaining second media data, storing the second media data on a network side and obtaining a link address of the second media data;

Sending a link address acquisition request to the service server; wherein the service server determines, in response to the link address acquisition request, one of the one or more link addresses of the current service obtained by the data processing server for each client The link address;

Receiving, from the service server, each of the link addresses available to the client;

Retrieving the second media data from the network side according to the selected link address in response to a user selecting operation of any one of the received link addresses; and

Playing the acquired second media data.
The method of claim 7 further comprising:

Sending an upload request to the service server; wherein the service server acquires the verification ticket from the data processing server in response to the upload request;

Receiving the verification ticket from the service server;

And uploading the verification ticket together with the first media data to the data processing server, so as to verify whether the first media data is legal according to the verification ticket, and when the verification result is the first media When the data is legal, the processing of processing the first media data and obtaining the second media data is performed.
The method of claim 7 further comprising:

Receiving, by the service server, a processing status, where the processing status is used to indicate whether the data processing server performs the processing on the first media data uploaded by the client; and

Show the processing status.
A media data processing apparatus, wherein the apparatus comprises a processor and a memory, the memory storing a computer readable instruction module configured to be executed by the processor; wherein the instruction module comprises:

Uploading a processing module, and receiving first media data uploaded by any client of the current service;

The data processing module processes the first media data and obtains the second media data according to the pre-configured media data processing requirement corresponding to the current service; and stores the second media data on the network side and obtains the second a link address of the media data, and saving the link address in the information providing module; and

The information providing module, in response to a link address obtaining request from the service server, transmitting one or more of the obtained link addresses of the current service to the service server, so that the service server is to the current service A client provides one or more of the linked addresses.
The apparatus of claim 10, wherein the instruction module further comprises:

The verification module sends a verification ticket to the service server in response to a request from the service server to obtain a verification ticket;

And receiving, by the upload processing module, the verification ticket corresponding to the first media data, and verifying whether the first media data is legal according to the verification ticket;

When the verification result is that the first media data is legal, the data processing module performs the process of processing the first media data and obtaining the second media data.
The device according to claim 10, wherein

The data processing module further pre-packages the media data processing requirements of the plurality of services into a plurality of processing processes, and associates the multiple processing processes with the service identifiers of the multiple services;

The processing, by the data processing module, processing the first media data, including: calling an associated processing process according to the service identifier carried by the first media data, to perform the first media data by the processing process deal with.
The device according to claim 10, wherein

The uploading processing module, after receiving the first media data, further sending the identifier of the first media data to the client, so that the client sends the identifier of the first media data To the business server;

The information providing module further saves a link address of the second media data obtained by the data processing module and an identifier of the first media data;

The information providing module sends one or more of the link addresses of the current service obtained by the data processing module to the service server, including: one or more carried according to the link address acquisition request And identifying, by the identifier of the first media data, one or more of the link addresses; and sending the one or more of the link addresses that are found to the service server.
The device according to claim 10, wherein the data processing module further records a processing state in the information providing module after processing the first media data and obtaining the second media data, wherein The processing status indicates that the processing is successful;

The data processing module, when the processing is successful, storing the second media data on a network side and obtaining the link address; the information providing module, when the data processing module is to any client When the processing of the uploaded first media data fails, the processing status is further provided to the service server, so that the service server provides the processing status to the client.
The apparatus of claim 10 wherein said apparatus comprises a plurality of sets of upload processing modules and data processing modules, wherein each set comprises an upload processing module and a data processing module.
A media data processing system, comprising: a service server, an upload server, a processing server, and an information providing server, wherein the information providing server is integrated in the processing server or is independently set;

The uploading server receives the first media data uploaded by any client of the current service, and provides the first media data to the processing server;

The processing server processes the first media data and obtains the second media data according to the media data processing requirement corresponding to the current service, and stores the second media data on the network side and obtains the a link address of the second media data, and saving the link address in the information providing server;

The service server sends a link address acquisition request to the information providing server in response to a request for uploading the first media data from any client of the current service;

The information providing server, in response to the link address obtaining request from the service server, transmitting one or more of the obtained link addresses of the current service to the service server;

The service server provides one or more of the link addresses available to the client to any client of the current service.
The system of claim 16 further comprising: an authentication server; wherein

The service server further sends a request for obtaining the verification ticket to the verification server in response to the request for uploading the first media data;

The verification server sends a verification ticket to the service server in response to the request for obtaining the verification ticket from the service server;

The service server further sends the verification ticket to the client;

And receiving, by the uploading server, the verification ticket corresponding to the first media data, and verifying, according to the verification ticket, whether the first media data is legal; when the verification result is the When the media data is legal, the first media data is provided to the data processing server.
The system of claim 16 wherein said system comprises a plurality of said plurality of said uploading servers and said processing server, wherein each set comprises one of said uploading server and said one of said processing servers, said information providing server being independent Setting

The service server further receives the first media data from the client, and selects a set of the upload for the client, in response to a request for uploading the first media data from any client of the current service. The server and the processing server send the first media data to the selected upload server;

The uploading server provides the first media data to the processing server of the same group.
A non-transitory computer readable storage medium storing computer readable instructions, which may cause at least one processor to perform the method of any one of claims 1 to 9.