CN117668263A

CN117668263A - Multimedia data processing method and device

Info

Publication number: CN117668263A
Application number: CN202211057573.9A
Authority: CN
Inventors: 欧阳晓
Original assignee: Shuhang Technology Beijing Co ltd
Current assignee: Shuhang Technology Beijing Co ltd
Priority date: 2022-08-31
Filing date: 2022-08-31
Publication date: 2024-03-08
Also published as: WO2024045961A1

Abstract

The embodiment of the application relates to the field of Internet and discloses a method and a device for processing multimedia data. The method comprises the following steps: determining a plurality of alternative multimedia data corresponding to the target multimedia data, wherein the plurality of alternative multimedia data are obtained by processing the target multimedia data based on the data characteristics of the target multimedia data, and the data content of each alternative multimedia data is matched with the data content of the target multimedia data; acquiring object information of a target client; searching the candidate multimedia data matched with the object information in a plurality of candidate multimedia data corresponding to the target multimedia data; and sending the searched alternative multimedia data to a target client, wherein the target client is used for outputting the searched alternative multimedia data. By adopting the embodiment of the invention, the multimedia data issued by the server to the client can be ensured to meet the requirement of the object of the client.

Description

Multimedia data processing method and device

Technical Field

The present disclosure relates to the field of the internet, and in particular, to a method and an apparatus for processing multimedia data.

Background

The publisher can publish the multimedia data to the server, taking the multimedia data as a video as an example, the server directly distributes the video published by the publisher to one or more clients, and the object of the client can only view the video published by the publisher. However, the above manner does not take into account the different requirements of different objects on the video, and based on this, how to ensure that the multimedia data sent by the server to the client meets the requirements of the objects of the client is a problem that needs to be solved at present.

Disclosure of Invention

The embodiment of the application provides a method and a device for processing multimedia data, which can ensure that the multimedia data issued by a server to a client meets the requirement of an object of the client.

In one aspect, an embodiment of the present application provides a method for processing multimedia data, where the method includes:

determining a plurality of alternative multimedia data corresponding to target multimedia data, wherein the plurality of alternative multimedia data are obtained by processing the target multimedia data based on the data characteristics of the target multimedia data, and the data content of each alternative multimedia data is matched with the data content of the target multimedia data;

acquiring object information of a target client;

searching the candidate multimedia data matched with the object information in a plurality of candidate multimedia data corresponding to the target multimedia data;

and sending the searched alternative multimedia data to the target client, wherein the target client is used for outputting the searched alternative multimedia data.

In one embodiment, further comprising: acquiring data characteristics of the target multimedia data;

judging whether to generate alternative multimedia data corresponding to the target multimedia data according to the data characteristics of the target multimedia data;

If yes, generating a plurality of alternative multimedia data corresponding to the target multimedia data.

In one embodiment, the determining whether to generate the candidate multimedia data corresponding to the target multimedia data according to the data characteristics of the target multimedia data includes:

comparing the data characteristics of the target multimedia data with preset data characteristics in a preset data characteristic set, wherein different objects have different requirements on the multimedia data with the preset data characteristics;

and if the preset data features with the similarity larger than a preset similarity threshold exist in the preset data feature set, determining to generate alternative multimedia data corresponding to the target multimedia data.

In one embodiment, the generating the plurality of candidate multimedia data corresponding to the target multimedia data includes:

acquiring a plurality of gear parameters according to the data characteristics of the target multimedia data;

adjusting the target multimedia data to obtain multimedia data with parameters of each gear parameter;

and taking the obtained multiple pieces of multimedia data as multiple pieces of alternative multimedia data corresponding to the target multimedia data.

adding one or more auxiliary information in the target multimedia data to update the target multimedia data to obtain one or more updated target multimedia data;

and taking the target multimedia data and the one or more updated target multimedia data as a plurality of alternative multimedia data corresponding to the target multimedia data.

In one embodiment, the object information includes one or more of the following: object behavior information, object emotion information;

the searching the candidate multimedia data matched with the object information in the plurality of candidate multimedia data corresponding to the target multimedia data comprises the following steps:

searching the candidate multimedia data matched with the object behavior information and/or the object emotion information in a plurality of candidate multimedia data corresponding to the target multimedia data.

In another aspect, an embodiment of the present application provides another method for processing multimedia data, where the method includes:

collecting object information and sending the object information to a server, wherein the object information is used for searching candidate multimedia data matched with the object information in a plurality of candidate multimedia data corresponding to target multimedia data by the server, the plurality of candidate multimedia data are obtained by processing the target multimedia data based on the data characteristics of the target multimedia data, and the data content of each candidate multimedia data is matched with the data content of the target multimedia data;

Receiving the searched alternative multimedia data sent by the server;

and outputting the searched alternative multimedia data.

In one embodiment, further comprising: obtaining object prediction information, wherein the object prediction information is used for indicating: whether the object has the requirement of repeated playing for the target multimedia data or not;

if the object estimated information indicates that the object has the requirement of repeated playing on the target multimedia data, storing the searched alternative multimedia data into a memory;

when receiving a playing request of the target multimedia data, acquiring the searched alternative multimedia data from the memory;

and outputting the searched alternative multimedia data.

In another aspect, an embodiment of the present application provides a processing apparatus for multimedia data, including:

the processing unit is used for determining a plurality of alternative multimedia data corresponding to target multimedia data, wherein the plurality of alternative multimedia data are obtained by processing the target multimedia data based on the data characteristics of the target multimedia data, and the data content of each alternative multimedia data is matched with the data content of the target multimedia data; acquiring object information of a target client; searching the candidate multimedia data matched with the object information in a plurality of candidate multimedia data corresponding to the target multimedia data;

And the output unit is used for sending the searched alternative multimedia data to the target client, and the target client is used for outputting the searched alternative multimedia data.

In another aspect, an embodiment of the present application provides another multimedia data processing apparatus, where the multimedia data processing apparatus includes:

the input unit is used for collecting object information;

the output unit is used for sending the object information to a server, wherein the object information is used for searching candidate multimedia data matched with the object information in a plurality of candidate multimedia data corresponding to target multimedia data by the server, the plurality of candidate multimedia data are obtained by processing the target multimedia data based on the data characteristics of the target multimedia data, and the data content of each candidate multimedia data is matched with the data content of the target multimedia data;

the input unit is also used for receiving the searched alternative multimedia data sent by the server;

the output unit is further configured to output the searched candidate multimedia data.

In another aspect, an embodiment of the present application provides a server, including a processor, a storage device, and a communication interface, where the processor, the storage device, and the communication interface are connected to each other, where the storage device is configured to store a computer program supporting a terminal to execute the method, the computer program includes program instructions, and the processor is configured to invoke the program instructions to perform the following steps:

acquiring object information of a target client;

In another aspect, an embodiment of the present application provides a client, including a processor, a storage device, and a communication interface, where the processor, the storage device, and the communication interface are connected to each other, where the storage device is configured to store a computer program supporting a terminal to execute the method, the computer program includes program instructions, and the processor is configured to invoke the program instructions to perform the following steps:

Receiving the searched alternative multimedia data sent by the server;

and outputting the searched alternative multimedia data.

In another aspect, embodiments of the present application provide a computer readable storage medium storing a computer program, where the computer program includes program instructions that, when executed by a processor, cause the processor to perform the method for processing multimedia data according to any one of the above aspects.

In another aspect, embodiments of the present application provide a computer product comprising a computer program adapted to be loaded by a processor and to perform the method of processing multimedia data according to any of the above aspects.

In this embodiment, the plurality of candidate multimedia data corresponding to the target multimedia data are obtained by processing the target multimedia data based on the data features of the target multimedia data, for example, assuming that the data features of the target multimedia data indicate that the target multimedia data is a high-heat video or that the data type of the target multimedia data is a tool type, it is indicated that different objects may have different requirements on the target multimedia data, for example, an object that is interested in the target multimedia data may wish to play target multimedia data with higher data quality, an object that is interested in the target multimedia data may wish to play data smoothly, and the data content of the target multimedia data is known, where exemplary data quality may be represented by parameters such as resolution, frame rate, or code rate. Based on the above, under the condition that the data characteristics of the target multimedia data determine that different objects may have different requirements on the target multimedia data, the target multimedia data is processed to obtain a plurality of alternative multimedia data corresponding to the target multimedia data, so that the adaptive alternative multimedia data can be conveniently issued to different clients, and therefore, the multimedia data issued to the clients by the server can be ensured to meet the requirements of the objects of the clients.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

Fig. 1 is a schematic flow chart of a communication system according to an embodiment of the present application;

fig. 2 is a flow chart of a method for processing multimedia data according to an embodiment of the present application;

fig. 3 is a flow chart of another method for processing multimedia data according to an embodiment of the present application;

fig. 4 is a flow chart of another method for processing multimedia data according to an embodiment of the present application;

fig. 5 is a schematic structural diagram of a processing device for multimedia data according to an embodiment of the present application;

fig. 6 is a schematic structural diagram of a server according to an embodiment of the present application;

fig. 7 is a schematic structural diagram of another multimedia data processing apparatus according to an embodiment of the present application;

fig. 8 is a schematic structural diagram of a client according to an embodiment of the present application.

Detailed Description

The following description of the embodiments of the present application will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are only some, but not all, of the embodiments of the present application. All other embodiments, which can be made by one of ordinary skill in the art based on the embodiments herein without making any inventive effort, are intended to be within the scope of the present application.

The method for processing multimedia data provided in the embodiment of the present application may be applied to a communication system shown in fig. 1, where the communication system may include a server 101 and at least one client 102, where the server 101 establishes a communication connection with any client 102 through any communication manner, so that the server 101 performs data interaction with the client 102 through the communication connection. Specifically, the target client may be any client that establishes communication connection with the server 101, the server 101 may determine a plurality of candidate multimedia data corresponding to the target multimedia data, obtain object information of the target client, then the server 101 searches for candidate multimedia data matched with the object information in the plurality of candidate multimedia data corresponding to the target multimedia data, and further the server 101 sends the searched candidate multimedia data to the target client, where the target client is used for outputting the searched candidate multimedia data.

In an exemplary scenario, if the server has a need to push the target multimedia data to the target client, the server may determine a plurality of candidate multimedia data corresponding to the target multimedia data, acquire object information of the target client, search for candidate multimedia data matching the object information in the plurality of candidate multimedia data corresponding to the target multimedia data, and send the searched candidate multimedia data to the target client. The target client can display a playing interface of the alternative multimedia data, and when a playing instruction of the alternative multimedia data is acquired, the target client can output the searched alternative multimedia data.

In another exemplary scenario, the target client may display a playing interface of the target multimedia data, and if the target client obtains a playing instruction of the object for the target multimedia data, the target client may generate a download request for the target multimedia data and send the download request to the server. The server may determine a plurality of candidate multimedia data corresponding to the target multimedia data in response to the download request, acquire object information of the target client, search candidate multimedia data matching the object information in the plurality of candidate multimedia data corresponding to the target multimedia data, and send the searched candidate multimedia data to the target client. The target client can output the searched alternative multimedia data.

The playing instruction for the candidate multimedia data (or the target multimedia data) may be generated when the display duration of the target client reaches the preset duration threshold value, alternatively, may be generated by the target client in response to the playing operation of the object on the candidate multimedia data (or the target multimedia data), and the playing operation may be, for example, the object clicking a playing control in the playing interface, or the object inputting a voice (such as "playing start") for indicating playing, and is not specifically limited by the embodiments of the present application.

For ease of understanding, the following description will explain and illustrate the relevant terms involved in the embodiments of the present application.

Target multimedia data: the target multimedia data refers to certain multimedia data or any multimedia data, that is, multimedia data which needs to be sent to the client by the server, and may be multimedia data which is pushed to the client by the server, or multimedia data which is sent by the client by the server. And multimedia data refers to data presented in the form of graphics, images, sounds, texts, animations or videos, etc. For example, the server is a video server, the client that establishes a communication connection with the server is a video playing client, and the target multimedia data may refer to video data uploaded to the server by the publisher, that is, video data to be delivered to the client by the server. For another example, the server is an audio server, the client establishing communication connection with the server is an audio playing client, and the target multimedia data may refer to audio data uploaded to the server by the publisher, that is, audio data to be sent to the client by the server.

Alternative multimedia data: the alternative multimedia data refers to multimedia data matching the data content of the target multimedia data.

In one example, there is a difference in data quality of the different alternative multimedia data, where the data quality may be embodied by parameters such as resolution, frame rate, code rate, or sharpness. For example, the server may adjust the resolution of the target multimedia data to obtain a plurality of resolution range candidates, where, by way of example, the resolution of the target multimedia data uploaded to the server by the publisher is 480P (letter P represents progressive scanning, numeral 480 represents vertical resolution, that is, a scanning line with 480 horizontal lines in the vertical direction), and the server may adjust the resolution of the target multimedia data to obtain four resolution range candidates, that is, candidate multimedia data corresponding to the target multimedia data, with resolutions of 1080p,720P,480P, and 270P, respectively, where the four resolution range candidates differ only in resolution, and the data content of the four candidate multimedia data is the same as the data content of the target multimedia data.

In another example, the plurality of candidate multimedia data includes multimedia data to which auxiliary information is added in the target multimedia data and the target multimedia data. The auxiliary information refers to information for auxiliary description of data contents of the target multimedia data, such as special effects, comments, props, etc. For example, the server may determine key unit data included in the target multimedia data, and add one or more auxiliary information to the key unit data to obtain one or more alternative multimedia data. The information amount of the data content of the key unit data may be higher than the information amount of the data content of other unit data in the target multimedia data, for example, the target multimedia data is tool multimedia data, then the key unit data may refer to unit data expressing the central idea of the data content of the target multimedia data, for example, the target multimedia data is entertainment multimedia data, then the key unit data may refer to unit data indicating the climax part of the data content of the target multimedia data, for example, the target multimedia data is athletic multimedia data, and then the key unit data may refer to unit data at a highlight moment included in the target multimedia data.

Object information: the object information refers to information for predicting an object requirement, and the object requirement refers to a requirement that an object views or listens to target multimedia data, for example, the object wants to know the data content of the target multimedia data on the premise that the multimedia data is smoothly played, and for example, the data quality of the multimedia data which the object wants to play is higher.

Specifically, the object information may include one or more of the following: object behavioral characteristics or object mood information. The object behavior features may be used to characterize the behavior of an object when viewing or listening to multimedia data, and may include one or more of the following: the data type of the multimedia data focused by the object, the data type of the multimedia data repeatedly watched by the object, such as entertainment class, athletic class or tool class, and the like, whether the object performs fast forward operation or double-speed playing operation, and the retention rate/amount, praying rate/amount, comment rate/amount, attention rate/amount, collection rate/amount, gifts rate/amount or sharing rate/amount of the object for different types of multimedia data. The object emotion information may be used to characterize an emotion of the object when viewing or listening to the multimedia data, and may include one or more of the following: the method comprises the steps that when a user watches or listens to different types of multimedia data, the facial expression or sign parameters of the user, the current facial expression or sign parameters of the user can be understood to be the facial expression or sign parameters of the user collected in the latest time period of the user, for example, when the user detects a playing instruction initiated by the user to the target multimedia data, the user collects the facial expression or sign parameters of the user, for example, before the server pushes the target multimedia data to the user, the user sends an object emotion information obtaining request to the user, and the user responds to the object emotion information obtaining request to collect the facial expression or sign parameters of the user. The sign parameters may include one or more of the following: heart rate, pulse, blood pressure, body temperature or respiratory rate, etc.

In the specific embodiments of the present application, the related objects may refer to users, related data about the users, such as object information, etc., when the embodiments of the present application are applied to specific products or technologies, user permission or consent needs to be obtained, and the collection, use and processing of related data needs to comply with local laws and regulations and standards.

It may be understood that, in the embodiment of the present application, the processing scheme of the multimedia data mentioned in the embodiment of the present application is described by taking the relevant scene in which the pulse multimedia data is video data as an example, and the embodiment of the present application does not limit the embodiment of the present application, and the processing scheme of the multimedia data mentioned in the embodiment of the present application may also be applied to scenes in which other multimedia data is played, such as audio data or text data, etc., which is not limited in this embodiment of the present application.

Referring to fig. 2 in conjunction with the communication system shown in fig. 1, fig. 2 is a flow chart of a method for processing multimedia data according to an embodiment of the present application; the processing scheme of the multimedia data as shown in fig. 2 includes, but is not limited to, steps S201 to S206, wherein:

s201, the server determines a plurality of alternative multimedia data corresponding to the target multimedia data.

Specifically, the server may process the target multimedia data based on the data characteristics of the target multimedia data to obtain a plurality of candidate multimedia data corresponding to the target multimedia data, where the data content of each candidate multimedia data matches the data content of the target multimedia data, and the server stores the plurality of candidate multimedia data corresponding to the target multimedia data in the memory. The server needs to push the target multimedia data to the target client, or when the server receives a download request from the target client, the server may obtain a plurality of candidate multimedia data corresponding to the target multimedia data from the memory.

Wherein the data characteristics refer to attribute information of the target multimedia data, such as a data type of the target multimedia data, or whether the target multimedia data is high-heat data, or the like. The high heat data refers to data having a heat higher than a preset threshold, and the high heat data refers to high heat video assuming that the target multimedia data is video data.

In one embodiment, the server may acquire the data characteristics of the target multimedia data, determine whether to generate the alternative multimedia data corresponding to the target multimedia data according to the data characteristics of the target multimedia data, and if so, generate a plurality of alternative multimedia data corresponding to the target multimedia data; if not, the server may send the target multimedia data to the target client, or the server may adjust the target multimedia data, generate multimedia data with parameters of a preset gear parameter, and send the multimedia data with parameters of the preset gear parameter to the target client, where the target client refers to a certain client or any client, for example, a client where the server is to push the target multimedia data, or a client where the server receives a download request for the target multimedia data. The preset gear parameter may refer to the lowest gear parameter among the plurality of gear parameters, and the preset gear parameter may be the resolution 270P assuming that the plurality of gear parameters refer to four resolution gear parameters of resolutions 1080P,720P,480P, and 270P.

In one embodiment, the server may compare the data features of the target multimedia data with preset data features in a preset data feature set, where different objects have different requirements for the multimedia data with the preset data features, and if the preset data features in the preset data feature set have preset data features with a similarity greater than a preset similarity threshold, determine to generate alternative multimedia data corresponding to the target multimedia data.

It has been shown that for multimedia data indicated as high-heat data, or multimedia data of which the data type is of a type that can be repeatedly watched, such as tools (e.g. tutorials), games or entertainment, different objects may have different demands on the multimedia data, for example, an object that is interested in the multimedia data may wish to play multimedia data with higher data quality, and an object that is less interested in the multimedia data may wish to know the data content of the multimedia data on the premise that the multimedia data is played smoothly. Based on this, after the server acquires the data characteristics of the target multimedia data, if the data characteristics indicate that the target multimedia data is high heat data, or the data type of the target multimedia data is tools, games or entertainment, the server may generate alternative multimedia data corresponding to the target multimedia data. For example, the preset data features in the set of preset data features may include one or more of the following: the high-heat data are in tool type, in athletic type and in entertainment type.

In one embodiment, the manner in which the server generates the plurality of candidate multimedia data corresponding to the target multimedia data may include two types of:

1. according to the data characteristics of the target multimedia data, a plurality of gear parameters are obtained, the target multimedia data are adjusted, the multimedia data with the parameters being the gear parameters are obtained respectively, and the obtained plurality of multimedia data are used as a plurality of alternative multimedia data corresponding to the target multimedia data. For a specific description of the manner of generating a plurality of candidate multimedia data corresponding to such target multimedia data, reference may be made to the embodiment shown in fig. 3 described below.

2. Adding one or more auxiliary information into the target multimedia data to update the target multimedia data to obtain one or more updated target multimedia data, and taking the target multimedia data and the one or more updated target multimedia data as a plurality of alternative multimedia data corresponding to the target multimedia data. For a specific description of the manner of generating a plurality of alternative multimedia data corresponding to such target multimedia data, reference may be made to the embodiment shown in fig. 4 described below.

S202, the target client acquires object information of the target client.

In a specific implementation, if the object information includes object behavior information, the target client may acquire the object behavior information of the object in the historical period through an input device of a terminal device running the target client, where the input device may include a touch panel, a microphone, or a key, for example. If the object information includes object emotion information, the target client may acquire an object image by running an image acquisition device of a terminal device of the target client, then perform expression recognition processing on the object image to obtain a facial expression of the object, and then generate object emotion information based on the facial expression, or may acquire physical sign parameters of the object by a wearable device worn by the object, and the wearable device sends the acquired physical sign parameters to the target client, and the target client generates the object emotion information based on the physical sign parameters.

In one example, if the server has a need to push target multimedia data to a target client, the server may send an object information acquisition request to the target client, and the target client may acquire object information in response to the object information acquisition request.

In another example, when the target client detects an object-initiated play instruction for the target multimedia data, the target client may acquire the object information, and then the target client may send a download request for the target multimedia data to the server, where the download request may carry the object information.

S203, the target client sends the object information of the target client to the server.

S204, the server searches the candidate multimedia data matched with the object information of the target client side in the plurality of candidate multimedia data corresponding to the target multimedia data.

Specifically, the server may search for candidate multimedia data matching the object behavior information and/or the object emotion information among a plurality of candidate multimedia data corresponding to the target multimedia data.

In one example, if the server determines, based on the object behavior information, that the object does not perform operations such as fast forward or double-speed play when viewing the multimedia data in a historical period, indicating that the object is used to viewing the multimedia data slowly, the candidate multimedia data that the server finds matches the object behavior information may be candidate multimedia data of the target gear parameter.

In another example, if the server determines that the object views multimedia data of the same data type as the target multimedia data for a history period based on the object emotion information, the facial expression is happy, indicating that the object likes to view the multimedia data of this type, i.e., the object is likely to like to view the target multimedia data, the candidate multimedia data that the server finds to match the object emotion information may be the candidate multimedia data of the target gear parameter. Or if the server determines that the current facial expression of the object is happy based on the emotion information of the object, which indicates that the object is likely to like to watch the target multimedia data, the candidate multimedia data which is searched by the server and matched with the emotion information of the object can be the candidate multimedia data of the target gear parameter. The target gear parameter is higher than the gear parameters of other alternative multimedia data in the plurality of alternative multimedia data corresponding to the target multimedia data. For example, the plurality of candidate multimedia data corresponding to the target multimedia data refer to multimedia data having resolutions of 1080P,720P,480P and 270P, respectively, and then the candidate multimedia data of the target gear parameter may refer to multimedia data having a resolution of 1080P.

In still another example, if the server determines, based on the object behavior information, that the retention rate/amount, the praise rate/amount, the comment rate/amount, the attention rate/amount, the collection rate/amount, the gift rate/amount, or the sharing rate/amount of the object is high when the object views the multimedia data, indicating that the frequency of man-machine interaction of the object is high when the object views the multimedia data, i.e., the object prefers to interact when the object views the multimedia data, the candidate multimedia data matching the object behavior information found by the server may be candidate multimedia data to which the auxiliary information is added, for example, candidate multimedia data obtained by adding a special effect to the unit data at a highlight time in the target multimedia data. Alternatively, if the candidate multimedia data corresponding to the target multimedia data includes a plurality of candidate multimedia data to which the auxiliary information is added, the server may select the candidate multimedia data, such as the higher the retention rate/amount, the praise rate/amount, the comment rate/amount, the attention rate/amount, the collection rate/amount, the gift rate/amount, or the sharing rate/amount, among the plurality of candidate multimedia data to which the auxiliary information is added, the more the auxiliary information is added in the selected candidate multimedia data, such as the retention rate/amount, the praise rate/amount, the comment rate/amount, the attention rate/amount, the collection rate/amount, the gift rate/amount, or the sharing rate/amount.

In one embodiment, the server may pre-establish a correspondence between the object information and the gear parameters, and after obtaining the object information of the target client, the server may obtain a similarity between the object information of the target client and the object information corresponding to each gear parameter, and use the candidate multimedia data with the gear parameter having the highest similarity as the candidate multimedia data matched with the object information of the target client. Alternatively, the server may pre-establish a correspondence between the object information and the auxiliary information, and after the server obtains the object information of the target client, the server may obtain the similarity between the object information of the target client and the object information corresponding to each auxiliary information, and use the candidate multimedia data added with the auxiliary information with the highest similarity as the candidate multimedia data matched with the object information of the target client.

In another embodiment, the server may pre-construct a multimedia data matching model, and then train the multimedia data matching model with historical object information of the object of the target client and gear parameters of the multimedia data watched by the object as samples when the object watches the multimedia data in the historical time period, so as to obtain a trained multimedia data matching model, so that candidate multimedia data obtained by matching the trained multimedia data matching model meets the requirements of the object of the target client. After the server obtains the object information of the target client, the trained multimedia data matching model can be called to match the object information of the target client with a plurality of gear parameters to obtain the gear parameters matched with the object information of the target client, and the candidate multimedia data of the gear parameters obtained by the trained multimedia data matching model is used as the candidate multimedia data matched with the object information of the target client.

Optionally, the server may pre-construct a multimedia data matching model, and then train the multimedia data matching model with historical object information of the object of the target client and auxiliary information in the multimedia data watched by the object as samples when the object watches the multimedia data in the historical time period, so as to obtain a trained multimedia data matching model, so that candidate multimedia data obtained by matching the trained multimedia data matching model meets the requirement of the object of the target client. After the server obtains the object information of the target client, the trained multimedia data matching model can be called to match the object information of the target client with a plurality of auxiliary information, the type of the auxiliary information matched with the object information of the target client is obtained, and the candidate multimedia data added with the auxiliary information of the type obtained by the trained multimedia data matching model is used as the candidate multimedia data matched with the object information of the target client.

S205, the server sends the searched alternative multimedia data to the target client.

S206, the target client outputs the searched alternative multimedia data.

In one embodiment, after the target client outputs the candidate multimedia data, the target client may obtain the object prediction information, where the object prediction information is used to indicate: if the object has a requirement of repeated playing for the target multimedia data, if the object estimated information indicates that the object has the requirement of repeated playing for the target multimedia data, the searched alternative multimedia data is stored in a memory. Then, when receiving a play request for the target multimedia data, the target client can acquire the searched alternative multimedia data from the memory and output the searched alternative multimedia data.

In this embodiment, if the target client estimates that the object will repeatedly watch the candidate multimedia data, the searched candidate multimedia data is cached, and when the object initiates the replay operation, the target client does not need to download the candidate multimedia data from the server again, but obtains the candidate multimedia data from the memory, and outputs the candidate multimedia data, so that bandwidth resources can be saved.

In another embodiment, if the object prediction information indicates that the object has a requirement of playing back the target multimedia data repeatedly, and the candidate multimedia data found by the server is not the candidate multimedia data with the highest data quality, the target client may send a data downloading request to the server, where the data downloading request is used to request to download the candidate multimedia data with the highest data quality, and after the server responds to the data downloading request to send the candidate multimedia data with the highest data quality to the target client, the target client may store the candidate multimedia data with the highest data quality in the memory. Then, the target client may acquire the alternative multimedia data from the memory and output the alternative multimedia data when receiving a play request for the target multimedia data. The candidate multimedia data with the highest data quality may refer to the candidate multimedia data with the largest gear parameter, or the candidate multimedia data with the largest amount of added auxiliary information, etc. The target client can send a data downloading request to the server after the downloading of the candidate multimedia data searched by the server is completed, so that the problem that the candidate multimedia data searched by the server and output by the target client currently has playing clamping can be avoided, and the candidate multimedia data searched by the server can be played more smoothly.

In this embodiment, when the target client initiates the replay operation, the target client may obtain the candidate multimedia data from the memory and output the candidate multimedia data, so as to smoothly play the multimedia data and ensure that the quality of the played multimedia data is higher, thereby improving the user viscosity.

The method for the target client to obtain the object pre-estimation information may include: the target client side determines that the object has the behavior of repeatedly watching the multimedia data in the historical time period based on the object behavior information, the target client side can acquire the data type of the multimedia data repeatedly watched by the object, if the data type of the target multimedia data is the same as the data type of the multimedia data repeatedly watched by the object, the requirement of the object on repeated playing of the target multimedia data is indicated, and the searched alternative multimedia data is stored in a memory; if the data type of the target multimedia data is different from the data type of the multimedia data repeatedly watched by the object, the object is indicated to have no requirement of repeated playing on the target multimedia data, and based on the requirement, the server does not need to store the searched alternative multimedia data into a memory.

In this embodiment of the present application, the server may determine a plurality of candidate multimedia data corresponding to the target multimedia data, where the plurality of candidate multimedia data is obtained by processing the target multimedia data based on data features of the target multimedia data, and data content of each candidate multimedia data is matched with data content of the target multimedia data; acquiring object information of a target client; searching the candidate multimedia data matched with the object information in a plurality of candidate multimedia data corresponding to the target multimedia data; and sending the searched alternative multimedia data to the target client so that the target client outputs the searched alternative multimedia data, and the multimedia data sent to the client by the server can be ensured to meet the requirement of the object of the client.

Referring to fig. 3 in conjunction with the communication system shown in fig. 1, fig. 3 is a flow chart of another method for processing multimedia data according to an embodiment of the present application; the processing scheme of the multimedia data as shown in fig. 3 includes, but is not limited to, steps S301 to S309, wherein:

s301, acquiring data characteristics of target multimedia data.

S302, according to the data characteristics of the target multimedia data, judging whether to generate the alternative multimedia data corresponding to the target multimedia data.

And S303, if yes, acquiring a plurality of gear parameters according to the data characteristics of the target multimedia data.

S304, adjusting the target multimedia data to obtain multimedia data with parameters of each gear parameter.

And S305, taking the obtained multiple pieces of multimedia data as multiple pieces of alternative multimedia data corresponding to the target multimedia data.

For example, if the data characteristics of the target multimedia data indicate that the target multimedia data is high heat data, or the data type of the target multimedia data is a tool class, an entertainment class, or an athletic class, the server may determine to generate alternative multimedia data corresponding to the target multimedia data, and then the server may transcode the target multimedia data to obtain the alternative multimedia data of the plurality of gear parameters. For example, if the target multimedia data is high-heat data, the server may acquire four gear parameters, which are respectively the resolutions 1080P,720P,480P and 270P, and then the server may transcode the target multimedia data to obtain multimedia data with the resolutions 1080P,720P,480P and 270P, respectively, and take the multimedia data with the resolutions 1080P,720P,480P and 270P as a plurality of candidate multimedia data corresponding to the target multimedia data.

S306, obtaining object information of the target client.

S307, searching the candidate multimedia data matched with the object information of the target client from the plurality of candidate multimedia data corresponding to the target multimedia data.

In one example, if the object information includes object behavior information, such as the server determines that the object is used to watch the video slowly based on the object behavior information, and does not perform operations such as fast forward or double-speed playing, the candidate multimedia data that the server finds and matches the object information may be candidate multimedia data of a higher gear parameter; otherwise, the candidate multimedia data matched with the object information searched by the server may be the candidate multimedia data of the lower gear parameter.

In another example, if the object information includes object emotion information, such as multimedia data in which the server determines that the object preference data type is the data type of the target multimedia data based on the object emotion information, the candidate multimedia data that the server finds to match the object information may be candidate multimedia data of a higher gear parameter; otherwise, the candidate multimedia data matched with the object information searched by the server may be the candidate multimedia data of the lower gear parameter. For example, the plurality of candidate multimedia data corresponding to the target multimedia data refers to the multimedia data having the resolutions of 1080P,720P,480P and 270P, and then the candidate multimedia data of the higher gear parameter may refer to the multimedia data having the resolution of 1080P or the multimedia data having the resolution of 720P, and the candidate multimedia data of the lower gear parameter may refer to the multimedia data having the resolution of 480P or the multimedia data having the resolution of 270P.

And S308, the searched alternative multimedia data is sent to a target client, and the target client is used for outputting the searched alternative multimedia data.

And S309, if not, sending the target multimedia data to a target client, wherein the target client is used for outputting the target multimedia data.

For example, if the data characteristic of the target multimedia data indicates that the target multimedia data is not high heat data, or the data type of the target multimedia data is not a tool class, an entertainment class, or an athletic class, the server may send the target multimedia data to the target client so that the target client outputs the target multimedia data.

Optionally, if the candidate multimedia data corresponding to the target multimedia data is not generated according to the data characteristics of the target multimedia data, the server may transcode only the multimedia data with the preset gear parameter for the target multimedia data, and then the server sends the multimedia data with the preset gear parameter to the target client, so that the target client outputs the multimedia data. Where the preset gear parameter refers to a gear parameter of a lower gear, for example, the gear parameter may include resolutions 720P,480P, and 270P, then the preset gear parameter may refer to a resolution 270P or 480P.

In the embodiment of the application, the data characteristics of target multimedia data are acquired, whether to generate alternative multimedia data corresponding to the target multimedia data is judged according to the data characteristics of the target multimedia data, if yes, a plurality of gear parameters are acquired according to the data characteristics of the target multimedia data, the target multimedia data are adjusted to obtain multimedia data with the parameters being each gear parameter, the acquired plurality of multimedia data are used as a plurality of alternative multimedia data corresponding to the target multimedia data, object information of a target client is acquired, the alternative multimedia data matched with the object information is searched in the plurality of alternative multimedia data corresponding to the target multimedia data, and the searched alternative multimedia data are sent to the target client; if not, the target multimedia data is sent to the target client, the transcoding gear of the target multimedia data can be determined according to the data characteristics, more gears are transcoded for the multimedia data with different requirements, and the alternative multimedia data matched with the object information is issued, so that the multimedia data issued to the client by the server is ensured to meet the requirements of the object of the client.

Referring to fig. 4 in conjunction with the communication system shown in fig. 1, fig. 4 is a flow chart of another method for processing multimedia data according to an embodiment of the present application; the processing scheme of the multimedia data as shown in fig. 4 includes, but is not limited to, steps S401 to S408, wherein:

s401, acquiring data characteristics of target multimedia data.

S402, judging whether to generate alternative multimedia data corresponding to the target multimedia data according to the data characteristics of the target multimedia data.

And S403, if yes, adding one or more pieces of auxiliary information into the target multimedia data to update the target multimedia data, thereby obtaining one or more pieces of updated target multimedia data.

And S404, taking the target multimedia data and one or more updated target multimedia data as a plurality of alternative multimedia data corresponding to the target multimedia data.

For example, if the data characteristics of the target multimedia data indicate that the target multimedia data is high heat data, or the data type of the target multimedia data is a tool class, an entertainment class, or an athletic class, the server may determine to generate alternative multimedia data corresponding to the target multimedia data, and then the server may add one or more auxiliary information to the target multimedia data to obtain a plurality of alternative multimedia data. For example, if the target multimedia data is high-heat data, the server may add an effect to the unit data at the high-light time in the target multimedia data, and use the target multimedia data and the target multimedia data to which the effect is added as a plurality of candidate multimedia data corresponding to the target multimedia data.

S405, obtaining object information of the target client.

And S406, searching the candidate multimedia data matched with the object information of the target client from a plurality of candidate multimedia data corresponding to the target multimedia data.

In one example, if the object information includes object behavior information, such as the server determines that the object prefers to interact based on the object behavior information, the candidate multimedia data that the server finds to match the object information may be target multimedia data to which the special effect is added; otherwise, the candidate multimedia data that the server finds to match the object information may be the target multimedia data.

In another example, if the object information includes object emotion information, such as multimedia data in which the server determines that the object preference data type is the data type of the target multimedia data based on the object emotion information, the candidate multimedia data that the server finds to match the object information may be the target multimedia data to which the special effect is added; otherwise, the candidate multimedia data that the server finds to match the object information may be the target multimedia data.

S407, the searched alternative multimedia data is sent to a target client, and the target client is used for outputting the searched alternative multimedia data.

And S408, if not, sending the target multimedia data to a target client, wherein the target client is used for outputting the target multimedia data.

Optionally, if it is determined that the candidate multimedia data corresponding to the target multimedia data is not generated according to the data characteristics of the target multimedia data, the server may transcode the target multimedia data into the multimedia data with the preset gear parameter, and then the server sends the multimedia data with the preset gear parameter to the target client, so that the target client outputs the multimedia data. Where the preset gear parameter refers to a gear parameter of a lower gear, for example, the gear parameter may include resolutions 720P,480P, and 270P, then the preset gear parameter may refer to a resolution 270P or 480P.

In the embodiment of the application, the data characteristics of target multimedia data are acquired, whether to generate alternative multimedia data corresponding to the target multimedia data is judged according to the data characteristics of the target multimedia data, if yes, one or more pieces of auxiliary information are added in the target multimedia data to update the target multimedia data, one or more pieces of updated target multimedia data are obtained, the target multimedia data and the one or more pieces of updated target multimedia data serve as a plurality of pieces of alternative multimedia data corresponding to the target multimedia data, object information of a target client is acquired, the alternative multimedia data matched with the object information is searched in the plurality of pieces of alternative multimedia data corresponding to the target multimedia data, and the searched alternative multimedia data is sent to the target client; if not, the target multimedia data is sent to the target client, auxiliary information added to the target multimedia data can be determined according to the data characteristics, one or more auxiliary information is added to the multimedia data with different requirements, and alternative multimedia data matched with the object information is issued, so that the multimedia data issued to the client by the server is ensured to meet the requirements of the object of the client.

The present embodiment also provides a computer storage medium having stored therein program instructions for implementing the corresponding method described in the above embodiments when executed.

Referring to fig. 5 again, fig. 5 is a schematic structural diagram of a processing device for multimedia data according to an embodiment of the present application.

In one implementation manner of the apparatus of the embodiment of the present application, the apparatus includes the following structure.

A processing unit 501, configured to determine a plurality of candidate multimedia data corresponding to a target multimedia data, where the plurality of candidate multimedia data are obtained by processing the target multimedia data based on data features of the target multimedia data, and data content of each candidate multimedia data is matched with data content of the target multimedia data; acquiring object information of a target client; searching the candidate multimedia data matched with the object information in a plurality of candidate multimedia data corresponding to the target multimedia data;

and the output unit 502 is configured to send the searched candidate multimedia data to the target client, where the target client is configured to output the searched candidate multimedia data.

In one embodiment, the processing unit 501 is further configured to: acquiring data characteristics of the target multimedia data; judging whether to generate alternative multimedia data corresponding to the target multimedia data according to the data characteristics of the target multimedia data; if yes, generating a plurality of alternative multimedia data corresponding to the target multimedia data.

In one embodiment, the processing unit 501 is configured to, when determining whether to generate the candidate multimedia data corresponding to the target multimedia data according to the data characteristics of the target multimedia data:

In one embodiment, the processing unit 501 is configured, when generating a plurality of candidate multimedia data corresponding to the target multimedia data, to:

the processing unit 501 is configured to, when searching for candidate multimedia data matching the object information in a plurality of candidate multimedia data corresponding to the target multimedia data: searching the candidate multimedia data matched with the object behavior information and/or the object emotion information in a plurality of candidate multimedia data corresponding to the target multimedia data.

In this embodiment of the present application, the processing unit 501 may determine a plurality of candidate multimedia data corresponding to the target multimedia data, where the plurality of candidate multimedia data is obtained by processing the target multimedia data based on data features of the target multimedia data, and data content of each candidate multimedia data is matched with data content of the target multimedia data; the processing unit 501 acquires object information of a target client; the processing unit 501 searches for candidate multimedia data matched with the object information from a plurality of candidate multimedia data corresponding to the target multimedia data; the output unit 502 sends the searched candidate multimedia data to the target client, so that the target client outputs the searched candidate multimedia data, and the multimedia data sent to the client by the server can be ensured to meet the requirement of the object of the client.

Referring to fig. 6 again, fig. 6 is a schematic structural diagram of a server according to an embodiment of the present application, where the server according to an embodiment of the present application includes a power supply module and other structures, and includes a processor 601, a storage device 602, and a communication interface 603. The processor 601, the storage device 602 and the communication interface 603 can interact data, and the processor 601 realizes a corresponding processing method of the multimedia data.

The storage 602 may include volatile memory (RAM), such as random-access memory (RAM); the storage device 602 may also include a non-volatile memory (non-volatile memory), such as a flash memory (flash memory), a Solid State Drive (SSD), etc.; the storage 602 may also include a combination of the types of memory described above.

The processor 601 may be a central processing unit (central processing unit, CPU). The processor 601 may also be a combination of a CPU and a GPU. In the server, a plurality of CPUs and GPUs can be included as required to perform corresponding data processing. In one embodiment, the storage 602 is used to store program instructions. The processor 601 may invoke program instructions to implement the various methods as referred to above in embodiments of the present application.

In a first possible implementation manner, the processor 601 of the server invokes the program instructions stored in the storage device 602, to determine a plurality of candidate multimedia data corresponding to the target multimedia data, where the plurality of candidate multimedia data is obtained by processing the target multimedia data based on the data characteristics of the target multimedia data, and the data content of each candidate multimedia data matches the data content of the target multimedia data; acquiring object information of a target client; searching the candidate multimedia data matched with the object information in a plurality of candidate multimedia data corresponding to the target multimedia data; the searched candidate multimedia data is sent to the target client through the communication interface 603, and the target client is used for outputting the searched candidate multimedia data.

In one embodiment, the processor 601 is further configured to: acquiring data characteristics of the target multimedia data; judging whether to generate alternative multimedia data corresponding to the target multimedia data according to the data characteristics of the target multimedia data; if yes, generating a plurality of alternative multimedia data corresponding to the target multimedia data.

In one embodiment, the processor 601 is configured to, when determining whether to generate the candidate multimedia data corresponding to the target multimedia data according to the data characteristics of the target multimedia data:

In one embodiment, the processor 601 is configured, when generating a plurality of candidate multimedia data corresponding to the target multimedia data, to:

the processor 601 is configured to, when searching for candidate multimedia data matching the object information in a plurality of candidate multimedia data corresponding to the target multimedia data: searching the candidate multimedia data matched with the object behavior information and/or the object emotion information in a plurality of candidate multimedia data corresponding to the target multimedia data.

In this embodiment of the present application, the processor 601 may determine a plurality of candidate multimedia data corresponding to the target multimedia data, where the plurality of candidate multimedia data is obtained by processing the target multimedia data based on data features of the target multimedia data, and data content of each candidate multimedia data is matched with data content of the target multimedia data; acquiring object information of a target client; searching the candidate multimedia data matched with the object information in a plurality of candidate multimedia data corresponding to the target multimedia data; the searched candidate multimedia data is sent to the target client through the communication interface 603, so that the target client outputs the searched candidate multimedia data, and the multimedia data sent to the client by the server can be ensured to meet the requirement of the object of the client.

Referring to fig. 7 again, fig. 7 is a schematic structural diagram of another multimedia data processing apparatus according to an embodiment of the present application.

In one implementation of the apparatus of the embodiments of the present application, the apparatus includes the following structure.

An input unit 701 for collecting object information;

an output unit 702, configured to send the object information to a server, where the object information is used for the server to find candidate multimedia data matched with the object information from multiple candidate multimedia data corresponding to the target multimedia data, where the multiple candidate multimedia data are obtained by processing the target multimedia data based on data features of the target multimedia data, and data content of each candidate multimedia data is matched with data content of the target multimedia data;

The input unit 701 is further configured to receive the searched alternative multimedia data sent by the server;

the output unit 702 is further configured to output the found candidate multimedia data.

In an embodiment, the apparatus of the embodiments of the present application may further include a processing unit 703, where the processing unit 703 is configured to obtain object prediction information, where the object prediction information is used to indicate: whether the object has the requirement of repeated playing for the target multimedia data or not; if the object estimated information indicates that the object has the requirement of repeated playing on the target multimedia data, storing the searched alternative multimedia data into a memory;

when the input unit 701 receives a play request for the target multimedia data, the processing unit 703 obtains the searched candidate multimedia data from the memory;

In the embodiment of the present application, the input unit 701 collects object information; the output unit 702 sends the object information to the server, where the object information is used for the server to find candidate multimedia data matched with the object information from a plurality of candidate multimedia data corresponding to the target multimedia data, where the plurality of candidate multimedia data is obtained by processing the target multimedia data based on the data characteristics of the target multimedia data, and the data content of each candidate multimedia data is matched with the data content of the target multimedia data; the input unit 701 receives the searched alternative multimedia data transmitted by the server; the output unit 702 outputs the searched alternative multimedia data, so that the multimedia data output by the client can be ensured to meet the requirement of the object of the client.

Referring to fig. 8 again, fig. 8 is a schematic structural diagram of a client provided in an embodiment of the present application, where the client in the embodiment of the present application includes a power supply module and other structures, and includes a processor 801, a storage device 802, and a communication interface 803. Data can be interacted among the processor 801, the storage device 802 and the communication interface 803, and a corresponding processing method of the multimedia data is realized by the processor 801.

The storage device 802 may include volatile memory (RAM), such as random-access memory (RAM); the storage device 802 may also include a non-volatile memory (non-volatile memory), such as a flash memory (flash memory), a Solid State Drive (SSD), etc.; the storage device 802 may also include a combination of the types of memory described above.

The processor 801 may be a central processing unit (central processing unit, CPU). The processor 801 may also be a combination of a CPU and a GPU. In the client, a plurality of CPUs and GPUs can be included to perform corresponding data processing according to requirements. In one embodiment, storage 802 is used to store program instructions. The processor 801 may invoke program instructions to implement the various methods as referred to above in embodiments of the present application.

In a first possible implementation manner, the processor 801 of the client invokes the program instructions stored in the storage device 802, to collect object information through the communication interface 803, and send the object information to the server, where the object information is used for the server to find candidate multimedia data matched with the object information from multiple candidate multimedia data corresponding to the target multimedia data, where the multiple candidate multimedia data is obtained by processing the target multimedia data based on the data characteristics of the target multimedia data, and the data content of each candidate multimedia data is matched with the data content of the target multimedia data; receiving the searched alternative multimedia data sent by the server through a communication interface 803; the searched candidate multimedia data is output through the communication interface 803.

In one embodiment, the processor 801 invokes program instructions stored in the storage 802, and is further configured to obtain object prediction information, where the object prediction information is used to indicate: whether the object has the requirement of repeated playing for the target multimedia data or not; if the object estimated information indicates that the object has the requirement of repeated playing on the target multimedia data, storing the searched alternative multimedia data into a memory; when receiving a playing request of the target multimedia data, acquiring the searched alternative multimedia data from the memory; the searched candidate multimedia data is output through the communication interface 803.

In the embodiment of the present application, the processor 801 collects object information; the method comprises the steps that object information is sent to a server, the object information is used for searching candidate multimedia data matched with the object information in a plurality of candidate multimedia data corresponding to target multimedia data, the plurality of candidate multimedia data are obtained by processing the target multimedia data based on data characteristics of the target multimedia data, and data content of each candidate multimedia data is matched with data content of the target multimedia data; receiving the searched alternative multimedia data sent by the server; and outputting the searched alternative multimedia data, so that the multimedia data output by the client can be ensured to meet the requirement of the object of the client.

Those skilled in the art will appreciate that the processes implementing all or part of the methods of the above embodiments may be implemented by a computer program for instructing relevant hardware, and the program may be stored in a computer readable storage medium, and the program may include the processes of the embodiments of the methods as above when executed. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a random-access Memory (Random Access Memory, RAM), or the like. The computer-readable storage medium of (a) may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function, and the like; the storage data area may store data created from the use of blockchain nodes, and the like.

The above disclosure is only a few examples of the present application, and it is not intended to limit the scope of the claims, and those skilled in the art will understand that all or a portion of the above-described embodiments may be implemented and equivalents may be substituted for elements thereof, which are included in the scope of the present invention.

Claims

1. A method for processing multimedia data, comprising:

acquiring object information of a target client;

2. The method according to claim 1, wherein the method further comprises:

Acquiring data characteristics of the target multimedia data;

3. The method according to claim 2, wherein the determining whether to generate the candidate multimedia data corresponding to the target multimedia data according to the data characteristics of the target multimedia data comprises:

4. The method of claim 2, wherein generating the plurality of candidate multimedia data corresponding to the target multimedia data comprises:

5. The method of claim 2, wherein generating the plurality of candidate multimedia data corresponding to the target multimedia data comprises:

6. The method of claim 1, wherein the object information comprises one or more of: object behavior information, object emotion information;

7. A method for processing multimedia data, comprising:

receiving the searched alternative multimedia data sent by the server;

and outputting the searched alternative multimedia data.

8. The method of claim 7, wherein the method further comprises:

obtaining object prediction information, wherein the object prediction information is used for indicating: whether the object has the requirement of repeated playing for the target multimedia data or not;

and outputting the searched alternative multimedia data.

9. A multimedia data processing apparatus, the apparatus comprising:

10. A multimedia data processing apparatus, the apparatus comprising:

the input unit is used for collecting object information;