CN113301427A - Data processing method and device, electronic equipment and storage medium - Google Patents

Data processing method and device, electronic equipment and storage medium Download PDF

Info

Publication number
CN113301427A
CN113301427A CN202011376609.0A CN202011376609A CN113301427A CN 113301427 A CN113301427 A CN 113301427A CN 202011376609 A CN202011376609 A CN 202011376609A CN 113301427 A CN113301427 A CN 113301427A
Authority
CN
China
Prior art keywords
video data
optimization
video
information
data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202011376609.0A
Other languages
Chinese (zh)
Inventor
陈石平
梅大为
陈卫伍
刘杰
彭爽
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Alibaba Group Holding Ltd
Original Assignee
Alibaba Group Holding Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Alibaba Group Holding Ltd filed Critical Alibaba Group Holding Ltd
Priority to CN202011376609.0A priority Critical patent/CN113301427A/en
Publication of CN113301427A publication Critical patent/CN113301427A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs
    • H04N21/23418—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/235—Processing of additional data, e.g. scrambling of additional data or processing content descriptors
    • H04N21/2353—Processing of additional data, e.g. scrambling of additional data or processing content descriptors specifically adapted to content descriptors, e.g. coding, compressing or processing of metadata
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
    • H04N21/266—Channel or content management, e.g. generation and management of keys and entitlement messages in a conditional access system, merging a VOD unicast channel into a multicast channel
    • H04N21/26603—Channel or content management, e.g. generation and management of keys and entitlement messages in a conditional access system, merging a VOD unicast channel into a multicast channel for automatically generating descriptors from content, e.g. when it is not made available by its provider, using content analysis techniques
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
    • H04N21/44008—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/83—Generation or processing of protective or descriptive data associated with content; Content structuring
    • H04N21/84—Generation or processing of descriptive data, e.g. content descriptors

Abstract

The embodiment of the application provides a data processing method, a data processing device, electronic equipment and a storage medium, wherein the data processing method can be applied to a video playing terminal, and comprises the following steps: receiving video transmission information; analyzing the video transmission information to obtain video data and optimized description information; optimizing the video data according to the optimization description information; playing the optimized video data; the optimization effect on the data can be improved.

Description

Data processing method and device, electronic equipment and storage medium
Technical Field
The present application relates to the field of computer technologies, and in particular, to a data processing method, a data processing apparatus, an electronic device, and a storage medium.
Background
The multimedia data refers to data fusing two or more media data, and is man-machine interactive information exchange and transmission data. The processing of multimedia data generally includes preprocessing and post-processing, wherein the post-processing corresponds to preprocessing, and refers to the next step after preprocessing, which is the step performed before the final processing is completed, or the step performed after a certain stage of work.
One existing method for post-processing multimedia data is to send multimedia data (e.g., video data including image data and audio data) to a terminal device, and the terminal device performs parameter analysis, image or audio scene analysis, image or audio semantic analysis, and the like on the multimedia data to determine corresponding optimization parameters and optimization characteristics, and performs optimization adjustment on the image according to the optimization parameters, the optimization characteristics, and the like. If the image can be identified, determining a sharpening adjustment parameter and a contrast adjustment parameter for optimizing the image, and adjusting the sharpening parameter and the contrast parameter of the image data according to the sharpening adjustment parameter and the contrast adjustment parameter. If the image can be subjected to scene recognition, a scene corresponding to the image is recognized, and the scene corresponding to the scene is adopted to perform optimization processing on the image.
The terminal device processes the multimedia data in the manner, and the terminal device needs to analyze and optimize the image, but due to the limitation of the computing power of the terminal device, the terminal device generally cannot perform complex analysis and computation, so the multimedia data is processed in the manner, the multimedia data is generally simply optimized, and the optimization effect on the multimedia data is poor.
Disclosure of Invention
The embodiment of the application provides a data processing method to improve the optimization effect of multimedia data.
Correspondingly, the embodiment of the application also provides a data processing device, an electronic device and a storage medium, which are used for ensuring the realization and the application of the system.
In order to solve the above problem, an embodiment of the present application discloses a data processing method, which is applied to a video playing terminal, and includes: receiving video transmission information; analyzing the video transmission information to obtain video data and optimized description information; optimizing the video data according to the optimization description information; and playing the optimized video data.
In order to solve the above problem, an embodiment of the present application discloses a data processing method, which is applied to a server and includes: analyzing the video data to determine optimization description information; and packaging the optimization description information and the video data to obtain video transmission information, and transmitting the video transmission information to a video playing terminal, wherein the video playing terminal is used for optimizing the video data according to the optimization description information and playing and outputting the optimized video data.
In order to solve the above problem, an embodiment of the present application discloses a data processing method, which is applied to a live video playing terminal, and includes: receiving live video transmission information; analyzing the live video transmission information to obtain live video data and optimized description information; optimizing the live video data according to the optimization description information; and playing the optimized live video data.
In order to solve the above problem, an embodiment of the present application discloses a data processing method, which is applied to a server and includes: acquiring live video data, analyzing the live video data, and determining optimized description information; and packaging the optimization description information and the live video data to obtain live video transmission information, and transmitting the live video transmission information to a live video playing terminal, wherein the live video playing terminal is used for optimizing the live video data according to the optimization description information and playing and outputting the optimized live video data.
In order to solve the above problem, an embodiment of the present application discloses a data processing method, which is applied to a monitoring video playing terminal, and includes: receiving monitoring video transmission information; analyzing the monitoring video transmission information to obtain monitoring video data and optimization description information; optimizing the monitoring video data according to the optimization description information; and playing the optimized monitoring video data.
In order to solve the above problem, an embodiment of the present application discloses a data processing method, which is applied to a server and includes: acquiring monitoring video data, analyzing the monitoring video data and determining optimization description information; and packaging the optimization description information and the monitoring video data to obtain monitoring video transmission information, and transmitting the monitoring video transmission information to a monitoring video playing terminal, wherein the monitoring video playing terminal is used for optimizing the monitoring video data according to the optimization description information and playing and outputting the optimized monitoring video data.
In order to solve the above problem, an embodiment of the present application discloses a data processing method, which is applied to a video playing terminal, and includes: receiving video transmission information; analyzing the video transmission information to obtain video data and corresponding optimization description information, wherein the video data comprises multiple paths of sub-video data; optimizing target sub-video data in the video data according to the optimization description information; and playing the optimized video data.
In order to solve the above problem, an embodiment of the present application discloses a data processing method, which is applied to a server and includes: acquiring multiple paths of sub-video data to determine video data; analyzing the target sub-video data and determining corresponding optimization description information; and packaging the optimization description information and the video data to obtain video transmission information, and transmitting the video transmission information to a video playing terminal, wherein the video playing terminal is used for optimizing target sub-video data in the video data according to the optimization description information and playing and outputting the optimized live video data.
In order to solve the above problem, an embodiment of the present application discloses a data processing method, which is applied to a video playing terminal, and includes: providing an optimization mode selection page, wherein the optimization mode selection page comprises a plurality of optimization selection controls corresponding to different optimization modes; and determining selection information according to the operation instruction of the optimization selection control, and transmitting the selection information to the server so that the server can process the video data in a corresponding optimization mode according to the selection information.
In order to solve the above problem, an embodiment of the present application discloses a data processing method, which is applied to a server and includes: receiving selection information, wherein the selection information is determined according to an optimization mode selection page; determining a target optimization mode aiming at the video data according to the selection information; and correspondingly processing the acquired video data according to the target optimization mode.
In order to solve the above problem, an embodiment of the present application discloses a data processing apparatus, including: the transmission information receiving module is used for receiving video transmission information; the transmission information analysis module is used for analyzing the video transmission information to obtain video data and optimized description information; the video data optimization module is used for optimizing the video data according to the optimization description information; and the video data playing module is used for playing the optimized video data.
In order to solve the above problem, an embodiment of the present application discloses a data processing apparatus, including: the video data analysis module is used for analyzing the video data and determining optimization description information; and the transmission information packaging module is used for packaging the optimization description information and the video data to obtain video transmission information and transmitting the video transmission information to the terminal, and the terminal is used for optimizing the video data according to the optimization description information and playing and outputting the optimized video data.
In order to solve the above problem, an embodiment of the present application discloses a data processing apparatus, including: the live broadcast transmission information receiving module is used for receiving live broadcast video transmission information; the live broadcast transmission information analysis module is used for analyzing the live broadcast video transmission information to obtain live broadcast video data and optimized description information; the live video data optimization module is used for optimizing the live video data according to the optimization description information; and the live video data playing module is used for playing the optimized live video data.
In order to solve the above problem, an embodiment of the present application discloses a data processing apparatus, including: the live video data analysis module is used for acquiring live video data, analyzing the live video data and determining optimized description information; and the live broadcast transmission information packaging module is used for packaging the optimization description information and the live broadcast video data to obtain live broadcast video transmission information and transmitting the live broadcast video transmission information to the live broadcast video playing terminal, and the live broadcast video playing terminal is used for optimizing the live broadcast video data according to the optimization description information and playing and outputting the optimized live broadcast video data.
In order to solve the above problem, an embodiment of the present application discloses a data processing apparatus, including: the monitoring transmission information receiving module is used for receiving monitoring video transmission information; the monitoring transmission information analysis module is used for analyzing the monitoring video transmission information to obtain monitoring video data and optimization description information; the monitoring video data optimization module is used for optimizing the monitoring video data according to the optimization description information; and the monitoring video data playing module is used for playing the optimized monitoring video data.
In order to solve the above problem, an embodiment of the present application discloses a data processing apparatus, including: the monitoring video data analysis module is used for acquiring monitoring video data, analyzing the monitoring video data and determining optimization description information; and the monitoring transmission information packaging module is used for packaging the optimization description information and the monitoring video data to obtain monitoring video transmission information and transmitting the monitoring video transmission information to the monitoring video playing terminal, and the monitoring video playing terminal is used for optimizing the monitoring video data according to the optimization description information and playing and outputting the optimized monitoring video data.
In order to solve the above problem, an embodiment of the present application discloses a data processing apparatus, including: the transmission information acquisition module is used for receiving video transmission information; the sub-video data acquisition module is used for analyzing the video transmission information to acquire video data and corresponding optimization description information, wherein the video data comprises a plurality of paths of sub-video data; the sub-video data optimization module is used for optimizing target sub-video data in the video data according to the optimization description information; and the multi-channel video playing module is used for playing the optimized video data.
In order to solve the above problem, an embodiment of the present application discloses a data processing apparatus, including: the sub-video data acquisition module is used for acquiring multiple paths of sub-video data to determine video data; the sub-video data analysis module is used for analyzing the target sub-video data and determining corresponding optimization description information; and the transmission information acquisition module is used for packaging the optimization description information and the video data to obtain video transmission information and transmitting the video transmission information to the video playing terminal, and the video playing terminal is used for optimizing target sub-video data in the video data according to the optimization description information and playing and outputting the optimized live video data.
In order to solve the above problem, an embodiment of the present application discloses a data processing apparatus, including: the selection page providing module is used for providing an optimization mode selection page, and the optimization mode selection page comprises a plurality of optimization selection controls corresponding to different optimization modes; and the selection information acquisition module is used for determining selection information according to the operation instruction of the optimization selection control and transmitting the selection information to the server so that the server can process the video data in a corresponding optimization mode according to the selection information.
In order to solve the above problem, an embodiment of the present application discloses a data processing apparatus, including: the selection information receiving module is used for receiving selection information, and the selection information is determined according to an optimization mode selection page; the optimization mode acquisition module is used for determining a target optimization mode aiming at the video data according to the selection information; and the optimized data acquisition module is used for carrying out corresponding processing on the acquired video data according to the target optimization mode.
In order to solve the above problem, an embodiment of the present application discloses an electronic device, including: a processor; and a memory having executable code stored thereon, which when executed, causes the processor to perform a method as described in one or more of the above method embodiments.
To address the above issues, embodiments of the present application disclose one or more machine-readable media having executable code stored thereon that, when executed, cause a processor to perform a method as described in one or more of the above method embodiments.
Compared with the prior art, the embodiment of the application has the following advantages:
in the embodiment of the application, the server side can analyze the video data by using the calculation power of the server side, determine the content to be optimized in the video data to serve as the optimization description information, and transcode the optimization description information and the video data into the video transmission information. And then the server transmits the video transmission information to the video playing terminal, the video playing terminal can analyze the video transmission information to obtain video data and optimized description information, then the video data is optimized according to the optimized description information, and the optimized video data is played. In the embodiment of the application, the video data can be analyzed by utilizing the large calculation power of the server, the content to be optimized is determined to be used as the optimization description information, and compared with a mode of analyzing the video data by adopting the small calculation power of a video playing terminal, the mode of the embodiment of the application can improve the data quality of the video data. In addition, the optimization description information is added to the video transmission information, the influence on the file size of the video transmission information is small, and compared with a mode of analyzing and optimizing video data at a server, the data transmission efficiency can be guaranteed.
Drawings
FIG. 1A is a system architecture diagram of a data processing system according to one embodiment of the present application;
FIG. 1B is a system architecture diagram of a data processing system according to another embodiment of the present application;
FIG. 2A is a schematic flow chart diagram of a data processing method according to an embodiment of the present application;
FIG. 2B is a schematic flow chart diagram of a data processing method according to another embodiment of the present application;
FIG. 2C is a schematic flow chart diagram of a data processing method according to yet another embodiment of the present application;
FIG. 3 is a schematic flow chart diagram of a data processing method according to yet another embodiment of the present application;
FIG. 4 is a schematic flow chart diagram of a data processing method according to yet another embodiment of the present application;
FIG. 5 is a schematic flow chart diagram of a data processing method according to yet another embodiment of the present application;
FIG. 6 is a schematic flow chart diagram of a data processing method according to yet another embodiment of the present application;
FIG. 7 is a schematic flow chart diagram of a data processing method according to yet another embodiment of the present application;
FIG. 8 is a schematic flow chart diagram of a data processing method according to yet another embodiment of the present application;
FIG. 9 is a schematic flow chart diagram of a data processing method according to yet another embodiment of the present application;
FIG. 10 is a schematic flow chart diagram of a data processing method according to yet another embodiment of the present application;
FIG. 11 is a schematic flow chart diagram of a data processing method according to yet another embodiment of the present application;
FIG. 12 is a schematic flow chart diagram of a data processing method according to yet another embodiment of the present application;
FIG. 13 is a schematic flow chart diagram of a data processing method according to yet another embodiment of the present application;
FIG. 14 is a block diagram of a data processing apparatus according to an embodiment of the present application;
FIG. 15 is a schematic block diagram of a data processing apparatus according to another embodiment of the present application;
FIG. 16 is a schematic block diagram of a data processing apparatus according to yet another embodiment of the present application;
FIG. 17 is a schematic block diagram of a data processing apparatus according to yet another embodiment of the present application;
FIG. 18 is a schematic block diagram of a data processing apparatus according to yet another embodiment of the present application;
FIG. 19 is a schematic block diagram of a data processing apparatus according to yet another embodiment of the present application;
FIG. 20 is a schematic block diagram of a data processing apparatus according to yet another embodiment of the present application;
FIG. 21 is a schematic block diagram of a data processing apparatus according to yet another embodiment of the present application;
FIG. 22 is a schematic block diagram of a data processing apparatus according to yet another embodiment of the present application;
FIG. 23 is a schematic block diagram of a data processing apparatus according to yet another embodiment of the present application;
fig. 24 is a schematic structural diagram of an exemplary apparatus provided in one embodiment of the present application.
Detailed Description
In order to make the aforementioned objects, features and advantages of the present application more comprehensible, the present application is described in further detail with reference to the accompanying drawings and the detailed description.
The embodiment of the application can be applied to the field of data optimization of multimedia data, wherein the multimedia data refers to the data which is formed by fusing two or more media data and is man-machine interactive information exchange and transmission data. Data optimization refers to analyzing and correspondingly adjusting multimedia data to improve the data quality of the multimedia data, such as improving the image quality of image data in the multimedia data and improving the sound quality of audio data in the multimedia data.
The multimedia data is taken as an example of video data, wherein the video data comprises image data and audio data, and the video data can be film and television works, live video data, video data shot by monitoring equipment, conference video data, commodity video data, navigation video data and the like. As shown in fig. 1A, in the embodiment of the present application, the analysis of the video data and the optimization process of the video data may be separated, the video data is analyzed at the service end, and the video data is optimized at the video playing terminal according to the analysis result at the service end. In the embodiment of the application, the multimedia data can be analyzed by using the large calculation power of the server and adopting a complex optimization algorithm, the content to be optimized is determined to be used as the optimization description information, and compared with a mode of analyzing the multimedia data by using the small calculation power of the terminal and adopting a simple optimization algorithm, the data quality of the multimedia data can be improved by using the mode of the embodiment of the application.
Specifically, as shown in fig. 1B, the server may analyze the video data by using the calculation power of the server itself, determine the content to be optimized in the video data as optimization description information, and add the optimization description information as a signaling to a code stream file after transcoding the video data to obtain video transmission information. For example, for image data, the server may analyze various parameters (such as contrast, balance, etc.) in an image, determine an adjustment amount corresponding to each parameter, and encapsulate the adjustment amount as optimization description information into video transmission information; semantic information of each object in the image can be analyzed (for example, the object in the image is determined to be a person, an animal, a tree, and the like), and the semantic information of the object is packaged into video transmission information as optimization description information. For another example, for audio data, the server may analyze the waveform of the audio, the noise in the audio, and the like to determine corresponding optimization description information, and encapsulate the optimization description information into the video transmission information.
After the video transmission information is determined, the server side can transmit the video transmission information to the video playing terminal, the video playing terminal can analyze the video transmission information to obtain video data and optimization description information, then, the video data is optimized according to the optimization description information, and the optimized video data is played. In the embodiment of the application, the multimedia data can be analyzed by utilizing the large calculation power of the server, the content to be optimized is determined to be used as the optimization description information, and compared with a mode of analyzing the multimedia data by utilizing the small calculation power of a video playing terminal, the mode of the embodiment of the application can improve the data quality of the multimedia data. In addition, the optimization description information is added to the video transmission information, the influence on the file size of the video transmission information is small, and compared with a mode of analyzing and optimizing video data at a server, the data transmission efficiency can be guaranteed.
The server in the embodiment of the application can include equipment for data storage and data forwarding, the video playing terminal can also be called a terminal, the video playing terminal can include an intelligent terminal such as an intelligent television, a projector, a mobile phone and a computer, and the video playing terminal can also include other internet of things equipment capable of outputting video data such as a central control system of a vehicle.
In the above embodiments, the optimization processing of the video data including the image data and the audio data is described as an example, and the embodiments of the present application may be applied to an optimization processing scenario of the video data, and may also be applied to an optimization processing scenario of the image data or the audio data alone. The image data may be a monitoring image, a stored photographic image, a commodity image, a navigation image, and the like, in a period of time. The audio data can be audio data of audio live broadcast, audio data of musical works, conference recording, navigation audio and the like. For example, the server may analyze the monitored image, determine the content to be optimized in the monitored image, and send the content to the terminal (which may be a video playing terminal or an image playing terminal) as the optimization description information, and after receiving the optimization description information, the terminal may optimize the monitored image based on the optimization description information, thereby improving the quality of the monitored image. For another example, the server may analyze audio data of audio live broadcast, determine content of the audio data to be optimized, serve as optimization description information, and send the optimization description information to the terminal (which may be a video playing terminal or an audio playing terminal), and the terminal may optimize the audio data according to the optimization description information, thereby improving the definition of the audio data.
The embodiment of the application provides a data processing method, which can be applied to a video playing terminal, wherein the video playing terminal can comprise an intelligent terminal such as an intelligent television, a projector, a mobile phone, a computer and the like, and the video playing terminal can also comprise other internet of things equipment capable of outputting video data such as a central control system of a vehicle and the like. As shown in fig. 2, the method includes:
step 202, receiving video transmission information. And the video playing terminal interacts with the server to acquire video transmission information. The video data may include image data and audio data. Before sending video transmission information to a terminal, a server can perform identification analysis (or called medium analysis) on video data to determine optimization description information for optimizing the video data, encapsulate the optimization description information and the video data to obtain video transmission information, and then transmit the video transmission information to the terminal. The optimization description information refers to information for optimizing image data and/or audio data, for example, for an image, the optimization description information may include various image parameter information (such as an image contrast parameter, an image equalization parameter, an image sharpening parameter, and the like) of the image, and the video playing terminal may adjust parameters of the image according to the image parameter information; in addition, the optimization description information may further include object feature information of the image, the object feature information of the image may be determined by performing semantic recognition on the image, and performing semantic recognition on the image may be understood as performing semantic recognition on each object in the image, for example, determining that the image includes a person and a position corresponding to the person, determining that the image includes a vehicle and a position corresponding to the vehicle, and correspondingly, the video playing terminal may perform optimization processing on the object in the image according to the object feature information of the image.
After the optimization description information of the video data is determined, the optimization description information and the video data may be encapsulated, wherein in an alternative embodiment, the video data may be transcoded into a code stream file, the optimization description information is transcoded into a separate signaling, and then the code stream file and the signaling are encapsulated into video transmission information for data transmission. In another alternative embodiment, the video data may be transcoded into a code stream file, the optimization description information is temporally matched with the code stream file, and a signaling corresponding to the optimization description information is inserted into the code stream file to transmit the data.
And step 204, analyzing the video transmission information to obtain video data and optimization description information. After receiving the video transmission information, the terminal may parse the video transmission information to obtain video data and optimized description information in the video transmission information. For different packaging modes, the terminal can adopt corresponding analysis modes. When the server encapsulates the signaling and the code stream file corresponding to the optimization description information, the terminal can analyze the code stream file and the signaling respectively to obtain the video data and the optimization description information. Specifically, as an optional embodiment, the analyzing the video transmission information to obtain video data and optimization description information includes: analyzing a code stream file in video transmission information to obtain video data; and analyzing the signaling in the video transmission information to obtain the optimized description information. The code stream file is data obtained by transcoding the video data through the server side, and the video playing terminal can perform video decoding on the code stream file to obtain the video data. The signaling is data obtained by transcoding the optimized description information obtained by analysis by the server side, and the video playing terminal can decode the signaling to obtain a code stream file.
In the embodiment of the application, a packaging rule can be preset in the server, and the packaging rule is used for packaging the optimization description information. The server can encapsulate the optimized description information and the rule information of the encapsulation rule according to the encapsulation rule to obtain the signaling. Correspondingly, after receiving the video transmission information, the video playing terminal can analyze the rule information of the encapsulation rule in the signaling, and then extract the corresponding optimization description information according to the rule information. Specifically, as an optional embodiment, the analyzing signaling in the video transmission information to obtain the optimized description information includes: identifying the rule identification information of the signaling in the video transmission information, and determining the encapsulation rule of the signaling; and extracting the optimization description information according to the packaging rule. The encapsulation rule may also be referred to as an encapsulation protocol, and the signaling in the video transmission information may include rule information of the encapsulation rule, and the rule information may include a rule identifier, a rule version number, an information amount of the encapsulated optimization description information, an information type of the encapsulated optimization description information, an information location of the encapsulated optimization description information, and the like. After receiving the video transmission information, the video playing terminal may first parse the rule information to determine a packaging rule for packaging the optimized description information, and then extract the corresponding optimized description information from the information location according to the packaging rule. After the video playing terminal extracts the optimization description information, an optimization task can be established for each optimization description information, and the optimization task is executed to optimize video data. The number of optimization tasks can be determined according to the information amount in the rule information, and the information amount of the encapsulated optimization description information can be understood as the number of bytes occupied by the optimization description information. The task type of the optimization task may be determined according to the information type in the rule information. For example, the optimization description information may include an image contrast parameter for optimizing the contrast of the image, the information type corresponding to the optimization description information in the rule information of the encapsulation rule may be a parameter type (the division of the information type may be set according to a requirement, and the division of the information type is not limited here), and after the terminal acquires the image contrast parameter, a parameter adjustment task may be newly created according to the information type of the image contrast parameter, and the parameter adjustment task may be executed to adjust the contrast of the image data. The optimization tasks can be executed in sequence, and correspondingly, the rule information of the encapsulation rule can also carry the execution sequence of each optimization description information.
When the server matches the signaling corresponding to the optimization description information with the code stream file and inserts the signaling into the code stream file, the video playing terminal can analyze the code stream file to obtain video data and optimization description information. Specifically, as an optional embodiment, the analyzing the video transmission information to obtain video data and optimization description information includes: and analyzing a code stream file in the video transmission information to obtain video data and optimization description information, wherein the optimization description information is packaged in the code stream file. The server can match (e.g., align) the optimization description information with the video data according to the timestamps of the optimization description information and the video data, and insert a signaling for transcoding the optimization description information into a code stream file after transcoding the video data. Therefore, after the video playing terminal receives the video transmission information, the code stream file can be analyzed to obtain the matched video data and the optimized description information, so that the subsequent optimization processing can be performed.
After the video playing terminal obtains the optimized description information and the video data through parsing, in step 206, the video data may be optimized according to the optimized description information. The video data may include image data and audio data, and the video playing terminal may divide optimization description information for optimizing the image data and optimization description information for optimizing the audio data, and perform optimization processing on the image data and the audio data, respectively, using the divided optimization description information. Specifically, as an optional embodiment, the video data includes image data and audio data, the optimization description information includes first optimization information for optimizing the image data and second optimization information for optimizing the audio data, and the optimizing the video data according to the optimization description information includes: matching the first optimization information with the image data according to the time stamps of the image data and the first optimization information, and performing image optimization processing on the image data; and matching the second optimization information with the audio data according to the time stamps of the audio data and the second optimization information, and performing audio optimization processing on the audio data. In this embodiment, in the process of analyzing the video data, the server may segment the image data and the audio data according to frames to obtain image data and audio data corresponding to different timestamps, and analyze optimization description information corresponding to the data, where a timestamp of the optimization description information corresponds to a timestamp of the data.
As shown in fig. 2B, after receiving the optimization description information, the video playing terminal may divide the optimization description information into first optimization information and second optimization information. Then, aligning the first optimization information with the image data by utilizing the time stamp of the first optimization information and the time stamp of the image data, and carrying out image optimization processing on the image data; and aligning the second optimization information and the audio data by using the time stamp of the second optimization information and the time stamp of the audio data, and performing audio optimization processing on the audio data. In this embodiment, the optimization description information is obtained by analyzing the bitstream file and the signaling, respectively. Therefore, in this embodiment, before performing optimization processing on the video data, the optimization description information needs to be aligned with the video data according to the time stamps of the optimization description information and the video data. In another optional embodiment, the optimization description information may be obtained by parsing the bitstream file, so that the video playing terminal may perform optimization processing on the image data matched with the optimization description information according to the first optimization information in the optimization description information; and optimizing the audio data matched with the optimization description information according to second optimization information in the optimization description information.
The video playing terminal may optimize the image data according to first optimization information, where the first optimization information may include at least one of image parameter information and object feature information of the image, the image parameter information is information used for optimizing and adjusting parameters of the image data, and the object feature information of the image is information characterizing features of content of the image, such as a scene of the image, a target object in the image, and the like, and specifically, as an optional embodiment, the image optimization processing on the image data includes at least one of the following steps: adjusting the parameters of the image according to the image parameter information; and according to the object characteristic information of the image, optimizing the corresponding object in the image. For different first optimization information, the video playing terminal may establish a corresponding data optimization task to perform image optimization according to the corresponding first optimization information, for example, for image parameter information, a parameter adjustment task for performing parameter adjustment according to parameters may be established; for object feature information of an image, a feature optimization task for optimizing an object in the image may be established. As an optional embodiment, the adjusting the parameters of the image according to the image parameter information includes at least one of the following steps: sharpening the image data according to the image sharpening parameter; adjusting the saturation of the image data according to the image saturation parameter; adjusting the contrast of the image data according to the image contrast parameter; denoising the image data according to the image denoising parameters; and according to the image deblocking parameters, performing deblocking processing on the image data.
The image feature information may include image scene feature information, image semantic feature information, image background feature information, image depth feature information, and the like. Specifically, as an optional embodiment, the optimizing the corresponding object in the image according to the object feature information of the image includes at least one of the following steps: performing scene optimization processing on the image data according to the image scene characteristic information; performing semantic optimization processing on the image data according to the image semantic feature information; performing background optimization processing on the image data according to the image background characteristic information; and performing depth-of-field optimization processing on the image data according to the depth-of-field characteristic information of the image.
The video playing terminal can perform optimization processing on the image data according to the image parameter information and the image characteristic information. The Image Sharpening parameter is used for Image Sharpening, and Image Sharpening (Image Sharpening) is a part for compensating the outline of an Image, enhancing the edge and the gray level jump of the Image and enabling the Image to be clear and is divided into two types of spatial domain processing and frequency domain processing. Image sharpening is to highlight edges, contours, or features of some linear target elements of a terrain on an image. This filtering method improves the contrast between the feature edges and the surrounding picture elements and is therefore also referred to as edge enhancement. The image Saturation parameter is used to adjust the Saturation of image colors, and the Saturation of image colors (Saturation) refers to the degree of vividness of colors, also called Saturation. The image contrast parameter is used to adjust the contrast of the image. Image Contrast (Image Contrast) refers to the measurement of the different brightness levels between the brightest white and darkest black of the brightest light and dark regions in an Image, i.e. the magnitude of the gray Contrast of an Image. A larger difference range represents a larger contrast, and a smaller difference range represents a smaller contrast. The Image Denoising parameter is used for performing Image Denoising (Image Denoising) which is a process of reducing noise in a digital Image. In reality, digital images are often affected by interference of imaging equipment and external environment noise during digitization and transmission, and are called noisy images or noisy images. The image block parameters are used for reducing Blocking artifacts (Blocking artifacts) of the image, the Blocking artifacts are widely applied to image compression coding based on block transform coding, and with the reduction of code rate, quantization becomes rough, discontinuity occurs at the boundary of a block, and an obvious defect of a reconstructed image is formed, which is called the Blocking artifacts.
The image scene feature information is used to perform scene optimization on the image data, and the image scene feature information may include a portrait scene, a fun-shot (adding a special photographing effect, such as a cat ear) scene, and the like. The image semantic feature information comprises a segmentation result and corresponding semantic information of each target object in the image, and the terminal can determine each region in the image according to the segmentation result and adopt corresponding optimization processing to each region according to the corresponding semantic information. For example, when the image of the cat and the image of the dog exist in the image, the server may perform semantic recognition on the image, recognize image areas corresponding to the cat and the dog, respectively, obtain a segmentation result corresponding to the "cat" and an image segmentation result corresponding to the "dog", and perform optimization processing on each image area by the video playing terminal according to the segmentation result and the corresponding semantic information. The image background feature information is used for performing background optimization processing on the image data, the image background feature includes features of a background of the image, such as background color, background shape, background content, and the like, wherein the background content may also be referred to as an image content attribute, the image content attribute of the image data may include mountains, nights, daytime, and the like, and the video playing terminal may perform corresponding processing on the image data according to the image content attribute, and for example, when the image content attribute is nighttime, the video playing terminal may adjust brightness of the image background, thereby improving definition of the background in the image. The image depth of field feature information is used for performing depth of field optimization processing on image data, the image depth of field feature information comprises depth of field (DOF) data of a target object in an image, and the depth of field refers to a range of front and rear distances of a shot object measured by imaging which can obtain a clear image at the front edge of a camera lens or other imagers. For a plurality of target objects with different depths of field in the image, different optimization processing algorithms can be adopted, for example, the definition of the focused target object can be improved, and the definition of other target objects can be reduced, so that the focused target object is blurred, and the image quality is improved.
In addition, the video data can also correspond to other attributes, and the server can encapsulate the other attributes of the video data in the video transmission information, so that the video playing terminal can optimize the video data by adopting an optimization mode corresponding to the attributes according to the corresponding attributes. For example, the image data in the video data may further correspond to an image interaction attribute, which may be understood as a mode in which an image is output to a user, for example, the image interaction attribute may include a movie mode, a game mode and a tv play mode, wherein the image data in the game mode may determine an image to be displayed next according to the operation of the user, and correspondingly, after the operation of the user is obtained, a next image to be displayed (a next frame in response to the game operation) may be determined and optimized. The image of the drama mode may arrange a plurality of images (corresponding to different episodes) in order, and perform optimization processing on the image data in the order of arrangement of the image data. The images in the movie mode can determine the images to be optimized according to the output progress (playing progress) of the images, and perform optimization processing on the images to be optimized. For another example, the video data may further correspond to a user preference attribute, so as to perform optimization processing on the image data by using a corresponding optimization mode according to different preferences of different users, for example, if the preference of the user 1 is high in requirement on sound quality and low in requirement on image pictures, the user 1 may select a corresponding optimization mode and transmit the optimization mode to the server, the server analyzes the video data according to the selection of the user 1 to obtain the image data and the audio data, analyzes the audio data, determines optimization description information of the audio data, encapsulates the optimization description information and transmits the optimization description information to the video playing terminal, and the video playing terminal may perform optimization processing on the audio data according to the optimization description information. For another example, if the preference of the user 2 is high in requirements on the image and the audio, the preference of the user 2 may be uploaded to the server, the server may analyze the image data and the audio data in the video data, determine the optimized description information corresponding to the image data and the audio data, encapsulate the optimized description information, and transmit the encapsulated optimized description information to the video playing terminal, and the video playing terminal may perform optimization processing on the image data and the audio data according to the optimized description information.
The video playing terminal can optimize and adjust the audio data according to the second optimization information. The second optimization information may include audio parameter information and audio feature information, the audio parameter information being information for performing optimization adjustment on a parameter of the audio data, and the audio feature information being information characterizing a content of the audio. Specifically, as an optional embodiment, the performing audio optimization processing on the audio data includes at least one of the following steps: adjusting the waveform of the audio data according to the audio waveform parameters; according to the audio filtering parameters, carrying out filtering processing on the audio data; according to the audio equalization parameters, performing equalization processing on the audio data; and carrying out scene optimization processing on the audio data according to the audio scene characteristic information.
The audio parameter information may include audio waveform parameters, audio filtering parameters, audio equalization parameters, and the like. The audio waveform parameters are used for adjusting sound waves, the audio filtering parameters are used for filtering audio data, the audio filtering parameters can be understood as coefficients of a filter, and the filter can be a hardware filter or a software filter. The audio equalization parameters are used for being input into a sound equalizer to perform equalization processing on audio data, the sound equalizer is an electronic device capable of adjusting the amplification amount of electric signals with various frequency components respectively, and the sound equalizer compensates the defects of a loudspeaker and a sound field through adjusting the electric signals with various frequencies, compensates and modifies various sound sources and has other special functions. The audio scene characteristic information can include audio scene characteristic information, the audio scene characteristic information represents a scene corresponding to the content of the audio, the video playing terminal can perform optimization processing on the audio data according to the scene, if the content of the audio contains drum sound, the video playing terminal can perform sound effect enhancement on the drum sound, and the quality of the audio data is improved.
According to the embodiment of the application, an optimization program can be preset in the video playing terminal, the optimization program can be understood as an algorithm, a script and the like, the video playing terminal inputs the optimization description information and the video data into the optimization program, and the optimization processing is carried out by utilizing the optimization program. Specifically, as an optional embodiment, the performing optimization processing on the video data according to the optimization description information includes: and inputting the optimization description information and the video data into a target optimizer, and determining the optimized video data. The target optimizer may be understood as an optimization program, and forms a plurality of optimization tasks by inputting the optimization description information and the video data into the optimization program, and performs optimization processing on the video data by executing the optimization tasks.
In this embodiment of the present application, hardware of the video playing terminal may also be used to perform optimization processing, specifically, as shown in fig. 2C, as an optional embodiment, the performing optimization processing on the video data according to the optimization description information includes: and inputting the optimization description information into a controller, wherein the controller is used for converting the optimization description information into a chip control instruction and controlling an optimization component to perform optimization processing on video data through the chip control instruction. An optimization component may be understood as hardware for optimizing video data, such as filters, equalizers, etc. The controller can be understood as a chip, the terminal can configure the chip as a controller for performing optimization processing, after receiving the optimization description information, the controller can convert the optimization description information into a chip control instruction, and by executing the chip control instruction, the corresponding optimization component can be controlled to perform optimization processing on the video data.
The server side can send corresponding configuration information to the video playing terminal according to chip information (such as a chip model and a chip version) of a processing chip of the video playing terminal, and the processing chip of the video playing terminal can be configured according to the configuration information to obtain a controller for optimizing video data. Specifically, as an optional embodiment, the method further includes: acquiring controller configuration information; and configuring the processing chip as the controller based on the controller configuration information. The server can preset chip capability index service, chip information of chips of various types and versions is stored in the chip capability index service, corresponding controller configuration information is set for the chips of various types and versions, and the controller configuration information is used for configuring the chips so as to generate a controller capable of converting optimization description information into a chip control instruction and optimizing video data. The video playing terminal can send the chip information of the processing chip of the terminal to the server side to obtain the corresponding controller configuration information, and then the processing chip is configured according to the controller configuration information to obtain the controller, so that the video data can be optimized.
After optimizing the video data, the terminal plays the optimized video data in step 208. The video playing terminal can output the optimized image data through the display and output the optimized audio data through the player.
In the embodiment of the application, the server side can analyze the video data by using the calculation power of the server side, determine the content to be optimized in the video data to serve as the optimization description information, and transcode the optimization description information and the video data into the video transmission information. And then the server transmits the video transmission information to the video playing terminal, the video playing terminal can analyze the video transmission information to obtain video data and optimized description information, then the video data is optimized according to the optimized description information, and the optimized video data is played. In the embodiment of the application, the video data can be analyzed by utilizing the large calculation power of the server, the content to be optimized is determined to be used as the optimization description information, and compared with the method for analyzing the video data by adopting the small calculation power of the video playing terminal, the data quality of the video data can be improved. In addition, the optimization description information is added to the video transmission information, the influence on the file size of the video transmission information is small, and compared with a mode of analyzing and optimizing video data at a server, the data transmission efficiency can be guaranteed.
On the basis of the above embodiments, the embodiment of the present application further provides a data processing method, which can be applied to a video playing terminal, where the video playing terminal may include an intelligent terminal, such as an intelligent television, a projector, a mobile phone, a computer, and the like, and the video playing terminal may further include other internet of things devices capable of outputting video data, such as a central control system of a vehicle, and the like. As shown in fig. 3, the method includes:
step 302, receiving video transmission information, wherein the video data includes image data and audio data.
And step 304, analyzing the code stream file in the video transmission information to obtain video data.
Step 306, identifying the rule identification information of the signaling in the video transmission information, and determining the encapsulation rule of the signaling.
And 308, extracting the optimization description information according to the packaging rule. Wherein the optimization description information includes first optimization information for optimizing the image data and second optimization information for optimizing the audio data. In addition, in another optional embodiment, the server may further align the optimization description information with the video data according to a timestamp in the server, and then insert the optimization description information into the code stream file, so that the video playing terminal may parse the code stream file to obtain the video data and the optimization description information, and specifically, parse the video transmission information, including: and analyzing a code stream file in the video transmission information to obtain video data and optimization description information, wherein the optimization description information is packaged in the code stream file.
And step 310, matching the first optimization information with the image data according to the time stamps of the image data and the first optimization information, and performing image optimization processing on the image data. As an optional embodiment, the performing the image optimization processing on the image data includes at least one of the following steps: sharpening the image data according to the image sharpening parameter; adjusting the saturation of the image data according to the image saturation parameter; adjusting the contrast of the image data according to the image contrast parameter; denoising the image data according to the image denoising parameters; according to the image deblocking parameters, performing deblocking processing on the image data; performing scene optimization processing on the image data according to the image scene characteristic information; performing semantic optimization processing on the image data according to the image semantic feature information; performing background optimization processing on the image data according to the image background characteristic information; and performing depth-of-field optimization processing on the image data according to the depth-of-field characteristic information of the image.
And step 312, matching the second optimization information with the audio data according to the time stamps of the audio data and the second optimization information, and performing audio optimization processing on the audio data. As an optional embodiment, the performing the audio optimization processing on the audio data includes at least one of the following steps: adjusting the waveform of the audio data according to the audio waveform parameters; according to the audio filtering parameters, carrying out filtering processing on the audio data; according to the audio equalization parameters, performing equalization processing on the audio data; and carrying out scene optimization processing on the audio data according to the audio scene characteristic information.
In addition, in this embodiment of the present application, hardware of the video playing terminal may also be used to perform optimization processing on the video data, and specifically, as an optional embodiment, the performing optimization processing on the video data according to the optimization description information includes: and inputting the optimization description information into a controller, wherein the controller is used for converting the optimization description information into a chip control instruction and controlling an optimization component to perform optimization processing on video data through the chip control instruction. Specifically, as an optional embodiment, the method further includes: acquiring controller configuration information; and configuring the processing chip as the controller based on the controller configuration information.
And step 314, playing the optimized video data.
In the embodiment of the application, the server side can analyze the video data by using the calculation power of the server side, determine the content to be optimized in the video data as the optimization description information, and transcode the optimization description information and the video data into the video transmission information. Then the server transmits the video transmission information to the video playing terminal, after the video playing terminal receives the video transmission information, the video playing terminal can analyze a code stream file in the video transmission information to obtain video data, can analyze a signaling in the video transmission information, determine an encapsulation rule of the signaling, and extract optimization description information according to the encapsulation rule, wherein the optimization description information can include first optimization information corresponding to the image data and second optimization information corresponding to the audio data. After the video playing terminal determines the optimization description information, the video playing terminal can match the first optimization information with the image data according to the time stamp and perform optimization processing on the image data, can match the second optimization information with the audio data according to the time stamp and perform optimization processing on the audio data to obtain optimized video data, and plays the optimized video data. In the embodiment of the application, the video data can be analyzed by utilizing the large calculation power of the server, the content to be optimized is determined to be used as the optimization description information, and compared with the method for analyzing the video data by adopting the small calculation power of the video playing terminal, the data quality of the video data can be improved. In addition, the optimization description information is added to the video transmission information, the influence on the file size of the video transmission information is small, and compared with a mode of analyzing and optimizing video data at a server, the data transmission efficiency can be guaranteed.
On the basis of the foregoing embodiments, an embodiment of the present application further provides a data processing method, which can be applied to a server, where the server can be understood as a device for storing video data or transferring video data, as shown in fig. 4, the method includes:
step 402, analyzing the video data and determining the optimization description information. The video data may include image data and audio data, and the server may slice the image data by frame (or in other ways), and analyze various parameters of the image and characteristics of the image to determine optimization description information, such as the following parameters and characteristics of the image: the image processing method comprises the following steps of image sharpening parameters, image saturation parameters, image contrast parameters, image denoising parameters, image deblocking parameters, image scene characteristic information, image semantic characteristic information, image background characteristic information and image depth characteristic information. The server-side can also segment the audio data by frames and analyze various parameters and characteristics of the audio to determine the optimization description information. The following parameters, characteristics of the audio can be analyzed: audio waveform parameters, audio filtering parameters, audio equalization parameters, and audio scene characteristic information.
And step 404, packaging the optimization description information and the video data to obtain video transmission information, and transmitting the video transmission information to a video playing terminal, wherein the video playing terminal is used for optimizing the video data according to the optimization description information and playing and outputting the optimized video data. In an optional embodiment, in the process of encapsulating the video transmission information, the server may further perform encapsulation in an encryption encapsulation manner to improve the security of the video transmission information, and accordingly, after receiving the video transmission information, the terminal may decrypt the video transmission information first and then analyze the optimization description information and the video data.
The implementation of the data processing method in the embodiment of the present application is similar to that of the method in the embodiment described above, and specific implementation may refer to the implementation of the embodiment described above, and details are not described here again.
In the embodiment of the application, the server side can analyze the video data by using the calculation power of the server side, determine the content to be optimized in the video data as the optimization description information, and transcode the optimization description information and the video data into the video transmission information. And then the server transmits the video transmission information to the video playing terminal, the video playing terminal can analyze the video transmission information to obtain video data and optimized description information, then the video data is optimized according to the optimized description information, and the optimized video data is played. In the embodiment of the application, the video data can be analyzed by utilizing stronger calculation power of the server, the content to be optimized is determined as the optimization description information, and compared with the method for analyzing the video data by adopting the small calculation power of the video playing terminal, the data quality of the video data can be improved. In addition, the optimization description information is added to the video transmission information, the influence on the file size of the video transmission information is small, and compared with a mode of analyzing and optimizing video data at a server, the data transmission efficiency can be guaranteed.
In this embodiment of the application, the server may transcode the optimized description information and the video data respectively to obtain the video transmission information, and specifically, as an optional embodiment, the encapsulating the optimized description information and the video data to obtain the video transmission information includes: according to the encapsulation rule, encapsulating the optimized description information into a signaling; packaging the video data into a code stream file; and determining video transmission information according to the signaling and the code stream file. The method and the device for optimizing the video data can transcode the optimized description information into the signaling, transcode the video data into the code stream file, and package the code stream file into the video transmission information to be transmitted to the video playing terminal. After receiving the video transmission information, the video playing terminal can analyze the code stream file and the signaling respectively to obtain the optimized description information and the video data.
In the embodiment of the application, the server can also match the optimization description information with the video data, and then insert the optimization description information into the code stream file correspondingly. Specifically, as an optional embodiment, the encapsulating the optimization description information and the video data to obtain the video transmission information includes: matching the video data with the optimization description information; and packaging the video data into a code stream file, and adding the optimized description information matched with the code stream file into the code stream file to obtain video transmission information. The server side can match the optimized description information with the video data according to the time stamps of the optimized description information and the video data, transcode the video data into a code stream file, correspondingly insert the optimized description information into the code stream file to obtain video transmission information, and send the video transmission information to the video playing terminal. After receiving the video transmission information, the video playing terminal can analyze the code stream file to obtain video data and optimized description information.
When the video playing terminal optimizes the video data according to the optimization description information, the controller can be adopted to control corresponding hardware to perform data optimization processing. The corresponding hardware may include a sound equalizer, filters, etc. The controller can be obtained by configuring the chip, the server can send the controller configuration parameters to the video playing terminal, and the video playing terminal configures the chip according to the controller configuration parameters to obtain the controller. Specifically, as an optional embodiment, the method further includes: acquiring chip information of a processing chip of a video playing terminal; determining corresponding controller configuration parameters according to the chip information; and issuing the controller configuration parameters to configure a processing chip as a controller based on the controller configuration information, wherein the controller is used for converting the optimization description information into a chip control instruction and controlling an optimization component to perform optimization processing on the video data through the chip control instruction. The server can preset chip capability index service, wherein chip information of chips of various types and versions can be stored, and corresponding controller configuration information is set for the chips of various types and versions, and the controller configuration information is used for configuring the chips to generate a controller which can convert optimization description information into a chip control instruction and optimize video data. The video playing terminal can send chip information of a processing chip of the video playing terminal to the server side to obtain corresponding controller configuration information, and then the processing chip is configured according to the controller configuration information to obtain the controller, so that the video data can be optimized.
On the basis of the foregoing embodiments, an embodiment of the present application further provides a data processing method, which can be applied to a server, where the server can be understood as a device for storing video data or transferring video data, as shown in fig. 5, the method includes:
step 502, analyzing the video data to determine the optimized description information.
Step 504, according to the encapsulation rule, the optimization description information is encapsulated into signaling.
Step 506, packaging the video data into a code stream file.
And step 508, determining video transmission information according to the signaling and the code stream file, and transmitting the video transmission information to a video playing terminal, wherein the video playing terminal is used for optimizing video data according to the optimization description information and playing and outputting the optimized video data.
And step 510, acquiring chip information of a processing chip of the video playing terminal.
And step 512, determining corresponding controller configuration parameters according to the chip information.
And 514, issuing the controller configuration parameters to configure a processing chip as a controller based on the controller configuration information, wherein the controller is used for converting the optimization description information into a chip control instruction and controlling an optimization component to perform optimization processing on the video data through the chip control instruction.
It should be noted that, in this embodiment of the present application, steps 510 to 514 are only for explaining a configuration process of a processing chip of a terminal by a server, and do not represent that there is an execution sequence between the step and other steps, for example, steps 510 to 514 may also be executed before step 502 in this embodiment of the present application.
In the embodiment of the application, the server can analyze the video data by using the calculation power of the server, determine the content to be optimized in the video data as the optimization description information, package the optimization description information into information according to the packaging rule, transcode the video data into a code stream file, and then determine the corresponding video transmission information according to the signaling and the code stream file. And then, the server transmits the video transmission information to the video playing terminal, after the video playing terminal receives the video transmission information, the video playing terminal can analyze a code stream file in the video transmission information to obtain video data, can analyze a signaling in the video transmission information, determines a packaging rule of the signaling, and extracts the optimization description information according to the packaging rule. And then, optimizing the video data by using the optimization description information to obtain optimized video data, and playing the optimized video data. The server side can obtain chip information of a processing chip of the video playing terminal, determines corresponding controller configuration parameters according to the chip information, then sends the controller configuration parameters to the video playing terminal, and after the video playing terminal receives the controller configuration parameters, the chip is configured to obtain a controller, and the controller is used for converting the optimization description information into a chip control instruction and controlling the optimization component to optimize video data through the chip control instruction. In the embodiment of the application, the video data can be analyzed by utilizing the large calculation power of the server, the content to be optimized is determined to be used as the optimization description information, and compared with the method for analyzing the video data by adopting the small calculation power of the video playing terminal, the data quality of the video data can be improved. In addition, the optimization description information is added to the video transmission information, the influence on the file size of the video transmission information is small, and compared with a mode of analyzing and optimizing video data at a server, the data transmission efficiency can be guaranteed.
On the basis of the above embodiment, the embodiment of the present application further provides a data processing method, which can be applied to a live video playing terminal, where the live video playing terminal may include an intelligent terminal, such as a smart television, a projector, a mobile phone, a computer, and the like. The method can determine the optimized description information aiming at the live video data by using the calculation power of the server, and optimizes the live video data according to the optimized description information at the live video playing terminal, so as to improve the data quality (image quality, audio quality and the like) of the live data and improve the user experience of a user watching the live broadcast, and specifically, as shown in fig. 6, the method comprises the following steps:
step 602, receiving live video transmission information.
And step 604, analyzing the live video transmission information to obtain live video data and optimized description information.
And 606, optimizing the live video data according to the optimization description information.
And 608, playing the optimized live video data.
The implementation of the data processing method in the embodiment of the present application is similar to that of the method in the embodiment described above, and specific implementation may refer to the implementation of the embodiment described above, and details are not described here again.
In the embodiment of the application, the server can analyze the live video data by using the calculation power of the server, determine the content to be optimized in the live video data to serve as the optimization description information, and transcode the optimization description information and the live video data into the live video transmission information. And then the server transmits the live video transmission information to a live video playing terminal, the live video playing terminal can analyze the live video transmission information to obtain live video data and optimized description information, then the live video data is optimized according to the optimized description information, and the optimized live video data is played. In the embodiment of the application, the server side can be used for analyzing the live video data with stronger calculation power, the content to be optimized is determined to serve as the optimization description information, and compared with the method for analyzing the live video data with low calculation power by adopting the live video playing terminal, the data quality of the live video data can be improved, and the user experience of a user watching the live video is further improved.
On the basis of the above embodiments, the embodiments of the present application further provide a data processing method, which can be applied to a server, and the server can be understood as a device that stores live video data or transfers live video data. The method can determine the optimized description information aiming at the live video data by using the calculation power of the server, optimizes the live video data according to the optimized description information at the live video playing terminal, further improves the data quality (image quality, audio quality and the like) of the live data, and improves the user experience of a user watching the live broadcast, as shown in fig. 7, the method comprises the following steps:
step 702, acquiring live video data, analyzing the live video data, and determining optimized description information.
And 704, packaging the optimization description information and the live video data to obtain live video transmission information, and transmitting the live video transmission information to a live video playing terminal, wherein the live video playing terminal is used for optimizing the live video data according to the optimization description information and playing and outputting the optimized live video data.
The implementation of the data processing method in the embodiment of the present application is similar to that of the method in the embodiment described above, and specific implementation may refer to the implementation of the embodiment described above, and details are not described here again.
In the embodiment of the application, the server can analyze the live video data by using the computing power of the server, determine the content to be optimized in the live video data to serve as the optimization description information, and transcode the optimization description information and the live video data into the live video transmission information. And then the server transmits the live video transmission information to a live video playing terminal, the live video playing terminal can analyze the live video transmission information to obtain live video data and optimized description information, then the live video data is optimized according to the optimized description information, and the optimized live video data is played. In the embodiment of the application, the server side can be used for analyzing the live video data with stronger calculation power, the content to be optimized is determined to serve as the optimization description information, and compared with the method for analyzing the live video data with low calculation power by adopting the live video playing terminal, the data quality of the live video data can be improved, and the user experience of a user watching the live video is further improved.
On the basis of the above embodiments, the present application further provides a data processing method, which can be applied to a surveillance video playing terminal, where the surveillance video playing terminal may include an intelligent terminal, such as an intelligent television, a projector, a mobile phone, and a computer. The method can determine the optimization description information aiming at the monitoring video data by using the calculation power of the server, and optimizes the monitoring video data according to the optimization description information at the monitoring video playing terminal, so as to improve the data quality (image quality, audio quality and the like) of the monitoring data, and better monitor, and specifically, as shown in fig. 8, the method comprises the following steps:
step 802, receiving the transmission information of the monitoring video.
And step 804, analyzing the monitoring video transmission information to obtain monitoring video data and optimization description information.
And 806, optimizing the monitoring video data according to the optimization description information.
And 808, playing the optimized monitoring video data.
The implementation of the data processing method in the embodiment of the present application is similar to that of the method in the embodiment described above, and specific implementation may refer to the implementation of the embodiment described above, and details are not described here again.
In the embodiment of the application, the server side can analyze the monitoring video data by using the calculation power of the server side, determine the content to be optimized in the monitoring video data to serve as the optimization description information, and transcode the optimization description information and the monitoring video data into the monitoring video transmission information. And then the server transmits the monitoring video transmission information to the monitoring video playing terminal, the monitoring video playing terminal can analyze the monitoring video transmission information to obtain monitoring video data and optimization description information, then the monitoring video data is optimized according to the optimization description information, and the optimized monitoring video data is played. In the embodiment of the application, the monitoring video data can be analyzed by utilizing stronger calculation power of the server, the content to be optimized is determined as the optimization description information, and compared with the method that the monitoring video data is analyzed by adopting the small calculation power of the monitoring video playing terminal, the data quality of the monitoring video data can be improved, and then better monitoring is carried out.
On the basis of the above embodiments, the embodiments of the present application further provide a data processing method, which can be applied to a server, and the server can be understood as a device for storing monitoring video data or transferring monitoring video data. The method can determine the optimization description information aiming at the monitoring video data by using the calculation power of the server, and optimizes the monitoring video data according to the optimization description information at the monitoring video playing terminal, so as to improve the data quality (image quality, audio quality and the like) of the monitoring data, and better monitor, and specifically, as shown in fig. 9, the method comprises the following steps:
and 902, acquiring the monitoring video data, analyzing the monitoring video data and determining the optimization description information.
And 904, packaging the optimization description information and the monitoring video data to obtain monitoring video transmission information, and transmitting the monitoring video transmission information to a monitoring video playing terminal, wherein the monitoring video playing terminal is used for optimizing the monitoring video data according to the optimization description information and playing and outputting the optimized monitoring video data.
The implementation of the data processing method in the embodiment of the present application is similar to that of the method in the embodiment described above, and specific implementation may refer to the implementation of the embodiment described above, and details are not described here again.
In the embodiment of the application, the server side can analyze the monitoring video data by using the calculation power of the server side, determine the content to be optimized in the monitoring video data to serve as the optimization description information, and transcode the optimization description information and the monitoring video data into the monitoring video transmission information. And then the server transmits the monitoring video transmission information to the monitoring video playing terminal, the monitoring video playing terminal can analyze the monitoring video transmission information to obtain monitoring video data and optimization description information, then the monitoring video data is optimized according to the optimization description information, and the optimized monitoring video data is played. In the embodiment of the application, the monitoring video data can be analyzed by utilizing stronger calculation power of the server, the content to be optimized is determined as the optimization description information, and compared with the method that the monitoring video data is analyzed by adopting the small calculation power of the monitoring video playing terminal, the data quality of the monitoring video data can be improved, and then better monitoring is carried out.
On the basis of the above embodiments, the present application further provides a data processing method, which can be applied to a video playing terminal, where the video playing terminal may include an intelligent terminal, such as a smart television, a projector, a mobile phone, and a computer. According to the method, one or more paths of video data in the multiple paths of video data can be analyzed by using the calculation power of the server, corresponding optimization description information is determined and is sent to the video playing terminal, the video playing terminal performs optimization processing on target sub-video data according to the optimization description information, and the data quality of key video data in the multiple paths of video data is improved, specifically, as shown in fig. 10, the method comprises the following steps:
step 1002, receiving video transmission information.
Step 1004, analyzing the video transmission information to obtain video data and corresponding optimization description information, wherein the video data comprises multiple paths of sub-video data.
And 1006, optimizing the target sub-video data in the video data according to the optimization description information.
And step 1008, playing the optimized video data.
The implementation of the data processing method in the embodiment of the present application is similar to that of the method in the embodiment described above, and specific implementation may refer to the implementation of the embodiment described above, and details are not described here again.
In the embodiment of the present application, the multi-channel sub-video data may be understood as video data acquired from different data sources, for example, video data acquired by a plurality of different image capturing apparatuses may be used as the multi-channel sub-video data. The multiple paths of sub-video data can be played in the same page respectively, and can also be analyzed and combined according to the spatial position (coordinate) of each sub-video data to form a scene video for playing. The server side can analyze the target sub-video data in the multi-channel sub-video data by utilizing the calculation power of the server side, determine the content to be optimized in the target sub-video data to serve as optimization description information, and transcode the optimization description information and the multi-channel video data into video transmission information. And then the server transmits the video transmission information to the video playing terminal, the video playing terminal can analyze the video transmission information to obtain multi-path sub-video data and optimization description information, then the target sub-video data is optimized according to the optimization description information, and the optimized video data is played. In the embodiment of the present application, optimization processing may be performed on one (or several) paths of sub-video data in multiple paths of sub-video data.
In this embodiment of the present application, virtual structure data may be added to sub-video data to achieve a combined effect of reality and virtual, and specifically, as an optional embodiment, the optimizing the target sub-video data in the video data according to the optimization description information includes: acquiring virtual structure data according to the optimization description information; and adding the virtual structure data into the target sub-video data to obtain the optimized target sub-video data. The virtual structure data can be understood as virtual information in an Augmented Reality technology, wherein the Augmented Reality (AR) technology is a technology for skillfully fusing virtual information and a real world, a plurality of technical means such as multimedia, three-dimensional modeling, real-time tracking and registration, intelligent interaction, sensing and the like are widely applied, virtual information such as characters, images, three-dimensional models, music, videos and the like generated by a computer are applied to the real world after being simulated, and the two kinds of information complement each other, so that the real world is enhanced. In the embodiment, the virtual structure data can be added to one path of sub-video data in the multi-path of sub-video data, so that the virtual structure data is combined with reality, and a better watching experience can be given to a user watching videos.
On the basis of the above embodiments, the embodiments of the present application further provide a data processing method, which can be applied to a server, and the server can be understood as a device for storing video data or transferring video data. The method can analyze one or more paths of video data in the multiple paths of video data by using the computing power of the server, determine corresponding optimization description information, and send the optimization description information to the video playing terminal, and the video playing terminal performs optimization processing on target sub-video data according to the optimization description information to improve the data quality of key video data in the multiple paths of video data, and specifically, as shown in fig. 11, the method comprises the following steps:
step 1102, obtaining multiple paths of sub-video data to determine video data.
And 1104, analyzing the target sub-video data and determining corresponding optimization description information.
And 1106, packaging the optimization description information and the video data to obtain video transmission information, and transmitting the video transmission information to a video playing terminal, wherein the video playing terminal is used for optimizing target sub-video data in the video data according to the optimization description information and playing and outputting the optimized live video data.
The implementation of the data processing method in the embodiment of the present application is similar to that of the method in the embodiment described above, and specific implementation may refer to the implementation of the embodiment described above, and details are not described here again.
In the embodiment of the application, the server can analyze the target sub-video data in the multi-channel sub-video data by using the calculation power of the server, determine the content to be optimized in the target sub-video data as the optimization description information, and transcode the optimization description information and the multi-channel video data into the video transmission information. And then the server transmits the video transmission information to the video playing terminal, the video playing terminal can analyze the video transmission information to obtain multi-path sub-video data and optimization description information, then the target sub-video data is optimized according to the optimization description information, and the optimized video data is played. In the embodiment of the present application, optimization processing may be performed on one (or several) paths of sub-video data in multiple paths of sub-video data.
On the basis of the above embodiments, the present application further provides a data processing method, which can be applied to a video playing terminal, where the video playing terminal may include an intelligent terminal, such as a smart television, a projector, a mobile phone, and a computer. The method can provide an optimization mode selection page, a user can select a corresponding optimization mode in the optimization mode selection page, determine selection information and send the selection information to the server, and the server processes video data in a corresponding optimization mode according to the selection information, specifically, as shown in fig. 12, the method includes:
step 1202, providing an optimization mode selection page, wherein the optimization mode selection page comprises a plurality of optimization selection controls corresponding to different optimization modes.
And 1204, determining selection information according to the operation instruction of the optimization selection control, and transmitting the selection information to the server, so that the server processes the video data by adopting a corresponding optimization mode according to the selection information.
The implementation of the data processing method in the embodiment of the present application is similar to that of the method in the embodiment described above, and specific implementation may refer to the implementation of the embodiment described above, and details are not described here again.
The method and the device for optimizing the video playing terminal can provide the optimizing mode selection page, the optimizing mode selection page can be generated and displayed in the video playing terminal, and the optimizing mode selection page can also be generated in the server and transmitted to the video playing terminal for displaying. Specifically, as an optional embodiment, the determining selection information according to an operation instruction for the optimization selection control includes: receiving a key instruction sent by a key of the target remote control equipment; and determining an operation instruction for the optimized selection control according to the key instruction so as to determine corresponding selection information. The target remote control device can be a remote controller, a mobile phone and the like, a user can send a key instruction to the video playing terminal through keys of the remote control device, and the video playing terminal determines a corresponding operation instruction according to the key instruction and obtains corresponding selection information. In the embodiment of the application, the user can select a more suitable optimization mode by himself to optimize data, and user experience is improved. Wherein, the optimization mode can comprise at least one of the following modes: the first, second, third and fourth aspects.
Specifically, the first method is that the server analyzes the acquired video data to determine optimization description information, packages the optimization description information and the video data into video transmission information, and sends the video transmission information to a video playing terminal, where the video playing terminal is configured to optimize the video data according to the optimization description information and play the optimized video data. The second mode is that the server divides the acquired video data, determines first divided video data and second divided video data, optimizes the first divided video data, analyzes the second divided video data, determines optimized description information of the second divided video data, packages the optimized description information, the second divided video data and the optimized first divided video data into video transmission information, and sends the video transmission information to the video playing terminal, and the video playing terminal is used for optimizing the second divided video data according to the optimized description information and playing the optimized first divided video data and the optimized second divided video data. The third mode is that the server encapsulates the acquired video data into video transmission information and sends the video transmission information to the video playing terminal, and the video playing terminal is used for analyzing the video data, determining the optimization description information, and performing optimization processing on the video data according to the optimization description information so as to output the video data after the optimization processing. The fourth mode is that the server encapsulates the acquired video data into video transmission information and sends the video transmission information to the video playing terminal, and the video playing terminal is used for analyzing the video transmission information, determining the video data and outputting the video data.
Briefly, the first method is to analyze video data at a server and optimize the video data at a video playing terminal according to an analysis result (optimization description information). The second mode is that a part of video data is optimized at the service terminal, and the other part of video data is optimized at the video playing terminal. The third mode is that the service end does not analyze and optimize the video data, and the video playing terminal analyzes and optimizes the video data. The fourth mode means that neither the server nor the video playing terminal analyzes and optimizes the video data.
In addition, the video data before and after optimization can be compared and displayed, and a user can conveniently know the difference between the video data before and after optimization. Specifically, as an optional embodiment, the method further includes: providing an image quality comparison page; acquiring optimized first video data and second video data before optimization; and playing the first video data and the second video data on an image quality comparison page for comparison. The optimized first video data and the optimized second video data can be displayed in the opposite page of the image quality, so that a user can know the difference of the image quality conveniently.
On the basis of the above embodiments, the embodiments of the present application further provide a data processing method, which can be applied to a server, and the server can be understood as a device for storing video data or transferring video data. The method can provide an optimization mode selection page, a user can select a corresponding optimization mode in the optimization mode selection page, determine selection information and send the selection information to the server, and the server processes video data in a corresponding optimization mode according to the selection information, specifically, as shown in fig. 13, the method includes:
step 1302, receiving selection information, wherein the selection information is determined according to an optimization mode selection page.
And 1304, determining a target optimization mode aiming at the video data according to the selection information.
And 1306, correspondingly processing the acquired video data according to the target optimization mode.
The implementation of the data processing method in the embodiment of the present application is similar to that of the method in the embodiment described above, and specific implementation may refer to the implementation of the embodiment described above, and details are not described here again.
The method and the device for optimizing the video playing terminal can provide the optimizing mode selection page, the optimizing mode selection page can be generated and displayed in the video playing terminal, and the optimizing mode selection page can also be generated in the server and transmitted to the video playing terminal for displaying. The optimization mode selection page comprises a plurality of optimization selection controls corresponding to different optimization modes, and a user can select the optimization selection controls through a remote controller and other equipment. In the embodiment of the application, the user can select a more suitable optimization mode by himself to optimize data, and user experience is improved. Wherein, the optimization mode can comprise at least one of the following modes: the first, second, third and fourth aspects. Specifically, as an optional embodiment, the correspondingly processing the acquired video data according to the target optimization manner includes at least one of the following steps: when the target optimization mode is the first mode, the server analyzes the acquired video data to determine optimization description information, packages the optimization description information and the video data into video transmission information, and sends the video transmission information to a video playing terminal, wherein the video playing terminal is used for optimizing the video data according to the optimization description information and playing the optimized video data; when the target optimization mode is a second mode, the server splits the acquired video data, determines first split video data and second split video data, optimizes the first split video data, analyzes the second split video data, determines optimized description information of the second split video data, packages the optimized description information, the second split video data and the optimized first split video data into video transmission information, and sends the video transmission information to a video playing terminal, wherein the video playing terminal is used for optimizing the second split video data according to the optimized description information and playing the optimized first split video data and the optimized second split video data; when the target optimization mode is the third mode, the server side packages the acquired video data into video transmission information and sends the video transmission information to the video playing terminal, and the video playing terminal is used for analyzing the video data, determining optimization description information, and optimizing the video data according to the optimization description information to output the optimized video data; and when the target optimization mode is the fourth mode, the server encapsulates the acquired video data into video transmission information and sends the video transmission information to the video playing terminal, and the video playing terminal is used for analyzing the video transmission information, determining the video data and outputting the video data.
Briefly, the first method is to analyze video data at a server and optimize the video data at a video playing terminal according to an analysis result (optimization description information). The second mode is that a part of video data is optimized at the service terminal, and the other part of video data is optimized at the video playing terminal. The third mode is that the service end does not analyze and optimize the video data, and the video playing terminal analyzes and optimizes the video data. The fourth mode means that neither the server nor the video playing terminal analyzes and optimizes the video data.
It should be noted that, for simplicity of description, the method embodiments are described as a series of acts or combination of acts, but those skilled in the art will recognize that the embodiments are not limited by the order of acts described, as some steps may occur in other orders or concurrently depending on the embodiments. Further, those skilled in the art will also appreciate that the embodiments described in the specification are presently preferred and that no particular act is required of the embodiments of the application.
On the basis of the foregoing embodiment, this embodiment further provides a data processing apparatus, as shown in fig. 14, which may specifically include the following modules:
a transmission information receiving module 1402, configured to receive video transmission information.
A transmission information analyzing module 1404, configured to analyze the video transmission information to obtain video data and optimization description information.
And the video data optimization module 1406 is configured to perform optimization processing on the video data according to the optimization description information.
And a video data playing module 1408, configured to play the optimized video data.
In summary, in the embodiment of the present application, the server may analyze the video data by using the calculation power of the server itself, determine the content to be optimized in the video data to be used as the optimization description information, and transcode the optimization description information and the video data into the video transmission information. And then the server transmits the video transmission information to the video playing terminal, the video playing terminal can analyze the video transmission information to obtain video data and optimized description information, then the video data is optimized according to the optimized description information, and the optimized video data is played. In the embodiment of the application, the video data can be analyzed by utilizing the large calculation power of the server, the content to be optimized is determined to be used as the optimization description information, and compared with a mode of analyzing the video data by adopting the small calculation power of a video playing terminal, the mode of the embodiment of the application can improve the data quality of the video data. In addition, the optimization description information is added to the video transmission information, the influence on the file size of the video transmission information is small, and compared with a mode of analyzing and optimizing video data at a server, the data transmission efficiency can be guaranteed.
On the basis of the foregoing embodiment, this embodiment further provides a data processing apparatus, which may specifically include the following modules:
and the information receiving and processing module is used for receiving video transmission information, wherein the video data comprises image data and audio data.
And the code stream file analyzing and processing module is used for analyzing the code stream file in the video transmission information to obtain video data.
And the encapsulation rule identification processing module is used for identifying the rule identification information of the signaling in the video transmission information and determining the encapsulation rule of the signaling.
And the optimization information extraction processing module is used for extracting the optimization description information according to the packaging rule. Wherein the optimization description information includes first optimization information for optimizing the image data and second optimization information for optimizing the audio data. In addition, in another optional embodiment, the server may further align the optimization description information with the video data according to a timestamp in the server, and then insert the optimization description information into the code stream file, so that the terminal may parse the code stream file to obtain the video data and the optimization description information, and specifically, the transmission information parsing module specifically includes: and analyzing a code stream file in the video transmission information to obtain video data and optimization description information, wherein the optimization description information is packaged in the code stream file.
And the image data optimization processing module is used for matching the first optimization information with the image data according to the image data and the time stamp of the first optimization information, and performing image optimization processing on the image data. As an optional embodiment, the image data optimization processing module specifically includes at least one of the following modules: the image sharpening processing submodule is used for sharpening the image data according to the image sharpening parameter; the image saturation adjusting processing submodule is used for adjusting the saturation of the image data according to the image saturation parameter; the image contrast adjustment processing submodule is used for adjusting the contrast of the image data according to the image contrast parameter; the image denoising processing submodule is used for denoising the image data according to the image denoising parameters; the image deblocking processing submodule is used for deblocking the image data according to the image deblocking parameters; the image scene adjusting and processing submodule is used for carrying out scene optimization processing on the image data according to the image scene characteristic information; the image semantic adjustment processing submodule is used for carrying out semantic optimization processing on the image data according to the image semantic feature information; the image background adjusting and processing submodule is used for carrying out background optimization processing on the image data according to the image background characteristic information; and the image depth of field adjusting processing submodule is used for carrying out depth of field optimization processing on the image data according to the image depth of field characteristic information.
And the audio data optimization processing module is used for matching the second optimization information with the audio data according to the audio data and the time stamps of the second optimization information, and performing audio optimization processing on the audio data. As an optional embodiment, the audio data optimization processing module includes at least one of the following modules: the audio waveform adjusting and processing submodule is used for adjusting the waveform of the audio data according to the audio waveform parameters; the audio filtering processing submodule is used for carrying out filtering processing on the audio data according to the audio filtering parameters; the audio balance adjustment processing submodule is used for carrying out balance processing on the audio data according to the audio balance parameters; and the audio scene adjusting processing submodule is used for carrying out scene optimization processing on the audio data according to the audio scene characteristic information.
In addition, in this embodiment of the present application, the video data may also be optimized by using hardware of the video playing terminal, and specifically, as an optional embodiment, the video data optimization module specifically includes: and inputting the optimization description information into a controller, wherein the controller is used for converting the optimization description information into a chip control instruction and controlling an optimization component to perform optimization processing on video data through the chip control instruction. Specifically, as an optional embodiment, the apparatus further includes: the processing chip configuration module is used for acquiring configuration information of the controller; and configuring the processing chip as the controller based on the controller configuration information.
And the optimized data playing processing module is used for playing the optimized video data.
In the embodiment of the application, the server side can analyze the video data by using the calculation power of the server side, determine the content to be optimized in the video data as the optimization description information, and transcode the optimization description information and the video data into the video transmission information. Then the server transmits the video transmission information to the video playing terminal, after the video playing terminal receives the video transmission information, the video playing terminal can analyze a code stream file in the video transmission information to obtain video data, can analyze a signaling in the video transmission information, determine an encapsulation rule of the signaling, and extract optimization description information according to the encapsulation rule, wherein the optimization description information can include first optimization information corresponding to image data and second optimization information corresponding to audio data for video playing. After the terminal determines the optimization description information, the video playing terminal can match the first optimization information with the image data according to the time stamp and perform optimization processing on the image data, can match the second optimization information with the audio data according to the time stamp and perform optimization processing on the audio data to obtain optimized video data, and plays the optimized video data. In the embodiment of the application, the video data can be analyzed by utilizing the large calculation power of the server, the content to be optimized is determined to be used as the optimization description information, and compared with the method for analyzing the video data by adopting the small calculation power of the video playing terminal, the data quality of the video data can be improved. In addition, the optimization description information is added to the video transmission information, the influence on the file size of the video transmission information is small, and compared with a mode of analyzing and optimizing video data at a server, the data transmission efficiency can be guaranteed.
On the basis of the foregoing embodiment, this embodiment further provides a data processing apparatus, as shown in fig. 15, which may specifically include the following modules:
the video data analysis module 1502 is configured to analyze the video data and determine optimization description information.
And the transmission information packaging module 1504 is used for packaging the optimization description information and the video data to obtain video transmission information and transmitting the video transmission information to a terminal, and the terminal is used for optimizing the video data according to the optimization description information and playing and outputting the optimized video data.
In summary, in the embodiment of the present application, the server may analyze the video data by using the computational power of the server, determine the content to be optimized in the video data as the optimization description information, and transcode the optimization description information and the video data into the video transmission information. And then the server transmits the video transmission information to the terminal, the terminal can analyze the video transmission information to obtain video data and optimization description information, then the video data is optimized according to the optimization description information, and the optimized video data is played. In the embodiment of the application, the video data can be analyzed by utilizing the large calculation power of the server, the content to be optimized is determined to be used as the optimization description information, and compared with the method for analyzing the video data by adopting the small calculation power of the terminal, the data quality of the video data can be improved. In addition, the optimization description information is added to the video transmission information, the influence on the file size of the video transmission information is small, and compared with a mode of analyzing and optimizing video data at a server, the data transmission efficiency can be guaranteed.
On the basis of the foregoing embodiment, this embodiment further provides a data processing apparatus, which may specifically include the following modules:
and the video data analysis processing module is used for analyzing the video data and determining the optimization description information.
And the optimization information packaging processing module is used for packaging the optimization description information into a signaling according to a packaging rule.
And the video data packaging processing module is used for packaging the video data into a code stream file.
And the transmission information packaging processing module is used for determining video transmission information according to the signaling and the code stream file and transmitting the video transmission information to the video playing terminal, and the video playing terminal is used for optimizing video data according to the optimization description information and playing and outputting the optimized video data.
And the chip information acquisition module is used for acquiring the chip information of the processing chip of the video playing terminal.
And the configuration parameter determining module is used for determining corresponding controller configuration parameters according to the chip information.
And the configuration parameter issuing module is used for issuing the configuration parameters of the controller so as to configure a processing chip into the controller based on the configuration information of the controller, and the controller is used for converting the optimization description information into a chip control instruction and controlling an optimization component to optimize video data through the chip control instruction.
In the embodiment of the application, the server can analyze the video data by using the calculation power of the server, determine the content to be optimized in the video data as the optimization description information, package the optimization description information into information according to the packaging rule, transcode the video data into a code stream file, and then determine the corresponding video transmission information according to the signaling and the code stream file. And then, the server transmits the video transmission information to the video playing terminal, after the video playing terminal receives the video transmission information, the video playing terminal can analyze a code stream file in the video transmission information to obtain video data, can analyze a signaling in the video transmission information, determines a packaging rule of the signaling, and extracts the optimization description information according to the packaging rule. And then, optimizing the video data by using the optimization description information to obtain optimized video data, and playing the optimized video data. The server side can obtain chip information of a processing chip of the video playing terminal, determines corresponding controller configuration parameters according to the chip information, then sends the controller configuration parameters to the video playing terminal, and after the video playing terminal receives the controller configuration parameters, the chip is configured to obtain a controller, and the controller is used for converting the optimization description information into a chip control instruction and controlling the optimization component to optimize video data through the chip control instruction. In the embodiment of the application, the video data can be analyzed by utilizing the large calculation power of the server, the content to be optimized is determined to be used as the optimization description information, and compared with the method for analyzing the video data by adopting the small calculation power of the video playing terminal, the data quality of the video data can be improved. In addition, the optimization description information is added to the video transmission information, the influence on the file size of the video transmission information is small, and compared with a mode of analyzing and optimizing video data at a server, the data transmission efficiency can be guaranteed.
On the basis of the foregoing embodiment, this embodiment further provides a data processing apparatus, as shown in fig. 16, which may specifically include the following modules:
and a live transmission information receiving module 1604, configured to receive live video transmission information.
And a live broadcast transmission information analyzing module 1604, configured to analyze the live broadcast video transmission information to obtain live broadcast video data and optimized description information.
And a live video data optimization module 1606, configured to perform optimization processing on the live video data according to the optimization description information.
A live video data playing module 1608, configured to play the optimized live video data.
In summary, in the embodiment of the present application, the server may analyze the live video data by using the computing power of the server itself, determine the content to be optimized in the live video data, use the content as the optimization description information, and transcode the optimization description information and the live video data into the live video transmission information. And then the server transmits the live video transmission information to a live video playing terminal, the live video playing terminal can analyze the live video transmission information to obtain live video data and optimized description information, then the live video data is optimized according to the optimized description information, and the optimized live video data is played. In the embodiment of the application, the server side can be used for analyzing the live video data with stronger calculation power, the content to be optimized is determined to serve as the optimization description information, and compared with the method for analyzing the live video data with low calculation power by adopting the live video playing terminal, the data quality of the live video data can be improved, and the user experience of a user watching the live video is further improved.
On the basis of the foregoing embodiment, this embodiment further provides a data processing apparatus, as shown in fig. 17, which may specifically include the following modules:
the live video data analysis module 1702 is configured to obtain live video data, analyze the live video data, and determine optimization description information.
And a live broadcast transmission information packaging module 1704, configured to package the optimization description information and the live broadcast video data to obtain live broadcast video transmission information, and transmit the live broadcast video transmission information to a live broadcast video playing terminal, where the live broadcast video playing terminal is configured to optimize the live broadcast video data according to the optimization description information, and play and output the optimized live broadcast video data.
In summary, in the embodiment of the present application, the server may analyze the live video data by using the computing power of the server, determine the content to be optimized in the live video data, use the content as the optimization description information, and transcode the optimization description information and the live video data into the live video transmission information. And then the server transmits the live video transmission information to a live video playing terminal, the live video playing terminal can analyze the live video transmission information to obtain live video data and optimized description information, then the live video data is optimized according to the optimized description information, and the optimized live video data is played. In the embodiment of the application, the server side can be used for analyzing the live video data with stronger calculation power, the content to be optimized is determined to serve as the optimization description information, and compared with the method for analyzing the live video data with low calculation power by adopting the live video playing terminal, the data quality of the live video data can be improved, and the user experience of a user watching the live video is further improved.
On the basis of the foregoing embodiment, this embodiment further provides a data processing apparatus, as shown in fig. 18, which may specifically include the following modules:
and a monitoring transmission information receiving module 1802, configured to receive monitoring video transmission information.
And a monitoring transmission information analyzing module 1804, configured to analyze the monitoring video transmission information to obtain monitoring video data and optimization description information.
And a monitoring video data optimizing module 1806, configured to perform optimization processing on the monitoring video data according to the optimization description information.
And a monitoring video data playing module 1808, configured to play the optimized monitoring video data.
In summary, in the embodiment of the present application, the server may analyze the surveillance video data by using the calculation power of the server itself, determine the content to be optimized in the surveillance video data, use the content as the optimization description information, and transcode the optimization description information and the surveillance video data into the surveillance video transmission information. And then the server transmits the monitoring video transmission information to the monitoring video playing terminal, the monitoring video playing terminal can analyze the monitoring video transmission information to obtain monitoring video data and optimization description information, then the monitoring video data is optimized according to the optimization description information, and the optimized monitoring video data is played. In the embodiment of the application, the monitoring video data can be analyzed by utilizing stronger calculation power of the server, the content to be optimized is determined as the optimization description information, and compared with the method that the monitoring video data is analyzed by adopting the small calculation power of the monitoring video playing terminal, the data quality of the monitoring video data can be improved, and then better monitoring is carried out.
On the basis of the foregoing embodiment, this embodiment further provides a data processing apparatus, as shown in fig. 19, which may specifically include the following modules:
the monitoring video data analyzing module 1902 is configured to obtain monitoring video data, analyze the monitoring video data, and determine optimization description information.
And a monitoring transmission information encapsulation module 1904, configured to encapsulate the optimization description information and the monitoring video data to obtain monitoring video transmission information, and transmit the monitoring video transmission information to a monitoring video playing terminal, where the monitoring video playing terminal is configured to optimize the monitoring video data according to the optimization description information, and play and output the optimized monitoring video data.
In summary, in the embodiment of the present application, the server may analyze the surveillance video data by using the calculation power of the server itself, determine the content to be optimized in the surveillance video data, use the content as the optimization description information, and transcode the optimization description information and the surveillance video data into the surveillance video transmission information. And then the server transmits the monitoring video transmission information to the monitoring video playing terminal, the monitoring video playing terminal can analyze the monitoring video transmission information to obtain monitoring video data and optimization description information, then the monitoring video data is optimized according to the optimization description information, and the optimized monitoring video data is played. In the embodiment of the application, the monitoring video data can be analyzed by utilizing stronger calculation power of the server, the content to be optimized is determined as the optimization description information, and compared with the method that the monitoring video data is analyzed by adopting the small calculation power of the monitoring video playing terminal, the data quality of the monitoring video data can be improved, and then better monitoring is carried out.
On the basis of the foregoing embodiment, this embodiment further provides a data processing apparatus, as shown in fig. 20, which may specifically include the following modules:
a transmission information obtaining module 2002 for receiving the video transmission information.
A sub-video data obtaining module 2004, configured to analyze the video transmission information to obtain video data and corresponding optimization description information, where the video data includes multiple paths of sub-video data.
And the sub-video data optimization module 2006 is configured to perform optimization processing on target sub-video data in the video data according to the optimization description information.
And a multi-channel video playing module 2008, configured to play the optimized video data.
In summary, in the embodiment of the present application, multiple paths of sub-video data may be played in the same page, or multiple paths of sub-video data may be analyzed and combined according to the spatial position (coordinate) of each sub-video data to form a scene video for playing. The server side can analyze the target sub-video data in the multi-channel sub-video data by utilizing the calculation power of the server side, determine the content to be optimized in the target sub-video data to serve as optimization description information, and transcode the optimization description information and the multi-channel video data into video transmission information. And then the server transmits the video transmission information to the video playing terminal, the video playing terminal can analyze the video transmission information to obtain multi-path sub-video data and optimization description information, then the target sub-video data is optimized according to the optimization description information, and the optimized video data is played. In the embodiment of the present application, optimization processing may be performed on one (or several) paths of sub-video data in multiple paths of sub-video data.
In this embodiment of the application, virtual structure data may be added to the sub-video data to achieve a combined effect of reality and virtual, and specifically, as an optional embodiment, the sub-video data optimization module 2006 specifically includes: acquiring virtual structure data according to the optimization description information; and adding the virtual structure data into the target sub-video data to obtain the optimized target sub-video data. The virtual structure data can be understood as virtual information in an augmented reality technology, and the virtual structure data can be added into one path of sub video data in the multi-path of sub video data, so that the virtual structure data is combined with reality, and a user watching videos can be provided with better watching experience.
On the basis of the foregoing embodiment, this embodiment further provides a data processing apparatus, as shown in fig. 21, which may specifically include the following modules:
a sub-video data obtaining module 2102 is configured to obtain multiple paths of sub-video data to determine video data.
And the sub-video data analysis module 2104 is configured to analyze the target sub-video data and determine corresponding optimization description information.
The transmission information obtaining module 2106 is configured to encapsulate the optimization description information and the video data to obtain video transmission information, and transmit the video transmission information to the video playing terminal, where the video playing terminal is configured to optimize target sub-video data in the video data according to the optimization description information, and play and output the optimized live video data.
In summary, in the embodiment of the present application, the server may analyze the target sub-video data in the multi-channel sub-video data by using the calculation power of the server itself, determine the content to be optimized in the target sub-video data, use the content as the optimization description information, and transcode the optimization description information and the multi-channel video data into the video transmission information. And then the server transmits the video transmission information to the video playing terminal, the video playing terminal can analyze the video transmission information to obtain multi-path sub-video data and optimization description information, then the target sub-video data is optimized according to the optimization description information, and the optimized video data is played. In the embodiment of the present application, optimization processing may be performed on one (or several) paths of sub-video data in multiple paths of sub-video data.
On the basis of the foregoing embodiment, this embodiment further provides a data processing apparatus, as shown in fig. 22, which may specifically include the following modules:
a selection page providing module 2202, configured to provide an optimization mode selection page, where the optimization mode selection page includes multiple optimization selection controls corresponding to different optimization modes.
The selection information obtaining module 2204 is configured to determine selection information according to an operation instruction for the optimization selection control, and transmit the selection information to the server, so that the server processes the video data in a corresponding optimization manner according to the selection information.
In summary, the embodiment of the application can provide the optimization mode selection page, the optimization mode selection page can be generated and displayed in the video playing terminal, and the optimization mode selection page can also be generated in the server and transmitted to the video playing terminal for displaying. The optimization mode selection page includes a plurality of optimization selection controls corresponding to different optimization modes, and a user may select the optimization selection controls through a remote controller or other devices, specifically, as an optional embodiment, the selection information obtaining module 2204 specifically includes: receiving a key instruction sent by a key of the target remote control equipment; and determining an operation instruction for the optimized selection control according to the key instruction so as to determine corresponding selection information. The target remote control device can be a remote controller, a mobile phone and the like, a user can send a key instruction to the video playing terminal through keys of the remote control device, and the video playing terminal determines a corresponding operation instruction according to the key instruction and obtains corresponding selection information. In the embodiment of the application, the user can select a more suitable optimization mode by himself to optimize data, and user experience is improved. Wherein, the optimization mode can comprise at least one of the following modes: the first, second, third and fourth aspects.
Specifically, as an optional embodiment, the apparatus further includes: the image quality comparison processing module is used for providing an image quality comparison page; acquiring optimized first video data and second video data before optimization; and playing the first video data and the second video data on an image quality comparison page for comparison. The optimized first video data and the optimized second video data can be displayed in the opposite page of the image quality, so that a user can know the difference of the image quality conveniently.
On the basis of the foregoing embodiment, this embodiment further provides a data processing apparatus, as shown in fig. 23, which may specifically include the following modules:
and the selection information receiving module 2302 is used for receiving selection information, and the selection information is determined according to the optimization mode selection page.
An optimization mode obtaining module 2304, configured to determine a target optimization mode for the video data according to the selection information.
An optimized data obtaining module 2306, configured to perform corresponding processing on the obtained video data according to the target optimization mode.
In summary, the embodiment of the application can provide the optimization mode selection page, the optimization mode selection page can be generated and displayed in the video playing terminal, and the optimization mode selection page can also be generated in the server and transmitted to the video playing terminal for displaying. The optimization mode selection page comprises a plurality of optimization selection controls corresponding to different optimization modes, and a user can select the optimization selection controls through a remote controller and other equipment. In the embodiment of the application, the user can select a more suitable optimization mode by himself to optimize data, and user experience is improved. Wherein, the optimization mode can comprise at least one of the following modes: the first, second, third and fourth aspects. Specifically, as an optional embodiment, the optimized data obtaining module 2306 specifically includes at least one of the following modules: the first optimization processing module is used for analyzing the acquired video data by the server to determine optimization description information when the target optimization mode is the first mode, packaging the optimization description information and the video data into video transmission information, and sending the video transmission information to a video playing terminal, wherein the video playing terminal is used for optimizing the video data according to the optimization description information and playing the optimized video data; the second optimization processing module is used for segmenting the acquired video data, determining first segmentation video data and second segmentation video data, optimizing the first segmentation video data, analyzing the second segmentation video data, determining optimization description information of the second segmentation video data, packaging the optimization description information, the second segmentation video data and the optimized first segmentation video data into video transmission information, and sending the video transmission information to the video playing terminal, wherein the video playing terminal is used for optimizing the second segmentation video data according to the optimization description information and playing the optimized first segmentation video data and the optimized second segmentation video data; the third optimization processing module is used for packaging the acquired video data into video transmission information by the server side and sending the video transmission information to the video playing terminal when the target optimization mode is the third mode, wherein the video playing terminal is used for analyzing the video data, determining optimization description information, and optimizing the video data according to the optimization description information so as to output the video data after optimization processing; and the fourth optimization processing module is used for packaging the acquired video data into video transmission information by the server side and sending the video transmission information to the video playing terminal when the target optimization mode is the fourth mode, and the video playing terminal is used for analyzing the video transmission information, determining the video data and outputting the video data.
The present application further provides a non-transitory, readable storage medium, where one or more modules (programs) are stored, and when the one or more modules are applied to a device, the device may execute instructions (instructions) of method steps in this application.
Embodiments of the present application provide one or more machine-readable media having instructions stored thereon, which when executed by one or more processors, cause an electronic device to perform the methods as described in one or more of the above embodiments. In the embodiment of the application, the electronic device includes a server, a terminal device and other devices.
Embodiments of the present disclosure may be implemented as an apparatus, which may comprise a server (cluster), a terminal, etc., electronic device, using any suitable hardware, firmware, software, or any combination thereof, in a desired configuration. Fig. 24 schematically illustrates an example apparatus 2400 that can be used to implement various embodiments described herein.
For one embodiment, fig. 24 illustrates an exemplary apparatus 2400 having one or more processors 2402, a control module (chipset) 2404 coupled to at least one of the processor(s) 2402, a memory 2406 coupled to the control module 2404, a non-volatile memory (NVM)/storage 2408 coupled to the control module 2404, one or more input/output devices 2410 coupled to the control module 2404, and a network interface 2412 coupled to the control module 2404.
Processor 2402 may include one or more single-core or multi-core processors, and processor 2402 may include any combination of general-purpose or special-purpose processors (e.g., graphics processors, application processors, baseband processors, etc.). In some embodiments, the apparatus 2400 can serve as a server, a terminal, and other devices described in this embodiment.
In some embodiments, the apparatus 2400 may include one or more computer-readable media (e.g., the memory 2406 or the NVM/storage 2408) having instructions 2414 and one or more processors 2402 in combination with the one or more computer-readable media configured to execute the instructions 2414 to implement modules to perform the actions described in this disclosure.
For one embodiment, control module 2404 may include any suitable interface controllers to provide any suitable interface to at least one of the processor(s) 2402 and/or any suitable device or component in communication with control module 2404.
The control module 2404 may include a memory controller module to provide an interface to the memory 2406. The memory controller module may be a hardware module, a software module, and/or a firmware module.
Memory 2406 may be used, for example, to load and store data and/or instructions 2414 for device 2400. For one embodiment, memory 2406 may comprise any suitable volatile memory, such as suitable DRAM. In some embodiments, memory 2406 may comprise double data rate type four synchronous dynamic random access memory (DDR4 SDRAM).
For one embodiment, the control module 2404 may include one or more input/output controllers to provide an interface to the NVM/storage 2408 and input/output device(s) 2410.
For example, the NVM/storage 2408 may be used to store data and/or instructions 2414. NVM/storage 2408 may include any suitable non-volatile memory (e.g., flash memory) and/or may include any suitable non-volatile storage device(s) (e.g., one or more hard disk drive(s) (HDD (s)), one or more Compact Disc (CD) drive(s), and/or one or more Digital Versatile Disc (DVD) drive (s)).
NVM/storage 2408 may include storage resources as part of the device on which apparatus 2400 is installed, or it may be accessible by the device and may not necessarily be part of the device. For example, the NVM/storage 2408 may be accessed over a network via the input/output device(s) 2410.
Input/output device(s) 2410 may provide an interface for apparatus 2400 to communicate with any other suitable devices, and input/output device(s) 2410 may include communication components, audio components, sensor components, and so forth. Network interface 2412 may provide an interface for device 2400 to communicate over one or more networks, and device 2400 may communicate wirelessly with one or more components of a wireless network in accordance with any of one or more wireless network standards and/or protocols, e.g., access a communication standard-based wireless network such as WiFi, 2G, 3G, 4G, 5G, etc., or a combination thereof.
For one embodiment, at least one of the processor(s) 2402 may be packaged together with logic for one or more controller(s) (e.g., memory controller module) of the control module 2404. For one embodiment, at least one of the processor(s) 2402 may be packaged together with logic for one or more controller(s) of control module 2404 to form a System In Package (SiP). For one embodiment, at least one of the processor(s) 2402 may be integrated on the same die with logic for one or more controller(s) of the control module 2404. For one embodiment, at least one of the processor(s) 2402 may be integrated on the same die with logic for one or more controller(s) of control module 2404 to form a system on chip (SoC).
In various embodiments, device 2400 may be, but is not limited to being: a server, a desktop computing device, or a mobile computing device (e.g., a laptop computing device, a handheld computing device, a tablet, a netbook, etc.), among other terminal devices. In various embodiments, device 2400 may have more or fewer components and/or different architectures. For example, in some embodiments, device 2400 includes one or more cameras, keyboards, Liquid Crystal Display (LCD) screens (including touch screen displays), non-volatile memory ports, multiple antennas, graphics chips, Application Specific Integrated Circuits (ASICs), and speakers.
The detection device can adopt a main control chip as a processor or a control module, sensor data, position information and the like are stored in a memory or an NVM/storage device, a sensor group can be used as an input/output device, and a communication interface can comprise a network interface.
An embodiment of the present application further provides an electronic device, including: a processor; and a memory having executable code stored thereon that, when executed, causes the processor to perform a method as described in one or more of the embodiments of the application.
Embodiments of the present application also provide one or more machine-readable media having executable code stored thereon that, when executed, cause a processor to perform a method as described in one or more of the embodiments of the present application.
For the device embodiment, since it is basically similar to the method embodiment, the description is simple, and for the relevant points, refer to the partial description of the method embodiment.
The embodiments in the present specification are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other.
Embodiments of the present application are described with reference to flowchart illustrations and/or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing terminal to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing terminal to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing terminal to cause a series of operational steps to be performed on the computer or other programmable terminal to produce a computer implemented process such that the instructions which execute on the computer or other programmable terminal provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While preferred embodiments of the present application have been described, additional variations and modifications of these embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including the preferred embodiment and all such alterations and modifications as fall within the true scope of the embodiments of the application.
Finally, it should also be noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or terminal. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or terminal that comprises the element.
The foregoing detailed description has provided a data processing method, a data processing apparatus, an electronic device, and a storage medium, and the principles and embodiments of the present application are described herein using specific examples, which are only used to help understand the method and the core ideas of the present application; meanwhile, for a person skilled in the art, according to the idea of the present application, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present application.

Claims (39)

1. A data processing method is applied to a video playing terminal and comprises the following steps:
receiving video transmission information;
analyzing the video transmission information to obtain video data and optimized description information;
optimizing the video data according to the optimization description information;
and playing the optimized video data.
2. The method of claim 1, wherein parsing the video transmission information to obtain video data and optimization description information comprises:
analyzing a code stream file in video transmission information to obtain video data;
and analyzing the signaling in the video transmission information to obtain the optimized description information.
3. The method of claim 2, wherein parsing signaling in the video transmission information to obtain optimized description information comprises:
identifying the rule identification information of the signaling in the video transmission information, and determining the encapsulation rule of the signaling;
and extracting the optimization description information according to the packaging rule.
4. The method of claim 2, wherein the video data comprises image data and audio data, the optimization description information comprises first optimization information for optimizing the image data and second optimization information for optimizing the audio data, and the optimizing the video data according to the optimization description information comprises:
matching the first optimization information with the image data according to the time stamps of the image data and the first optimization information, and performing image optimization processing on the image data;
and matching the second optimization information with the audio data according to the time stamps of the audio data and the second optimization information, and performing audio optimization processing on the audio data.
5. The method of claim 4, the first optimization information comprising at least one of image parameter information and object feature information of an image, the performing image optimization processing on the image data comprising at least one of:
adjusting the parameters of the image according to the image parameter information;
and according to the object characteristic information of the image, optimizing the corresponding object in the image.
6. The method of claim 5, wherein adjusting the parameters of the image according to the image parameter information comprises at least one of:
sharpening the image data according to the image sharpening parameter;
adjusting the saturation of the image data according to the image saturation parameter;
adjusting the contrast of the image data according to the image contrast parameter;
denoising the image data according to the image denoising parameters;
and according to the image deblocking parameters, performing deblocking processing on the image data.
7. The method according to claim 5, wherein the optimizing the corresponding object in the image according to the object feature information of the image comprises at least one of the following steps:
performing scene optimization processing on the image data according to the image scene characteristic information;
performing semantic optimization processing on the image data according to the image semantic feature information;
performing background optimization processing on the image data according to the image background characteristic information;
and performing depth-of-field optimization processing on the image data according to the depth-of-field characteristic information of the image.
8. The method of claim 4, the audio optimization processing of audio data comprising at least one of:
adjusting the waveform of the audio data according to the audio waveform parameters;
according to the audio filtering parameters, carrying out filtering processing on the audio data;
according to the audio equalization parameters, performing equalization processing on the audio data;
and carrying out scene optimization processing on the audio data according to the audio scene characteristic information.
9. The method of claim 1, wherein parsing the video transmission information to obtain video data and optimization description information comprises:
and analyzing a code stream file in the video transmission information to obtain video data and optimization description information, wherein the optimization description information is packaged in the code stream file.
10. The method of claim 1, wherein the optimizing the video data according to the optimization description information comprises:
and inputting the optimization description information into a controller, wherein the controller is used for converting the optimization description information into a chip control instruction and controlling an optimization component to perform optimization processing on video data through the chip control instruction.
11. The method of claim 10, further comprising:
acquiring controller configuration information;
and configuring the processing chip as the controller based on the controller configuration information.
12. A data processing method is applied to a server and comprises the following steps:
analyzing the video data to determine optimization description information;
and packaging the optimization description information and the video data to obtain video transmission information, and transmitting the video transmission information to a video playing terminal, wherein the video playing terminal is used for optimizing the video data according to the optimization description information and playing and outputting the optimized video data.
13. The method of claim 12, wherein encapsulating the optimization description information and the video data to obtain video transmission information comprises:
according to the encapsulation rule, encapsulating the optimized description information into a signaling;
packaging the video data into a code stream file;
and determining video transmission information according to the signaling and the code stream file.
14. The method of claim 12, wherein encapsulating the optimization description information and the video data to obtain video transmission information comprises:
matching the video data with the optimization description information;
and packaging the video data into a code stream file, and adding the optimized description information matched with the code stream file into the code stream file to obtain video transmission information.
15. The method of claim 12, further comprising:
acquiring chip information of a processing chip of a video playing terminal;
determining corresponding controller configuration parameters according to the chip information;
and issuing the controller configuration parameters to configure a processing chip as a controller based on the controller configuration information, wherein the controller is used for converting the optimization description information into a chip control instruction and controlling an optimization component to perform optimization processing on the video data through the chip control instruction.
16. A data processing method is applied to a live video playing terminal and comprises the following steps:
receiving live video transmission information;
analyzing the live video transmission information to obtain live video data and optimized description information;
optimizing the live video data according to the optimization description information;
and playing the optimized live video data.
17. A data processing method is applied to a server and comprises the following steps:
acquiring live video data, analyzing the live video data, and determining optimized description information;
and packaging the optimization description information and the live video data to obtain live video transmission information, and transmitting the live video transmission information to a live video playing terminal, wherein the live video playing terminal is used for optimizing the live video data according to the optimization description information and playing and outputting the optimized live video data.
18. A data processing method is applied to a monitoring video playing terminal and comprises the following steps:
receiving monitoring video transmission information;
analyzing the monitoring video transmission information to obtain monitoring video data and optimization description information;
optimizing the monitoring video data according to the optimization description information;
and playing the optimized monitoring video data.
19. A data processing method is applied to a server and comprises the following steps:
acquiring monitoring video data, analyzing the monitoring video data and determining optimization description information;
and packaging the optimization description information and the monitoring video data to obtain monitoring video transmission information, and transmitting the monitoring video transmission information to a monitoring video playing terminal, wherein the monitoring video playing terminal is used for optimizing the monitoring video data according to the optimization description information and playing and outputting the optimized monitoring video data.
20. A data processing method is applied to a video playing terminal and comprises the following steps:
receiving video transmission information;
analyzing the video transmission information to obtain video data and corresponding optimization description information, wherein the video data comprises multiple paths of sub-video data;
optimizing target sub-video data in the video data according to the optimization description information;
and playing the optimized video data.
21. The method of claim 20, wherein the optimizing the target sub-video data in the video data according to the optimization description information comprises:
acquiring virtual structure data according to the optimization description information;
and adding the virtual structure data into the target sub-video data to obtain the optimized target sub-video data.
22. A data processing method is applied to a server and comprises the following steps:
acquiring multiple paths of sub-video data to determine video data;
analyzing the target sub-video data and determining corresponding optimization description information;
and packaging the optimization description information and the video data to obtain video transmission information, and transmitting the video transmission information to a video playing terminal, wherein the video playing terminal is used for optimizing target sub-video data in the video data according to the optimization description information and playing and outputting the optimized live video data.
23. A data processing method is applied to a video playing terminal and comprises the following steps:
providing an optimization mode selection page, wherein the optimization mode selection page comprises a plurality of optimization selection controls corresponding to different optimization modes;
and determining selection information according to the operation instruction of the optimization selection control, and transmitting the selection information to the server so that the server can process the video data in a corresponding optimization mode according to the selection information.
24. The method of claim 23, wherein determining selection information according to the operation instruction on the optimization selection control comprises:
receiving a key instruction sent by a key of the target remote control equipment;
and determining an operation instruction for the optimized selection control according to the key instruction so as to determine corresponding selection information.
25. The method of claim 23, further comprising:
providing an image quality comparison page;
acquiring optimized first video data and second video data before optimization;
and playing the first video data and the second video data on an image quality comparison page for comparison.
26. A data processing method is applied to a server and comprises the following steps:
receiving selection information, wherein the selection information is determined according to an optimization mode selection page;
determining a target optimization mode aiming at the video data according to the selection information;
and correspondingly processing the acquired video data according to the target optimization mode.
27. The method of claim 26, wherein the corresponding processing of the acquired video data according to the target optimization manner comprises at least one of the following steps:
when the target optimization mode is the first mode, the server analyzes the acquired video data to determine optimization description information, packages the optimization description information and the video data into video transmission information, and sends the video transmission information to a video playing terminal, wherein the video playing terminal is used for optimizing the video data according to the optimization description information and playing the optimized video data;
when the target optimization mode is a second mode, the server splits the acquired video data, determines first split video data and second split video data, optimizes the first split video data, analyzes the second split video data, determines optimized description information of the second split video data, packages the optimized description information, the second split video data and the optimized first split video data into video transmission information, and sends the video transmission information to a video playing terminal, wherein the video playing terminal is used for optimizing the second split video data according to the optimized description information and playing the optimized first split video data and the optimized second split video data;
when the target optimization mode is the third mode, the server side packages the acquired video data into video transmission information and sends the video transmission information to the video playing terminal, and the video playing terminal is used for analyzing the video data, determining optimization description information, and optimizing the video data according to the optimization description information to output the optimized video data;
and when the target optimization mode is the fourth mode, the server encapsulates the acquired video data into video transmission information and sends the video transmission information to the video playing terminal, and the video playing terminal is used for analyzing the video transmission information, determining the video data and outputting the video data.
28. A data processing apparatus comprising:
the transmission information receiving module is used for receiving video transmission information;
the transmission information analysis module is used for analyzing the video transmission information to obtain video data and optimized description information;
the video data optimization module is used for optimizing the video data according to the optimization description information;
and the video data playing module is used for playing the optimized video data.
29. A data processing apparatus comprising:
the video data analysis module is used for analyzing the video data and determining optimization description information;
and the transmission information packaging module is used for packaging the optimization description information and the video data to obtain video transmission information and transmitting the video transmission information to the terminal, and the terminal is used for optimizing the video data according to the optimization description information and playing and outputting the optimized video data.
30. A data processing apparatus comprising:
the live broadcast transmission information receiving module is used for receiving live broadcast video transmission information;
the live broadcast transmission information analysis module is used for analyzing the live broadcast video transmission information to obtain live broadcast video data and optimized description information;
the live video data optimization module is used for optimizing the live video data according to the optimization description information;
and the live video data playing module is used for playing the optimized live video data.
31. A data processing apparatus comprising:
the live video data analysis module is used for acquiring live video data, analyzing the live video data and determining optimized description information;
and the live broadcast transmission information packaging module is used for packaging the optimization description information and the live broadcast video data to obtain live broadcast video transmission information and transmitting the live broadcast video transmission information to the live broadcast video playing terminal, and the live broadcast video playing terminal is used for optimizing the live broadcast video data according to the optimization description information and playing and outputting the optimized live broadcast video data.
32. A data processing apparatus comprising:
the monitoring transmission information receiving module is used for receiving monitoring video transmission information;
the monitoring transmission information analysis module is used for analyzing the monitoring video transmission information to obtain monitoring video data and optimization description information;
the monitoring video data optimization module is used for optimizing the monitoring video data according to the optimization description information;
and the monitoring video data playing module is used for playing the optimized monitoring video data.
33. A data processing apparatus comprising:
the monitoring video data analysis module is used for acquiring monitoring video data, analyzing the monitoring video data and determining optimization description information;
and the monitoring transmission information packaging module is used for packaging the optimization description information and the monitoring video data to obtain monitoring video transmission information and transmitting the monitoring video transmission information to the monitoring video playing terminal, and the monitoring video playing terminal is used for optimizing the monitoring video data according to the optimization description information and playing and outputting the optimized monitoring video data.
34. A data processing apparatus comprising:
the transmission information acquisition module is used for receiving video transmission information;
the sub-video data acquisition module is used for analyzing the video transmission information to acquire video data and corresponding optimization description information, wherein the video data comprises a plurality of paths of sub-video data;
the sub-video data optimization module is used for optimizing target sub-video data in the video data according to the optimization description information;
and the multi-channel video playing module is used for playing the optimized video data.
35. A data processing apparatus comprising:
the sub-video data acquisition module is used for acquiring multiple paths of sub-video data to determine video data;
the sub-video data analysis module is used for analyzing the target sub-video data and determining corresponding optimization description information;
and the transmission information acquisition module is used for packaging the optimization description information and the video data to obtain video transmission information and transmitting the video transmission information to the video playing terminal, and the video playing terminal is used for optimizing target sub-video data in the video data according to the optimization description information and playing and outputting the optimized live video data.
36. A data processing apparatus comprising:
the selection page providing module is used for providing an optimization mode selection page, and the optimization mode selection page comprises a plurality of optimization selection controls corresponding to different optimization modes;
and the selection information acquisition module is used for determining selection information according to the operation instruction of the optimization selection control and transmitting the selection information to the server so that the server can process the video data in a corresponding optimization mode according to the selection information.
37. A data processing apparatus comprising:
the selection information receiving module is used for receiving selection information, and the selection information is determined according to an optimization mode selection page;
the optimization mode acquisition module is used for determining a target optimization mode aiming at the video data according to the selection information;
and the optimized data acquisition module is used for carrying out corresponding processing on the acquired video data according to the target optimization mode.
38. An electronic device, comprising: a processor; and
memory having stored thereon executable code which, when executed, causes the processor to perform the method of one or more of claims 1-27.
39. One or more machine-readable media having executable code stored thereon that, when executed, causes a processor to perform the method of one or more of claims 1-27.
CN202011376609.0A 2020-11-30 2020-11-30 Data processing method and device, electronic equipment and storage medium Pending CN113301427A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011376609.0A CN113301427A (en) 2020-11-30 2020-11-30 Data processing method and device, electronic equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011376609.0A CN113301427A (en) 2020-11-30 2020-11-30 Data processing method and device, electronic equipment and storage medium

Publications (1)

Publication Number Publication Date
CN113301427A true CN113301427A (en) 2021-08-24

Family

ID=77318399

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011376609.0A Pending CN113301427A (en) 2020-11-30 2020-11-30 Data processing method and device, electronic equipment and storage medium

Country Status (1)

Country Link
CN (1) CN113301427A (en)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103702180A (en) * 2014-01-14 2014-04-02 北京奇艺世纪科技有限公司 Media file playing method and device
CN103702182A (en) * 2014-01-14 2014-04-02 北京奇艺世纪科技有限公司 Video playing method and device
US9407944B1 (en) * 2015-05-08 2016-08-02 Istreamplanet Co. Resource allocation optimization for cloud-based video processing
CN110753243A (en) * 2019-11-05 2020-02-04 深圳市巨潮科技股份有限公司 Image processing method, image processing server and image processing system

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103702180A (en) * 2014-01-14 2014-04-02 北京奇艺世纪科技有限公司 Media file playing method and device
CN103702182A (en) * 2014-01-14 2014-04-02 北京奇艺世纪科技有限公司 Video playing method and device
US9407944B1 (en) * 2015-05-08 2016-08-02 Istreamplanet Co. Resource allocation optimization for cloud-based video processing
CN110753243A (en) * 2019-11-05 2020-02-04 深圳市巨潮科技股份有限公司 Image processing method, image processing server and image processing system

Similar Documents

Publication Publication Date Title
KR101342781B1 (en) Transforming video data in accordance with three dimensional input formats
US10956766B2 (en) Bit depth remapping based on viewing parameters
KR101311622B1 (en) Encapsulating three-dimensional video data in accordance with transport protocols
JP5567132B2 (en) Transform video data according to human visual feedback metrics
Moorthy et al. Visual quality assessment algorithms: what does the future hold?
US10720091B2 (en) Content mastering with an energy-preserving bloom operator during playback of high dynamic range video
US10484727B2 (en) Methods and systems for using 2D captured imagery of a scene to provide virtual reality content
WO2018009828A1 (en) Systems and methods for region-of-interest tone remapping
US10783670B2 (en) Method for compression of 360 degree content and electronic device thereof
CN116980604A (en) Video encoding method, video decoding method and related equipment
CN113301427A (en) Data processing method and device, electronic equipment and storage medium
Lee et al. Real‐time multi‐GPU‐based 8KVR stitching and streaming on 5G MEC/Cloud environments
KR101724309B1 (en) Guided 3d display adaptation
CN114207606A (en) Source classification using HDMI audio metadata
CN102630025B (en) A kind of method and system of processing signals
US11908340B2 (en) Magnification enhancement of video for visually impaired viewers
US20220398216A1 (en) Appliances and methods to provide robust computational services in addition to a/v encoding, for example at edge of mesh networks
US20230088144A1 (en) Data processing method and apparatus for immersive media, related device, and storage medium
WO2016111888A1 (en) Method and apparatus for provision of enhanced multimedia content
KR20220029715A (en) Volumetric data processing
CN111601157A (en) Audio output method and display device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination