CN117979090A

CN117979090A - Media data processing method and device, electronic equipment and storage medium

Info

Publication number: CN117979090A
Application number: CN202410057629.3A
Authority: CN
Inventors: 成艺丰
Original assignee: Shuhang Technology Beijing Co ltd
Current assignee: Shuhang Technology Beijing Co ltd
Priority date: 2024-01-15
Filing date: 2024-01-15
Publication date: 2024-05-03

Abstract

The application discloses a media data processing method, a device, electronic equipment and a storage medium. The method comprises the following steps: acquiring a media playing file corresponding to media to be played, wherein the media playing file comprises parameter information of a plurality of media fragments corresponding to the media to be played and m interrupt labels, and each interrupt label is used for separating the parameter information corresponding to two adjacent media fragments with discontinuous playing time in the media playing file; correcting the initial playing time corresponding to the media frames in each media slice based on the m interrupt labels and the parameter information of each media slice to obtain the target playing time corresponding to the media frames in each media slice.

Description

Media data processing method and device, electronic equipment and storage medium

Technical Field

The present application relates to the field of data processing technologies, and in particular, to a media data processing method, a device, an electronic device, and a storage medium.

Background

With the development of the internet, streaming media services have become an integral part of people's daily life. The streaming media protocol (HTTP LIVE STREAMING, HLS) is a protocol widely used for streaming media playback, and can carry out fragmented transmission on media. However, when the play time (Presentation TIME STAMP, PTS) corresponding to two consecutive media frames jumps or is discontinuous, the media play progress is discontinuous, and the effect of media play is reduced.

Disclosure of Invention

The application provides a media data processing method, a device, electronic equipment and a storage medium, which are used for correcting the playing time of media frames in media fragments, so that the playing time of the media frames in a plurality of media fragments is continuous, the continuity of the media playing progress is ensured, and the media playing effect is improved.

In a first aspect, the present application provides a media data processing method, where the method is applied to a playing end, and the method includes:

Acquiring a media playing file corresponding to media to be played, wherein the media playing file comprises parameter information of a plurality of media fragments corresponding to the media to be played and m interrupt labels, and each interrupt label is used for separating the parameter information corresponding to two adjacent media fragments with discontinuous playing time in the media playing file;

Correcting the initial playing time corresponding to the media frames in each media slice based on the m interrupt labels and the parameter information of each media slice to obtain the target playing time corresponding to the media frames in each media slice.

In a second aspect, the present application provides a media data processing method, the method being applied to a server, the method comprising:

Acquiring media to be played;

Transcoding the media to be played to obtain a media stream;

slicing the media stream based on preset slicing time length and playing time corresponding to media frames in the media stream to obtain a plurality of media slices, wherein the playing time of the media frames in each media slice is continuous;

Generating a media playing file based on a plurality of media fragments, wherein the media playing file comprises parameter information of the plurality of media fragments corresponding to media to be played and m interrupt labels, and each interrupt label is used for separating the parameter information corresponding to two adjacent media fragments with discontinuous playing time in the media playing file;

And sending the media playing file to the playing end, so that the playing end corrects the initial playing time corresponding to the media frames in each media slice based on the m interrupt labels and the parameter information of each media slice, and obtains the target playing time corresponding to the media frames in each media slice.

In a third aspect, the present application provides a media data processing device, the device comprising: an acquisition unit and a processing unit;

The device comprises an acquisition unit, a display unit and a display unit, wherein the acquisition unit is used for acquiring a media playing file corresponding to media to be played, the media playing file comprises parameter information of a plurality of media fragments corresponding to the media to be played and m interrupt labels, and each interrupt label is used for separating the parameter information corresponding to two adjacent media fragments with discontinuous playing time in the media playing file;

The processing unit is used for correcting the initial playing time corresponding to the media frames in each media slice based on the m interrupt labels and the parameter information of each media slice, and obtaining the target playing time corresponding to the media frames in each media slice.

In a fourth aspect, the present application provides a server comprising: the device comprises a receiving and transmitting unit and a first processing unit;

The receiving and transmitting unit is used for acquiring media to be played;

The processing unit is used for transcoding the media to be played to obtain a media stream;

The processing unit is further used for slicing the media stream based on preset slicing time length and playing time corresponding to the media frames in the media stream to obtain a plurality of media slices, wherein the playing time of the media frames in each media slice is continuous;

The processing unit is further used for generating a media playing file based on the plurality of media fragments, wherein the media playing file comprises parameter information of the plurality of media fragments corresponding to media to be played and m interrupt labels, and each interrupt label is used for separating the parameter information corresponding to two adjacent media fragments with discontinuous playing time in the media playing file;

The receiving and transmitting unit is further configured to send a media playing file to the playing end, so that the playing end corrects an initial playing time corresponding to a media frame in each media slice based on m interrupt tags and parameter information of each media slice, and obtains a target playing time corresponding to the media frame in each media slice.

In a fifth aspect, the present application provides an electronic device, comprising: a processor and a memory, the processor being connected to the memory, the memory being for storing a computer program, the processor being for executing the computer program stored in the memory to cause an electronic device to perform a method as in the first or second aspect.

In a sixth aspect, the present application provides a computer-readable storage medium storing a computer program that causes a computer to perform the method as in the first or second aspect.

In a seventh aspect, the present application provides a computer program product comprising a non-transitory computer readable storage medium storing a computer program, the computer being operable to cause a computer to perform a method as in the first or second aspect.

The implementation of the application has the following beneficial effects:

Firstly, acquiring a media playing file corresponding to media to be played, wherein the media playing file comprises parameter information of a plurality of media fragments corresponding to the media to be played and m interrupt labels, each interrupt label is used for separating the parameter information corresponding to two adjacent media fragments with discontinuous playing time in the media playing file, that is, the playing time of a media frame in each media fragment is continuous, and the parameter information of two adjacent video fragments with discontinuous playing time (for example, the playing time of the former media fragment in the two adjacent media fragments is 5 s-10 s, the playing time of the latter media fragment is 15 s-20 s, 10s and 15s are discontinuous) is separated by the interrupt label; and correcting the initial playing time corresponding to the media frames in each media slice based on the m interrupt labels and the parameter information of each media slice to obtain the target playing time corresponding to the media frames in each media slice, so that the playing time of the media frames in the plurality of media slices is continuous, further playing is performed based on the target playing time corresponding to the media frames, the continuity of the playing progress is ensured, and the media playing effect is improved.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for the description of the embodiments will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present application, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

FIG. 1 is a schematic diagram of a media playing file according to an embodiment of the present application;

fig. 2 is a schematic diagram of a system architecture based on HLS according to an embodiment of the present application;

FIG. 3 is a schematic diagram of a media data processing system according to an embodiment of the present application;

FIG. 4 is a schematic diagram illustrating separation of interrupt labels in a media playback file according to an embodiment of the present application;

fig. 5 is a flowchart of a media data processing method according to an embodiment of the present application;

Fig. 6 is a schematic diagram of a media data processing method according to an embodiment of the present application;

FIG. 7 is a block diagram illustrating functional units of a media data processing device according to an embodiment of the present application;

FIG. 8 is a block diagram illustrating functional units of a server according to an embodiment of the present application;

fig. 9 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

Detailed Description

The following description of the embodiments of the present application will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are some, but not all embodiments of the application. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.

The terms "first," "second," "third," and "fourth" and the like in the description and in the claims and drawings are used for distinguishing between different objects and not necessarily for describing a particular sequential or chronological order. Furthermore, the terms "comprise" and "have," as well as any variations thereof, are intended to cover a non-exclusive inclusion. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those listed steps or elements but may include other steps or elements not listed or inherent to such process, method, article, or apparatus.

Reference herein to "an embodiment" means that a particular feature, result, or characteristic described in connection with the embodiment may be included in at least one embodiment of the application. The appearances of such phrases in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. Those of skill in the art will explicitly and implicitly appreciate that the embodiments described herein may be combined with other embodiments.

First, the related art and related terms related to the present application will be explained:

PTS: the PTS of a media frame indicates the display time or play time (i.e., the time at which play is started) of the media frame, indicating the point in time at which the media frame should be displayed when played, for example, when a player plays a media file, it determines when to display the media frame based on the PTS value of each media frame of the media file.

Media play file (MEDIA PLAYLIST): in HLS, the format of the media playlist and the secondary index file may be M3U8 format; the media play file includes parameter information (such as a download address of each media slice, a duration of each media slice, etc.) of a plurality of media slices corresponding to media (such as audio, video, etc.) to be played.

For easy understanding, referring to fig. 1, fig. 1 is a schematic diagram of a media playing file according to an embodiment of the present application, as shown in fig. 1, "# EXTM U" indicates that the media playing file is an M3U8 file, "# EXT-X-target data: 6 "indicates that the maximum duration of the MEDIA slice is 6 seconds," #EXT-X-MEDIA-SEQUENCE:1 "indicates that the serial number or sequence number of the first media slice in the media play file is 1," #EXT-X-PLAYLIST-TYPE: VOD "indicates that the use mode of the media play file is on demand (VOD)," #EXTINF:6.0 "indicates that the duration of the media slice is 6 seconds," media_ fi leSequence (1, 2,3, 4, 5, 6, 7, 8, 9, 10), "#EXT-X-ENDLIST" indicates that the download address or uniform resource locator (Uniform Resource Locator, URL) corresponding to the media slice exists when the use mode of the media play file is on demand, and indicates that the end is reached.

Referring to fig. 2, fig. 2 is a schematic diagram of a system architecture based on HLS according to an embodiment of the present application. The system architecture shown in fig. 2 includes a server, a distributing end and a playing end, wherein the server includes a multimedia encoder (Media Encoder) and a slicing tool (SEGMENTER); the multimedia encoder mainly encodes (or transcodes) and encapsulates (for example, encapsulates into MPEG-TS format) the collected media data (for example, audio, video, etc.), and then the multimedia encoder sends TS (Transport Stream) stream files obtained by encapsulation to the slicing tool; then the slicing tool slices the TS stream file according to the preset slicing time length to obtain a plurality of TS slices, and generates an index file (only one index file is needed to be generated when the TS stream file is in a video-on-demand scene, and real-time live media streams are needed to be sliced and the index file is updated when the TS stream file is in a live broadcast scene), wherein the index file comprises a main index file (MASTER PLAYLIST) and a secondary index file, the main index file generally comprises downloading addresses of the plurality of secondary index files, the resolutions corresponding to the plurality of secondary index files are different, and each secondary index file comprises downloading addresses of the plurality of TS slices under the resolution corresponding to each secondary index file; then the server sends the obtained TS fragments and index files to a distributing end; when a playing end prepares to play media, a downloading request aiming at a main index file is sent to a distributing end, then the playing end receives and analyzes the main index file from the distributing end, then the playing end sends a downloading request aiming at a target secondary index file to the distributing end, then the playing end analyzes the target secondary index file to obtain downloading addresses of a plurality of TS fragments, then the 1 st TS fragment is downloaded and decoded and played based on the downloading addresses of the 1 st TS fragment, the 2 nd TS fragment is downloaded while the 1 st TS fragment is played, the 2 nd TS fragment is played after the 1 st TS fragment is completely played, and all subsequent TS fragments are continuously downloaded, decoded and played in the same way in sequence.

As can be seen from the above background art, the PTS of two consecutive media frames corresponding to a media file are easy to jump, for example, the PTS of a media frame changes due to a problem in the transcoding process of a multimedia encoder, which affects the playing effect; for example, in a live broadcast scene, each video stream has a unique identifier for distinguishing different streams, if a stream is switched or a stream is re-pushed in the live broadcast process, a change of a push parameter, such as resolution, frame rate, coding mode, etc., may be caused, so that the PTS of the video stream after switching and the PTS of the video stream before switching are discontinuous, and the live broadcast effect is affected.

Therefore, aiming at the problem that the playing time of the media frame is jumped, the acquired media playing file comprises parameter information of a plurality of media fragments corresponding to the media to be played and m interrupt labels, the playing time of the media frame in each media fragment is continuous, and the parameter information of two adjacent video fragments with discontinuous playing time (for example, the playing time of the former media fragment in the two adjacent media fragments is 5 s-10 s, the playing time of the latter media fragment is 15 s-20 s, and 10s and 15s are discontinuous) is separated based on the interrupt labels; and correcting the initial playing time corresponding to the media frames in each media slice based on the m interrupt labels and the parameter information of each media slice to obtain the target playing time corresponding to the media frames in each media slice, so that the playing time of the media frames in the plurality of media slices is continuous, further playing is performed based on the target playing time corresponding to the media frames, the continuity of the playing progress is ensured, and the media playing effect is improved.

Referring to fig. 3, fig. 3 is a schematic diagram of a media data processing system according to an embodiment of the application.

The system shown in fig. 3 includes a server and a media data processing device; the server may be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, or a cloud server for providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, basic cloud computing services such as big data and an artificial intelligent platform, and the application is not particularly limited; the media data processing device may be a playing end, and the playing end may be an intelligent terminal such as a smart phone, a tablet computer, a notebook computer, a desktop computer, an intelligent television, a desktop computer, an intelligent watch, an intelligent vehicle, etc., but is not limited thereto.

Firstly, a server acquires media to be played, and then the server transcodes and encapsulates the media to be played to obtain a media stream, for example, transcodes the media to be played into a coding format which is adaptive to an HLS protocol and encapsulates the media to be played in a TS format; then the server slices the media stream according to the preset slice duration and the playing time PTS corresponding to each media frame in the media stream to obtain a plurality of media slices, and generates an index file (comprising a main index file and a plurality of secondary index files), wherein when slicing, the server can detect whether the PTS of the continuous media frames in the media stream is jumped, for example, if the slice duration is assumed to be 5 seconds, if the PTS corresponding to all media frames in the 1 st to 5 th seconds in the media stream is continuous, the 0 th to 5 th seconds are determined to be a first media slice, then if the PTS corresponding to all media frames in the 6 th to 10 th seconds is continuous, the 6 th to 10 th seconds are determined to be a second media slice, in principle, the third media slice is 11 th to 15 th seconds, if the server detects that the PTS corresponding to the media frames from 13 th to 13 th seconds is discontinuous, the PTS corresponding to the media frames in the 12 th second is illustrated, if the PTS corresponding to the 13 th seconds is discontinuous, the 0 th to 5 th seconds are determined to be a first media slice, then the 6 th to 10 th second are determined to be a second media slice, and in principle, and when the PTS corresponding to the 3 th to 13 th second is discontinuous, the third media slice is determined to be a second media slice, and the fourth media slice is determined to be a second media slice, and the 3 to be the continuous media slice is the second, and the 3 second to be the 5 th media slice is determined to be the continuous media frame; accordingly, when the secondary index file is generated, as long as the PTS jump of the continuous media frame occurs, for example, the PTS of the media frame corresponding to the 12 th second and the 13 th second in the above example is discontinuous, since the secondary index file (i.e., the media play file) includes the parameter information (such as the duration, the download address, etc.) of the plurality of media slices, the server may separate the parameter information of the third media slice and the parameter information of the fourth media slice by interrupting the tag, for example, inserting the tag between the parameter information of the third media slice and the parameter information of the fourth media slice to indicate that the third media slice and the fourth media slice are discontinuous.

For easy understanding, referring to fig. 4 based on the media play file shown in fig. 1, fig. 4 is a schematic diagram illustrating separation of interrupt labels in the media play file according to an embodiment of the present application, and it should be noted that the meaning of each parameter in the media play file shown in fig. 4 may refer to the explanation of fig. 1 and will not be repeated here; as shown in fig. 4, from top to bottom, the parameter information corresponding to the first media slice includes "#extnf: 6.0, media_filesequence1.Ts", the parameter information corresponding to the second media slice includes "#extinf:6.0, the parameter information corresponding to the media_filesequence2.Ts", the third media slice includes "#extinf: the parameter information corresponding to the fourth media slice comprises" # EXTINF:6.0, the parameter information corresponding to the fourth media slice comprises "# EXTINF:6.0,medi a_fileSequence5.ts", the parameter information corresponding to the sixth media slice comprises "# EXTINF:6.0,media_fileSequ ence6.ts", the parameter information corresponding to the seventh media slice comprises "# EXTINF:6.0, the parameter information corresponding to the fourth media slice comprises" # EXTINF:6.0, the parameter information corresponding to the eighth media slice comprises "# EXTINF:6.0, the parameter information corresponding to the ninth media slice comprises" # EXTINF:6.0, the parameter information corresponding to the tenth media slice comprises "# EXTINF:6.0, the parameter information corresponding to the seventh media slice comprises" # EXTINF: 10.0, the parameter information corresponding to the fifth media slice is inserted in the corresponding to the third media slice (i.e., the parameter information corresponding to the fifth media slice is not shown in the PTS) and the corresponding to the fifth media slice is a specific interrupt label (i.e., the parameter information corresponding to the first media slice is not shown in the PTS) and the fifth media slice is not shown in the audio interrupt mode) is inserted between the corresponding to the fifth media slice, the PTS of the media frame in the fifth media slice (namely, the PTS of the tail frame) is discontinuous with the PTS of the media frame in the sixth media slice (namely, the PTS of the head frame), and similarly, an interrupt label is inserted between the corresponding parameter information of the eighth media slice and the corresponding parameter information of the ninth media slice, so that the PTS of the media frame in the eighth media slice (namely, the PTS of the tail frame) is discontinuous with the PTS of the media frame in the ninth media slice (namely, the PTS of the head frame); or the figure is understood as the right side of fig. 4, assuming that 1.ts to 10.ts correspond to the first media slice to the tenth media slice described above, respectively, 2.ts and 3.ts interrupts, 5.ts and 6.ts interrupts, 8.ts and 9.ts interrupts are indicated.

Then, when the media data processing device wants to play the media to be played, the media data processing device sends a first request to the server, wherein the first request is used for requesting to acquire a main index file corresponding to the media to be played; then, the server responds to the first request and sends a main index file of the media to be played to the media data processing device, for example, the transmission protocol is HLS protocol; then the media data processing device analyzes the main index file to obtain the download addresses of a plurality of secondary index files corresponding to the media to be played, wherein the resolution or code rate of the plurality of secondary index files is different; then the media data processing device determines a target secondary index file from the plurality of secondary index files, for example, based on the current bandwidth, determines a target secondary index file corresponding to the resolution matching the current bandwidth; then the media data processing device downloads the target secondary index file based on the download address of the target secondary index file, namely the media data processing device acquires a media playing file corresponding to the media to be played, wherein the media playing file comprises parameter information of a plurality of media fragments corresponding to the media to be played and m interrupt labels, and each interrupt label is used for separating the parameter information corresponding to two adjacent media fragments with discontinuous playing time in the media playing file; finally, the media data processing device corrects the initial playing time corresponding to the media frames in each media slice based on the m interrupt labels and the parameter information of each media slice to obtain the target playing time corresponding to the media frames in each media slice, so that the playing time of the media frames in the plurality of media slices is continuous, further, the playing is performed based on the target playing time corresponding to the media frames, the continuity of the playing progress is ensured, and the media playing effect is improved.

Referring to fig. 5, fig. 5 is a flowchart of a media data processing method according to an embodiment of the present application, where the method is applied to the media data processing device, and the method includes, but is not limited to, steps S501-S502:

S501, acquiring a media playing file corresponding to media to be played.

In the embodiment of the present application, the media playing file obtained in step S501 may be understood as a secondary index file corresponding to the media to be played, where the media playing file includes parameter information of a plurality of media slices corresponding to the media to be played, and m interrupt tags, where each interrupt tag is used to separate parameter information corresponding to two adjacent media slices with discontinuous playing time in the media playing file, and m is an integer, based on the above embodiment, it is known that PTS of a media frame in each media slice is continuous and is not explained here; in addition, the description of the media play file, the interrupt tag, and the parameter information of the media slice may refer to the description of the above embodiments, and will not be repeated.

In an alternative embodiment, each interrupt label is inserted between the parameter information corresponding to two adjacent media fragments with discontinuous playing time in the media playing file, so as to separate the parameter information corresponding to the two adjacent media fragments; or each interruption tab is located at the head end of the position where the parameter information corresponding to the latter of the two adjacent media fragments with discontinuous playing time is located, and the specific form of the parameter information corresponding to each interruption tab for dividing the two adjacent media fragments with discontinuous playing time is not limited specifically.

S502, correcting the initial playing time corresponding to the media frames in each media slice based on m interrupt labels and parameter information of each media slice to obtain the target playing time corresponding to the media frames in each media slice.

Illustratively, step S502 may include, but is not limited to, steps S11-S13:

S11, based on m interrupt labels, grouping the plurality of media fragments to obtain m+1 media fragment groups.

By way of illustration in fig. 4, the number of interruption tabs in the media play file in fig. 4 is 3, i.e., m=3, and then a plurality of media slices (i.e., 1.ts-10.ts) are grouped to obtain 4 media slice groups, where the 1 st media slice group includes a first media slice (1.ts), a second media slice (2.ts), the 2 nd media slice group includes a third media slice (3.ts), a fourth media slice (4.ts), and a fifth media slice (5.ts), and similarly the 3 rd media slice group includes a sixth media slice (6.ts), a seventh media slice (7.ts), an eighth media slice (8.ts), and the 4 th media slice group includes a ninth media slice (9.ts), and a tenth media slice (10.ts).

S12, determining the initial playing time corresponding to each media fragment group based on the parameter information of each media fragment.

First, it is noted that each media slice group may include one or more media slices, and each media slice includes a certain number of media frames, and then the start playing time corresponding to each media slice group may be understood as the playing time of the first media frame in the first media slice in each media slice group, for example, the playing time of the first media frame in the first media slice (3. Ts) in the 2 nd media slice group. For the ith media slice group, where the ith media slice group is any one of m+1 media slice groups, i is a positive integer, when i=1, the start playing time corresponding to the ith media slice group is a first preset value, that is, when i=1, i.e., the 1 st media slice group, the playing time of the first media slice in the 1 st media slice group is the first preset value, and when the server slices the media, the first preset value is generally 0 since the server generally begins slicing from the zero time of the media; similarly, when i > 1, the start playing time corresponding to the i-1 th media slice group is determined based on the start playing time corresponding to the i-1 th media slice group and the duration of the media slices in the i-1 th media slice group, and for example, since the parameter information of each media slice includes the duration of each media slice, the start playing time corresponding to the i-1 th media slice group may be the sum of the start playing time corresponding to the i-1 th media slice group and the duration of each media slice in the i-1 th media slice group, for example, the duration of the 1 st media slice may be 6 seconds from "# extin: 6.0, media_ fileSeq ue 1.Ts" in fig. 4, and the duration of the 2 nd media slice may be 6 seconds from "# extin: 6.0, media_ fileSeque ce2.Ts", and the start playing time corresponding to the 2 nd media slice group may be (first preset value+6).

In an optional embodiment, the grouping information of each media slice and the initial playing time corresponding to each media slice group may be cached to a preset structure, where the grouping information of each media slice may include a sequence number of the media slice group where each media slice is located in m+1 media slice groups or which one of the media slice groups, so that when the playing end corrects, the playing end may directly obtain, based on each media slice, the grouping information corresponding to each media slice from the preset structure and the initial playing time corresponding to the media slice group where each media slice is located, thereby accelerating the correction efficiency.

S13, correcting the initial playing time corresponding to the media frames in each media slice based on the initial playing time corresponding to each media slice group to obtain the target playing time corresponding to the media frames in each media slice.

For example, each media slice is decoded first to obtain an initial playing time corresponding to a media frame in each media slice; then, determining a target playing time corresponding to the media frames in each media slice based on the initial playing time corresponding to the media frames in each media slice and the initial playing time corresponding to each media slice group, and specifically: based on the grouping information of each media slice, acquiring the initial playing time corresponding to the media slice group where each media slice is located from a preset structure body, for example, based on the grouping information of each media slice, determining the sequence number (namely, the arrangement sequence number) of the media slice group where each media slice is located in m+1 media slice groups, then determining the initial playing time corresponding to the media slice group where each media slice is located, for example, if the sequence number of the media slice group where the current media slice is located in m+1 media slice groups is 1, namely, the initial playing time corresponding to the 1 st media slice group is represented; then, based on the initial playing time corresponding to the media frame in each media slice and the initial playing time corresponding to the media slice group in which each media slice is located, determining the target playing time corresponding to the media frame in each media slice, specifically, may determine the target playing time corresponding to each media frame in each media slice based on the initial playing time corresponding to each media frame in each media slice, the initial playing time corresponding to the media slice group in which each media slice is located, and the initial playing time corresponding to the first media frame in each media slice, for example, the target playing time corresponding to the kth media frame in the kth media slice in the ith media slice group may be obtained by formula (1):

P_izk＝A_i+(B_izk-C_i11)(1)

Wherein, P _izk is the target playing time corresponding to the kth media frame in the z-th media slice in the ith media slice group, a _i is the starting playing time corresponding to the ith media slice group, B _izk is the initial playing time corresponding to the kth media frame in the z-th media slice in the ith media slice group, C _i11 is the initial playing time corresponding to the first media frame in the first media slice in the ith media slice group, and the ith media slice group is any one of m+1 media slice groups.

It should be noted that in the HLS protocol, the playing end plays the media slice while downloading the media slice, that is, downloads the next media slice of the current media slice when playing the current media slice, so that the correction is the same, decoding the current media slice to obtain the initial playing time corresponding to the media frame in the current media slice, then directly correcting the initial playing time corresponding to the media frame in the current media slice to obtain the target playing time (which can be understood as the correct playing time) corresponding to the media frame in the current media slice, and then playing the current media slice based on the target playing time corresponding to the media frame in the current media slice, so that the progress bar presented by playing the plurality of media slices is continuous forward flow.

Further, correcting the initial playing time corresponding to the media frames in the plurality of media fragments to obtain a target playing time corresponding to the media frames in each media fragment, wherein the playing end can obtain a progress bar corresponding to the media to be played based on the target playing time corresponding to each media fragment; then, the playing end responds to the search seek operation of the user aiming at the progress bar, and the playing time corresponding to the search operation is obtained; then, the playing end determines the target media fragments corresponding to the playing time based on the playing time and the target playing time corresponding to each media fragment; and then the playing end plays the target media fragments corresponding to the playing time, specifically, the target media frames corresponding to the playing time are found from the target media fragments, and then the target media frames and the subsequent media frames of the target media frames are played. That is, when the player wants to search for seek to play the target position, the player can find the target media slice corresponding to the target position based on the target play time corresponding to the media frame in the plurality of media slices, then the player determines the media frame corresponding to the play time corresponding to the target position from the target media slices based on the play time corresponding to the target position and starts playing, and if the PTS of the continuous media frame is hopped and not corrected, when the player wants to execute the operation of seek to the target position, the seek is made to not reach the media slice corresponding to the target position, for example, the time corresponding to the plurality of continuous media slices is assumed to be in principle: 0-5 th second, 5 th second-10 th second, 10 th second-15 th second, 15 th second-20 th second, 20 th second-25 th second, 25 th second-30 th second, … … th, 95 th second-100 th second, assuming that the hopping sequence becomes: the application corrects PTS of media frames in the media fragments, and the time corresponding to the media fragments returns to the time before the jump, namely the original jump from 20 th second to 25 th second to 05 th second, the original jump from 25 th second to 30 th second to 10 th second to 15 th second, if the jump from seek to 17 th second is needed, the time corresponding to a plurality of media fragments after the jump can be found out not to include 17 th second, so that the playing end can continuously correct the PTS of the media frames in the media fragments, and the time corresponding to the media fragments can return to the time before the jump, at this time, the jump from the 17 th second to the corresponding target media fragment from 15 th second to 20 th second, and then the playing end decodes the target media fragment, and the playing end can find out the media frame corresponding to the target media fragment, and the media frame corresponding to the 17 th second, thereby providing accurate media experience for each media frame corresponding to the media frame in the media fragment.

For easy understanding, referring to fig. 6, fig. 6 is a schematic diagram of a media data processing method according to an embodiment of the present application.

Assuming that the total duration of one media to be played is 50 seconds, the total duration of the media to be played is divided into 10 media fragments, the duration of each media fragment is 5 seconds, then fig. 6 shows parameter information (sequentially represented by 1.ts, 2.ts, … … and 10.ts) and m interrupt tags (m=3) of a plurality of media fragments (namely 10) in a media play file corresponding to the media to be played, since the parameter information of each media fragment includes the duration of each media fragment, the playing end can know the duration of each media fragment from the parameter information corresponding to each media fragment (all shown as 5 seconds in fig. 6), and can know that the 2 nd media fragment is discontinuous with the 3 rd media fragment, that is, the 5 th media fragment is discontinuous with the 6 th media fragment, that is, the 8 th media fragment is discontinuous with the 9 th media fragment, that is, from the 3 interrupt tags shown in fig. 6; then, the playing end groups 10 media fragments based on the 3 interrupt labels to obtain m+1 (4) media fragments (the serial numbers correspond to 1,2,3 and 4 in sequence), wherein the 1 st media fragment group (the serial number: 1) comprises the 1 st media fragment and the 2 nd media fragment, the 2 nd media fragment group (the serial number: 2) comprises the 3 rd media fragment, the 4 th media fragment and the 5 th media fragment, the 3 rd media fragment group (the serial number: 3) comprises the 6 th media fragment, the 7 th media fragment and the 8 th media fragment, and the 4 th media fragment group (the serial number: 4) comprises the 9 th media fragment and the 10 th media fragment; then, based on the duration of each media slice, determining the initial playing time of each media slice group, wherein the initial playing time corresponding to the 1 st media slice group is 0 (namely Base: 0), the initial playing time corresponding to the 2 nd media slice group is 10 (namely Base: 10), the initial playing time corresponding to the 3 rd media slice group is 25 (namely Base: 25), and the initial playing time corresponding to the 4 th media slice group is 40 (namely Base: 40); then, the playing end may decode each media slice to obtain an initial playing time corresponding to a media frame in each media slice, and it should be noted that the initial playing time corresponding to the media frame in each media slice is continuous, so, during correction, the target playing time corresponding to the media frame in each media slice may be determined based on the initial playing time corresponding to the media frame in each media slice and the initial playing time corresponding to each media slice group, for simple understanding, fig. 6 only shows the initial playing time of each media slice (which may be understood as the initial playing time of the first media frame in each media slice, in turn, 0 th second, 5 th second, 15 th second, 20 th second, 25 th second, 30 th second, 15 th second, 20 th second), and the initial playing time corresponding to each media frame in each media slice is not shown, because the initial playing time corresponding to each media frame in each media slice is continuous, and the initial playing time of each media frame in each media slice is known; then the initial playing time of the media frame in each media slice may be corrected based on the initial playing time of each media slice and the initial playing time corresponding to the media slice group where each media slice is located, so as to obtain the target playing time of the media frame in each media slice, fig. 6 also only shows the target playing time corresponding to each media slice (which may also be understood as the target playing time of the first media frame in each media slice, and is sequentially 0 th second, 5 th second, 10 th second, 15 th second, 20 th second, 25 th second, 30 th second, 35 th second, 40 th second, and 45 th second), so that the target playing time of the remaining media frames in each media slice may be known based on the target playing time of the first media frame in each media slice.

In addition, another media data processing method provided in the embodiment of the present application is applied to a server, and includes, but is not limited to, steps S21-S25:

S21, acquiring media to be played;

s22, transcoding the media to be played to obtain a media stream;

S23, slicing the media stream based on preset slicing time length and play time corresponding to media frames in the media stream to obtain a plurality of media slices, wherein the play time of the media frames in each media slice is continuous;

S24, generating a media playing file based on a plurality of media fragments, wherein the media playing file comprises parameter information of the plurality of media fragments corresponding to media to be played and m interrupt labels, and each interrupt label is used for separating the parameter information corresponding to two adjacent media fragments with discontinuous playing time in the media playing file;

And S25, sending a media playing file to a playing end, so that the playing end corrects the initial playing time corresponding to the media frames in each media slice based on the m interrupt labels and the parameter information of each media slice, and obtains the target playing time corresponding to the media frames in each media slice.

It should be noted that, the specific principles of steps S21 to S25 may be described with reference to the above embodiments, and the server may also execute the steps executed by the server in the above embodiments, so that the same technical effects may be achieved, which is not described herein.

It can be seen that, in the embodiment of the present application, by obtaining a media play file corresponding to a to-be-played medium, where the media play file includes parameter information of a plurality of media slices corresponding to the to-be-played medium, and m interrupt tags, each interrupt tag is used to separate parameter information corresponding to two adjacent media slices with discontinuous play times in the media play file, that is, the play times of media frames in each media slice are continuous, and parameter information of two adjacent video slices with discontinuous play times (for example, the play time of the former media slice in two adjacent media slices is 5 s-10 s, and the play time of the latter media slice is 15 s-20 s, 10s and 15s are discontinuous) is separated by the interrupt tag; and correcting the initial playing time corresponding to the media frames in each media slice based on m interrupt labels and the parameter information of each media slice to obtain the target playing time corresponding to the media frames in each media slice, so that the playing time of the media frames in the plurality of media slices is continuous, further playing is performed based on the target playing time corresponding to the media frames, the continuity of the playing progress is ensured, and if a playing end wants to execute the seek operation aiming at the target position of the progress bar, the seek can be performed to the media slice corresponding to the target position, and then playing is performed, thereby improving the playing effect of the media.

Referring to fig. 7, fig. 7 is a block diagram illustrating functional units of a media data processing device according to an embodiment of the present application. The media data processing device 700 comprises: an acquisition unit 701 and a processing unit 702;

An obtaining unit 701, configured to obtain a media playing file corresponding to a media to be played, where the media playing file includes parameter information of a plurality of media slices corresponding to the media to be played, and m interrupt tags, where each interrupt tag is configured to separate parameter information corresponding to two adjacent media slices with discontinuous playing time in the media playing file;

The processing unit 702 is configured to correct an initial playing time corresponding to a media frame in each media slice based on the m interrupt tags and parameter information of each media slice, so as to obtain a target playing time corresponding to the media frame in each media slice.

In one embodiment of the present application, in correcting the initial playing time corresponding to the media frame in each media slice based on the m interrupt tags and the parameter information of each media slice, the processing unit 702 is specifically configured to:

Based on m interrupt labels, grouping a plurality of media fragments to obtain m+1 media fragment groups;

Determining the initial playing time corresponding to each media fragment group based on the parameter information of each media fragment;

And correcting the initial playing time corresponding to the media frames in each media slice based on the initial playing time corresponding to each media slice group to obtain the target playing time corresponding to the media frames in each media slice.

In one embodiment of the present application, the parameter information of each media slice includes a duration of each media slice, and for the ith media slice group, in determining a start playing time corresponding to each media slice group based on the parameter information of each media slice, the processing unit 702 is specifically configured to:

when i=1, the initial playing time corresponding to the ith media slice group is a first preset value;

When i is more than 1, determining the initial playing time corresponding to the ith media slice group based on the initial playing time corresponding to the ith-1 media slice group and the duration of the media slices in the ith-1 media slice group, wherein the ith media slice group is any one of m+1 media slice groups, and i is a positive integer.

In one embodiment of the present application, in correcting the initial playing time corresponding to the media frame in each media slice based on the initial playing time corresponding to each media slice group, the processing unit 702 is specifically configured to:

Decoding each media slice to obtain the initial playing time corresponding to the media frame in each media slice;

and determining the target playing time corresponding to the media frames in each media slice based on the initial playing time corresponding to the media frames in each media slice and the initial playing time corresponding to each media slice group.

In one embodiment of the present application, the processing unit 702 is specifically configured to, in determining, based on the initial playing time corresponding to the media frame in each media slice and the initial playing time corresponding to each media slice group, a target playing time corresponding to the media frame in each media slice before:

buffering grouping information of each media slice and initial playing time corresponding to each media slice group to a preset structural body;

In determining a target playing time aspect corresponding to the media frame in each media slice based on the initial playing time corresponding to the media frame in each media slice and the initial playing time corresponding to each media slice group, the processing unit 702 is specifically configured to:

based on grouping information of each media slice, acquiring initial playing time corresponding to a media slice group where each media slice is located from a preset structure body;

And determining the target playing time corresponding to the media frames in each media slice based on the initial playing time corresponding to the media frames in each media slice and the initial playing time corresponding to the media slice group in which each media slice is positioned.

In one embodiment of the present application, the processing unit 702 is specifically configured to determine, based on an initial playing time corresponding to a media frame in each media slice and an initial playing time corresponding to a media slice group in which each media slice is located, a target playing time corresponding to a media frame in each media slice:

and determining the target playing time corresponding to each media frame in each media slice based on the initial playing time corresponding to each media frame in each media slice, the initial playing time corresponding to the media slice group in which each media slice is positioned and the initial playing time corresponding to the first media frame in each media slice.

In one embodiment of the present application, the target playing time corresponding to the kth media frame in the z-th media slice in the ith media slice group is obtained by formula (2):

P_izk＝A_i+(B_izk-C_i11)(2)

In one embodiment of the present application, each interrupt label is inserted between parameter information corresponding to two adjacent media clips having discontinuous playing time in the media playing file, so as to separate the parameter information corresponding to the two adjacent media clips.

In a specific implementation, the acquiring unit 701 and the processing unit 702 described in the embodiments of the present invention may execute other implementations described in the embodiments of the media data processing method provided in the embodiments of the present invention, which are not described herein again.

Referring to fig. 8, fig. 8 is a block diagram illustrating functional units of a server according to an embodiment of the present application. The server 800 includes: a transceiver unit 801 and a first processing unit 802;

a transceiver unit 801, configured to obtain a media to be played;

A first processing unit 802, configured to transcode media to be played to obtain a media stream;

The first processing unit 802 is further configured to slice the media stream based on a preset slicing duration and a play time corresponding to a media frame in the media stream, so as to obtain a plurality of media slices, where the play time of the media frame in each media slice is continuous;

The first processing unit 802 is further configured to generate a media play file based on the plurality of media fragments, where the media play file includes parameter information of the plurality of media fragments corresponding to media to be played and m interrupt tags, and each interrupt tag is configured to separate parameter information corresponding to two adjacent media fragments with discontinuous play time in the media play file;

The transceiver 801 is further configured to send a media play file to the play end, so that the play end corrects an initial play time corresponding to a media frame in each media slice based on the m interrupt tags and parameter information of each media slice, to obtain a target play time corresponding to the media frame in each media slice.

In a specific implementation, the transceiver unit 801 and the first processing unit 802 described in the embodiments of the present invention may execute other implementations described in the embodiments of the media data processing method provided in the embodiments of the present invention, which are not described herein again.

Referring to fig. 9, fig. 9 is a schematic structural diagram of an electronic device according to an embodiment of the present application. As shown in fig. 9, the electronic device 900 includes a transceiver 901, a processor 902, and a memory 903. Which are connected by a bus 904. The memory 903 is used to store computer programs and data, and the data stored in the memory 903 may be transferred to the processor 902.

The electronic device 900 may be the above media data processing device and a server;

When the electronic device 900 is a media data processing apparatus, the processor 902 is configured to read a computer program in the memory 903 to perform the following operations:

The method comprises the steps that a control transceiver 901 obtains a media playing file corresponding to media to be played, wherein the media playing file comprises parameter information of a plurality of media fragments corresponding to the media to be played and m interrupt labels, and each interrupt label is used for separating the parameter information corresponding to two adjacent media fragments with discontinuous playing time in the media playing file;

In one embodiment of the present application, the processor 902 is specifically configured to perform the following steps in correcting an initial playing time corresponding to a media frame in each media slice based on m interrupt tags and parameter information of each media slice to obtain a target playing time corresponding to the media frame in each media slice:

In one embodiment of the present application, the parameter information of each media slice includes a duration of each media slice, and for the ith media slice group, the processor 902 is specifically configured to perform the following steps in determining a start playing time corresponding to each media slice group based on the parameter information of each media slice:

In one embodiment of the present application, the processor 902 is specifically configured to perform the following steps in correcting the initial playing time corresponding to the media frame in each media slice based on the initial playing time corresponding to each media slice group to obtain the target playing time corresponding to the media frame in each media slice:

In one embodiment of the present application, the processor 902 is specifically configured to perform the following steps in determining, based on the initial playing time corresponding to the media frame in each media slice and the initial playing time corresponding to each media slice group, before the target playing time corresponding to the media frame in each media slice:

in determining a target play time aspect corresponding to a media frame in each media slice based on an initial play time corresponding to the media frame in each media slice and a start play time corresponding to each media slice group, the processor 902 is specifically configured to perform the following steps:

In one embodiment of the present application, the processor 902 is specifically configured to determine a target playing time corresponding to a media frame in each media slice based on an initial playing time corresponding to a media frame in each media slice and a starting playing time corresponding to a media slice group in which each media slice is located, where the steps are specifically executed:

P_izk＝A_i+(B_izk-C_i11)(3)

Specifically, the transceiver 901 may be the acquisition unit 701 of the media data processing device 700 of the embodiment of fig. 7, and the processor 902 may be the processing unit 702 of the media data processing device 700 of the embodiment of fig. 7.

In a specific implementation, the transceiver 901 and the processor 902 described in the embodiments of the present invention may also execute other implementations described in the embodiments of the media data processing method provided in the embodiments of the present invention, which are not described herein again.

When the electronic device is a server, the processor 902 is configured to read the computer program in the memory 903 to perform the following operations:

The method comprises the steps that a transceiver 901 is controlled to acquire media to be played;

Transcoding the media to be played to obtain a media stream;

The transceiver 901 is controlled to send a media playing file to a playing end, so that the playing end corrects the initial playing time corresponding to the media frame in each media slice based on m interrupt labels and parameter information of each media slice, and obtains the target playing time corresponding to the media frame in each media slice.

Specifically, the transceiver 901 may be the transceiver unit 801 of the server 800 in the embodiment of fig. 8, and the processor 902 may be the first processing unit 802 of the server 800 in the embodiment of fig. 8.

It should be understood that when the electronic device is a media data processing device, the electronic device in the present application may include a smart Phone (such as an Android Phone, an iOS Phone, a Windows Phone, etc.), a tablet computer, a palm computer, a notebook computer, a Mobile internet device MID (Mobile INTERNET DEVICES, abbreviated as MID), a wearable device, or the like. The above-described electronic devices are merely examples and are not intended to be exhaustive and include, but are not limited to, the above-described electronic devices. In practical applications, the electronic device may further include: intelligent vehicle terminals, computer devices, etc.

When the electronic device is a server, the electronic device of the present application may include a cloud computing server, a Content Delivery Network (CDN) server, a Network time protocol (Network Time Protocol, NTP), a Domain name resolution system (Domain NAME SYSTEM, DNS) server, and other various types of servers. The servers described above are merely examples, and are not exhaustive, including but not limited to the servers described above.

It should be appreciated that embodiments of the present application also provide a computer readable storage medium storing a computer program for execution by a processor to perform part or all of the steps of any one of the media data processing methods as described in the method embodiments above.

Embodiments of the present application also provide a computer program product comprising a non-transitory computer readable storage medium storing a computer program operable to cause a computer to perform part or all of the steps of any one of the media data processing methods described in the method embodiments above.

It should be noted that, for simplicity of description, the foregoing method embodiments are all described as a series of acts, but it should be understood by those skilled in the art that the present application is not limited by the order of acts described, as some steps may be performed in other orders or concurrently in accordance with the present application. Further, those skilled in the art will also appreciate that the embodiments described in the specification are alternative embodiments, and that the acts and modules referred to are not necessarily required for the present application.

In the foregoing embodiments, the descriptions of the embodiments are emphasized, and for parts of one embodiment that are not described in detail, reference may be made to related descriptions of other embodiments.

In the several embodiments provided by the present application, it should be understood that the disclosed apparatus may be implemented in other manners. For example, the apparatus embodiments described above are merely illustrative, such as a division of units, merely a division of logic functions, and there may be additional divisions in actual implementation, such as multiple units or components may be combined or integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, or may be in electrical or other forms.

The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed over a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in the embodiments of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units described above may be implemented either in hardware or in software program modules.

The integrated units, if implemented in the form of software program modules and sold or used as a stand-alone product, may be stored in a computer readable memory. Based on this understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the prior art or in whole or in part in the form of a software product stored in a memory, comprising several instructions for causing a computer device (which may be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method of the various embodiments of the present application. And the aforementioned memory includes: a usb disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a removable hard disk, a magnetic disk, or an optical disk, or other various media capable of storing program codes.

Those of ordinary skill in the art will appreciate that all or a portion of the steps in the various methods of the above embodiments may be implemented by a program that instructs associated hardware, and the program may be stored in a computer readable memory, which may include: flash disk, read-Only Memory (ROM), random access Memory (Ra ndom Access Memory, RAM), magnetic disk or optical disk.

The foregoing has outlined rather broadly the more detailed description of embodiments of the application, wherein the principles and embodiments of the application are explained in detail using specific examples, the above examples being provided solely to facilitate the understanding of the method and core concepts of the application; meanwhile, as those skilled in the art will vary in the specific embodiments and application scope according to the idea of the present application, the present disclosure should not be construed as limiting the present application in summary.

Claims

1. A method of media data processing, the method comprising:

And correcting the initial playing time corresponding to the media frames in each media slice based on the m interrupt labels and the parameter information of each media slice to obtain the target playing time corresponding to the media frames in each media slice.

2. The method of claim 1, wherein correcting the initial playing time corresponding to the media frame in each media slice based on the m interrupt tags and the parameter information of each media slice to obtain the target playing time corresponding to the media frame in each media slice includes:

grouping the plurality of media fragments based on the m interrupt tags to obtain m+1 media fragment groups;

3. The method of claim 2, wherein the parameter information of each media slice includes a duration of each media slice, and wherein for an ith media slice group, the determining a start playing time corresponding to each media slice group based on the parameter information of each media slice includes:

When i is more than 1, determining the initial playing time corresponding to the ith media slice group based on the initial playing time corresponding to the ith-1 media slice group and the duration of the media slices in the ith-1 media slice group, wherein the ith media slice group is any one of the m+1 media slice groups, and i is a positive integer.

4. A method according to claim 2 or 3, wherein correcting the initial playing time corresponding to the media frame in each media slice based on the initial playing time corresponding to each media slice group to obtain the target playing time corresponding to the media frame in each media slice comprises:

5. The method of claim 4, wherein prior to the determining the target play time for the media frame in each media slice based on the initial play time for the media frame in each media slice and the start play time for each media slice group, the method further comprises:

The determining the target playing time corresponding to the media frame in each media slice based on the initial playing time corresponding to the media frame in each media slice and the initial playing time corresponding to each media slice group comprises the following steps:

based on grouping information of each media slice, acquiring initial playing time corresponding to a media slice group where each media slice is located from the preset structural body;

6. The method of claim 5, wherein determining the target play time corresponding to the media frame in each media slice based on the initial play time corresponding to the media frame in each media slice and the initial play time corresponding to the media slice group in which each media slice is located comprises:

7. The method of any of claims 1-6, wherein each interrupt label is inserted between parameter information corresponding to two adjacent media slices in the media play file that have discontinuous play times to separate the parameter information corresponding to the two adjacent media slices.

8. A media data processing device, the device comprising: an acquisition unit and a processing unit;

The obtaining unit is configured to obtain a media playing file corresponding to a media to be played, where the media playing file includes parameter information of a plurality of media slices corresponding to the media to be played, and m interrupt tags, where each interrupt tag is configured to separate parameter information corresponding to two adjacent media slices with discontinuous playing time in the media playing file;

The processing unit is configured to correct an initial playing time corresponding to a media frame in each media slice based on the m interrupt tags and parameter information of each media slice, so as to obtain a target playing time corresponding to the media frame in each media slice.

9. An electronic device, comprising: a processor and a memory, the processor being connected to the memory, the memory being for storing a computer program, the processor being for executing the computer program stored in the memory to cause the electronic device to perform the method of any one of claims 1-7.

10. A computer readable storage medium, characterized in that the computer readable storage medium stores a computer program, which is executed by a processor to implement the method of any of claims 1-7.