CN118158484A - Multimedia data processing method and device - Google Patents
Multimedia data processing method and device Download PDFInfo
- Publication number
- CN118158484A CN118158484A CN202211557922.3A CN202211557922A CN118158484A CN 118158484 A CN118158484 A CN 118158484A CN 202211557922 A CN202211557922 A CN 202211557922A CN 118158484 A CN118158484 A CN 118158484A
- Authority
- CN
- China
- Prior art keywords
- playing
- multimedia data
- current
- data
- time
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 20
- 238000000034 method Methods 0.000 claims abstract description 40
- 238000004590 computer program Methods 0.000 claims description 15
- 230000008569 process Effects 0.000 abstract description 16
- 230000000694 effects Effects 0.000 abstract description 9
- 238000010586 diagram Methods 0.000 description 13
- 230000006870 function Effects 0.000 description 6
- 230000002452 interceptive effect Effects 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 2
- 230000003139 buffering effect Effects 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000005055 memory storage Effects 0.000 description 1
- 230000002085 persistent effect Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 230000011664 signaling Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
- 230000001052 transient effect Effects 0.000 description 1
Landscapes
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
The embodiment of the application discloses a multimedia data processing method and device. The method comprises the following steps: receiving an audio track switching instruction aiming at multimedia data, wherein the audio track switching instruction comprises identification information of a target audio track to be switched; determining the current playing time corresponding to the current playing frame of the multimedia data; sending an acquisition request for target data to a network side; the target data are data of the multimedia data, wherein the playing time of the data is located after the current playing time, and the obtaining request comprises the current playing time; receiving the target data sent by the network side; and playing the target data by utilizing the target audio track according to the identification information of the target audio track. According to the technical scheme, the multimedia data can be ensured to be smoothly played when the audio tracks are switched in the playing process, and the real-time smooth switching effect among the audio tracks is realized.
Description
Technical Field
The present disclosure relates to the field of data processing technologies, and in particular, to a method and an apparatus for processing multimedia data.
Background
The digital television is beneficial to thousands of households, and the set top box is taken as terminal equipment of the digital television and is an indispensable household equipment for the households. RTSP (REAL TIME STREAMING Protocol, real-time streaming Protocol) is widely applied to interactive network television services of digital televisions, and can provide various viewing services such as live broadcast (time shift) and on-demand for users, and is widely and well-established at present to play services at terminals such as set top boxes.
In the related art, for multimedia data playing, especially time-shift playing and on-demand playing, the streaming rate of the server is generally higher than the encoding rate of the source, and a part of data is buffered during playing. It is assumed that the track switching is performed after a period of playing, because the common player only retains the data of the currently played track and discards the non-selected track data, and it is assumed that the track 1 is selected when playing, and the track 2 needs to be switched after a period of playing, because the player discards the non-current track data, only the data of the track 1 is in the cache data in the player when playing the time t, and the data of the track 2 is not available, and because the user has already selected to switch to the track 2 at this time, the player cannot select the valid track data, only the playing of the video data can be performed, and the track switching will be effective after a period of delay, thereby causing the phenomenon that no sound and no audio are asynchronous in a period of time. Since most player video plays are synchronized based on audio, it may result in video fast-forward audio tracking over a period of time.
Disclosure of Invention
An object of the embodiments of the present application is to provide a method and an apparatus for processing multimedia data, which are used for solving the problem that audio is easy to be out of sync when switching between multiple audio tracks.
In order to solve the technical problems, the embodiment of the application is realized as follows:
in one aspect, an embodiment of the present application provides a multimedia data processing method, including:
receiving an audio track switching instruction aiming at multimedia data, wherein the audio track switching instruction comprises identification information of a target audio track to be switched;
Determining the current playing time corresponding to the current playing frame of the multimedia data;
Sending an acquisition request for target data to a network side; the target data are data of the multimedia data, wherein the playing time of the data is located after the current playing time, and the obtaining request comprises the current playing time;
Receiving the target data sent by the network side;
and playing the target data by utilizing the target audio track according to the identification information of the target audio track.
In another aspect, an embodiment of the present application provides a multimedia data processing apparatus, including:
A first receiving module, configured to receive an audio track switching instruction for multimedia data, where the audio track switching instruction includes identification information of a target audio track to be switched;
The first determining module is used for determining the current playing time corresponding to the current playing frame of the multimedia data;
The sending module is used for sending an acquisition request for target data to the network side; the target data are data of the multimedia data, wherein the playing time of the data is located after the current playing time, and the obtaining request comprises the current playing time;
the second receiving module is used for receiving the target data sent by the network side;
and the playing module is used for playing the target data by utilizing the target audio track according to the identification information of the target audio track.
In still another aspect, an embodiment of the present application provides a multimedia data processing apparatus, including a processor and a memory electrically connected to the processor, where the memory stores a computer program, and the processor is configured to call and execute the computer program from the memory to implement the foregoing multimedia data processing method.
In yet another aspect, an embodiment of the present application provides a storage medium storing a computer program executable by a processor to implement the above-described multimedia data processing method.
After receiving the audio track switching instruction for the multimedia data, the technical scheme of the embodiment of the application sends the acquisition request for the target data to the network side by determining the current playing time corresponding to the current playing frame of the multimedia data, wherein the target data is the data of the multimedia data, the playing time of which is positioned after the current playing time, and receives the target data sent by the network side, and further plays the target data by utilizing the target audio track indicated by the audio track switching instruction. When the audio track is switched, the playing device acquires the data after the current playing time from the network side again, and plays based on the acquired data, and the playing device does not depend on the buffer data before the audio track is switched, so that the phenomenon of asynchronous audio can be avoided, the multimedia data can still be smoothly played when the audio track is switched in the playing process, and the real-time smooth switching effect among multiple audio tracks is realized.
Drawings
In order to more clearly illustrate one or more embodiments of the present specification or the prior art, the drawings that are required for the description of the embodiments or the prior art will be briefly described, and it is apparent that the drawings in the following description are only some embodiments described in one or more embodiments of the present specification, and other drawings may be obtained according to these drawings without inventive effort for a person of ordinary skill in the art.
Fig. 1 is a schematic flow chart of a multimedia data processing method according to an embodiment of the present specification;
Fig. 2 is a schematic flowchart of a multimedia data processing method according to another embodiment of the present specification;
Fig. 3 is a schematic diagram of a multimedia data processing method according to an embodiment of the present specification;
Fig. 4 is a schematic diagram of a multimedia data processing method according to another embodiment of the present specification;
fig. 5 is a schematic block diagram of a multimedia data processing apparatus according to an embodiment of the present specification;
Fig. 6 is a schematic block diagram of a multimedia data processing apparatus according to an embodiment of the present specification.
Detailed Description
The embodiment of the application provides a multimedia data processing method and device, which are used for solving the problem that audio frequency is easy to be asynchronous when a plurality of audio tracks are switched.
In order to make the technical solution of the present application better understood by those skilled in the art, the technical solution of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application, and it is apparent that the described embodiments are only some embodiments of the present application, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, shall fall within the scope of the application.
Fig. 1 is a schematic flow chart of a multimedia data processing method according to an embodiment of the present application, as shown in fig. 1, the method includes the following steps S102-S110:
s102, receiving an audio track switching instruction aiming at multimedia data, wherein the audio track switching instruction comprises identification information of a target audio track to be switched.
The audio track switching instruction can be initiated by a user through a front-end interactive interface of the playing device, and can also be initiated automatically by the playing device according to the current playing state. For example, the playing device detects the current playing state of the multimedia data, and if the current playing state is poor, such as a clamping phenomenon and a buffering phenomenon occur, the audio track switching instruction is automatically initiated.
S104, determining the current playing time corresponding to the current playing frame of the multimedia data.
S106, sending an acquisition request for target data to a network side; the target data is data, wherein the playing time of the data is located after the current playing time corresponding to the current playing frame, in the multimedia data, and the obtaining request comprises the current playing time corresponding to the current playing frame.
Alternatively, the acquisition request may be sent to the network side based on RTSP, and the acquisition request may be in the form of rtsp_play signaling.
S108, receiving target data sent by the network side.
S110, playing the target data by using the target track according to the identification information of the target track.
In this embodiment, the multimedia data may include RTSP-based live video and/or on-demand video.
After receiving the audio track switching instruction for the multimedia data, the technical scheme of the embodiment of the application sends the acquisition request for the target data to the network side by determining the current playing time corresponding to the current playing frame of the multimedia data, wherein the target data is the data of the multimedia data, the playing time of which is positioned after the current playing time, and receives the target data sent by the network side, and further plays the target data by utilizing the target audio track indicated by the audio track switching instruction. When the audio track is switched, the playing device acquires the data after the current playing time from the network side again, and plays based on the acquired data, and the playing device does not depend on the buffer data before the audio track is switched, so that the phenomenon of asynchronous audio can be avoided, the multimedia data can still be smoothly played when the audio track is switched in the playing process, and the real-time smooth switching effect among multiple audio tracks is realized.
In one embodiment, before performing step S102, a play instruction for the multimedia data is first acquired, where the play instruction includes identification information of an initial audio track, and the initial audio track is different from a target audio track. Next, based on the play instruction, the multimedia data is played by using the initial audio track, and a start play time and a start time stamp of the multimedia data are determined.
The starting playing time of the multimedia data is the playing time corresponding to the first frame of the multimedia data, and the starting timestamp is the timestamp corresponding to the first frame of the multimedia data. Taking multimedia data as an example of video-on-demand, a user sends a playing instruction to play from the 10 th minute of the video-on-demand, then the 10 th minute of the video-on-demand is the starting playing time of the video-on-demand, and the timestamp corresponding to the video frame corresponding to the 10 th minute of the video-on-demand is the starting timestamp of the video-on-demand.
In one embodiment, when determining the current playing time corresponding to the current playing frame of the multimedia data, the following steps may be specifically performed: firstly, a current time stamp corresponding to a current playing frame is obtained, and secondly, the current playing time corresponding to the multimedia data is determined according to the current time stamp corresponding to the current playing frame, the starting playing time and the starting time stamp of the multimedia data.
The network side stores timestamp information corresponding to each frame of the multimedia data in advance for each multimedia data, and based on the timestamp information, the current timestamp corresponding to the current playing frame can be obtained no matter which frame of the multimedia data is played when an audio track switching instruction is received.
Considering that the playing mode of the multimedia data includes time-shift playing and on-demand playing, the following situations may occur: when receiving the track switching instruction, the current time stamp corresponding to the current playing frame is earlier than the starting time stamp of the multimedia data, or the current time stamp corresponding to the current playing frame is later than the starting time stamp of the multimedia data. Therefore, when determining the current playing time corresponding to the multimedia data, the current time stamp corresponding to the multimedia data and the starting time stamp can be compared first to determine whether the current time stamp is earlier than the starting time stamp; and then determining the current playing time corresponding to the multimedia data according to the comparison result.
Optionally, comparing the current timestamp corresponding to the multimedia data with the starting timestamp, where the comparison result includes: the current timestamp is earlier than the start timestamp, or the current timestamp is later than the start timestamp. A timestamp offset value is calculated, the timestamp offset value being the absolute value of the difference between the current timestamp and the start timestamp. If the current time stamp is earlier than the start time stamp, determining the difference between the start playing time and the time stamp deviation value as the current playing time of the multimedia data. If the current time stamp is later than the starting time stamp, determining the sum of the starting playing time and the time stamp deviation value as the current playing time of the multimedia data.
In this embodiment, it can be considered that the playing method of the multimedia data includes time-shift playing and on-demand playing, and the current playing time of the multimedia data is determined in a corresponding manner according to two different situations, so that in the process of playing the multimedia data, no matter when the audio track is switched, the current timestamp corresponding to the current video frame is earlier or later than the starting timestamp of the multimedia data, the current playing time of the multimedia data can be accurately determined, and therefore, an accurate multimedia data code stream, that is, the target data, can be obtained from the network side.
Fig. 2 is a schematic flow chart of a multimedia data processing method according to another embodiment of the present application, as shown in fig. 2, the method includes the following steps S201 to S210:
s201, a playing instruction of the multimedia data is obtained, wherein the playing instruction comprises identification information of an initial audio track.
The user can initiate a playing instruction through a front-end interactive interface of the playing device. Taking live video as an example, when a user clicks a play button of the live video (e.g., clicks a first drawing of the live video), it can be determined that the user has sent a play instruction for the live video. Taking video-on-demand as an example, when a user clicks a play button of the video-on-demand (e.g., clicks a first drawing of the video-on-demand), it can be determined that the user has sent a play instruction of the video-on-demand.
S202, playing the multimedia data by using the initial audio track based on the playing instruction, and determining the initial playing time and the initial time stamp of the multimedia data.
When the user sends a playing instruction to the multimedia data, a starting playing position of the multimedia data can be selected, and the starting playing position is used for determining a starting playing time and a starting time stamp of the multimedia data.
For example, when the user sends a play command to start playing from the 10 th minute of the video on demand, the 10 th minute of the video on demand is the start playing time of the video on demand, and the timestamp corresponding to the video frame corresponding to the 10 th minute of the video on demand is the start timestamp of the video on demand. For another example, the user sends a play instruction to the live video, the start play time of the live video is the play time corresponding to the video frame currently being live, and the start time stamp of the live video is the time stamp corresponding to the video frame currently being live.
S203, receiving an audio track switching instruction aiming at the multimedia data, wherein the audio track switching instruction comprises identification information of a target audio track to be switched.
Wherein the target track and the initial track are different. The track switching instruction can be initiated by a user through a front-end interactive interface of the playing device, and can also be initiated automatically by the playing device according to the current playing state. For example, the playing device detects the current playing state of the multimedia data, and if the current playing state is poor, such as a clamping phenomenon and a buffering phenomenon occur, the audio track switching instruction is automatically initiated.
S204, based on the track switching instruction, a current time stamp corresponding to the current playing frame of the multimedia data is obtained.
The network side stores timestamp information corresponding to each frame of the multimedia data in advance for each multimedia data, and based on the timestamp information, the current timestamp corresponding to the current playing frame can be obtained no matter which frame of the multimedia data is played when an audio track switching instruction is received.
In this embodiment, FIRSTPLAYTIME represents a start playing time of the multimedia data, firstPts represents a start time stamp of the multimedia data, and currentPts represents a current time stamp corresponding to a current playing frame of the multimedia data.
S205, calculating an absolute value of the difference between the current time stamp and the start time stamp, and taking the absolute value as a time stamp offset value between the current time stamp and the start time stamp.
The absolute value of the difference between the current timestamp and the start timestamp may be expressed as | currentPts-firstPts |, i.e., the timestamp offset value between the current timestamp and the start timestamp.
S206, judging whether the current time stamp corresponding to the current playing frame is earlier than the starting time stamp of the multimedia data; if yes, then execute S207; if not, S208 is performed.
S207, determining the difference between the initial playing time of the multimedia data and the time stamp deviation value as the current playing time of the multimedia data.
The current playing time of the multimedia data is denoted by currentPlayTime, and the step S207 can be expressed as the following formula:
currentPlayTime＝firstPlayTime-|currentPts-firstPts|。
s208, determining the sum of the initial playing time of the multimedia data and the timestamp deviation value as the current playing time of the multimedia data.
Step S208 may be expressed as the following formula:
currentPlayTime＝firstPlayTime+|currentPts-firstPts|。
S209, sending an acquisition request for target data to the network side, wherein the target data is data of which the playing time is positioned behind the current playing time in the multimedia data, and the acquisition request comprises the current playing time of the multimedia data.
S210, receiving target data sent by a network side, and playing the target data by utilizing the target track according to the identification information of the target track.
It can be seen that, by adopting the technical solution of this embodiment, after receiving the track switching instruction for the multimedia data, the obtaining request for the target data is sent to the network side by determining the current playing time corresponding to the current playing frame of the multimedia data, where the target data is the data in the multimedia data whose playing time is located after the current playing time, and the target data sent by the network side is received, and then the target data is played by using the target track indicated by the track switching instruction. When the audio track is switched, the playing device acquires the data after the current playing time from the network side again, and plays based on the acquired data, and the playing device does not depend on the buffer data before the audio track is switched, so that the multimedia data can be ensured to be played smoothly when the audio track is switched in the playing process, and the real-time smooth switching effect among the audio tracks is realized. In addition, when determining the current playing time corresponding to the current playing frame of the multimedia data, the current playing time of the multimedia data can be determined by taking into account that the playing mode of the multimedia data comprises time shifting playing and on-demand playing, and the corresponding mode is adopted for two different situations in a targeted manner, so that in the process of playing the multimedia data, no matter when the audio track is switched, the current time stamp corresponding to the current video frame is earlier or later than the starting time stamp of the multimedia data, the current playing time of the multimedia data can be accurately determined, thereby acquiring accurate target data from a network side, and ensuring that smooth playing of the multimedia data cannot be influenced when the audio tracks are switched.
The data processing method provided by the present application is described in several specific embodiments below.
In one embodiment, the multimedia data is video-on-demand. The user selects the video-on-demand and starts playing from the beginning, at which point the playback device determines the start playback time and start timestamp of the video-on-demand. Assuming that after playing for 5 minutes, a user sends an audio track switching instruction, and when receiving the audio track switching instruction sent by the user, the playing device obtains a current time stamp corresponding to a current playing frame of the video on demand, and determines the current playing time corresponding to the video on demand based on the current time stamp corresponding to the current playing frame, the starting playing time and the starting time stamp of the video on demand. Because the video-on-demand is played from the head, the current time stamp corresponding to the current playing frame is necessarily later than the starting time stamp, and at the moment, the sum of the starting playing time of the multimedia data and the time stamp deviation value can be determined as the current playing time of the multimedia data, wherein the time stamp deviation value is the absolute value of the difference value between the current time stamp and the starting time stamp. And then, acquiring target data corresponding to the video-on-demand from a network side, wherein the target data is the data of which the playing time is positioned after the current playing time in the video-on-demand. And further, playing the acquired target data by utilizing the target audio track to be switched.
In one embodiment, the multimedia data is live video. Live video can enter time-shift playing through pausing or dragging a progress bar leftwards, and the live video entering time-shift playing is hereinafter called time-shift video. When the live video enters time-shift playing, the playing device determines the starting playing time and the starting time stamp of the time-shift video. Assuming that after playing for 5 minutes, a user sends an audio track switching instruction, and when receiving the audio track switching instruction sent by the user, the playing device obtains a current time stamp corresponding to a current playing frame of the time-shifting video, and determines the current playing time corresponding to the time-shifting video based on the current time stamp corresponding to the current playing frame, the starting playing time and the starting time stamp of the time-shifting video. Because the current time stamp corresponding to the current playing frame of the time-shifting video is later than the starting time stamp, the sum of the starting playing time of the time-shifting video and the time stamp deviation value can be determined as the current playing time of the time-shifting video, and the time stamp deviation value is the absolute value of the difference value between the current time stamp and the starting time stamp. And then, acquiring target data corresponding to the time-shifting video from a network side, wherein the target data is the data of which the playing time is positioned behind the current playing time in the time-shifting video. And further, playing the acquired target data by utilizing the target audio track to be switched.
In one embodiment, the multimedia data is video-on-demand. The user selects the video on demand and plays the video on demand at a breakpoint, that is, the video on demand starts playing from the middle position of the video on demand, and at this time, the playing device determines the start playing time and the start time stamp of the video on demand. It is assumed that after 5 minutes of playing, the user drags the progress bar of the video on demand to a position before the initial playing position for playing, where the initial playing position is a video position corresponding to the initial playing time. After playing for 2 minutes, the user sends out an audio track switching instruction, and when receiving the audio track switching instruction sent by the user, the playing device acquires a current time stamp corresponding to a current playing frame of the video on demand, and determines the current playing time corresponding to the video on demand based on the current time stamp corresponding to the current playing frame, the starting playing time and the starting time stamp of the video on demand. The difference between the starting playing time of the multimedia data and the timestamp deviation value can be determined as the current playing time of the multimedia data, and the timestamp deviation value is the absolute value of the difference between the current timestamp and the starting timestamp. And then, acquiring target data corresponding to the video-on-demand from a network side, wherein the target data is the data of which the playing time is positioned after the current playing time in the video-on-demand. And further, playing the acquired target data by utilizing the target audio track to be switched.
Fig. 3 is a schematic diagram of a multimedia data processing method according to an embodiment of the present application. As can be seen from fig. 3, the playback device starts playing the video-on-demand from the beginning using the audio track 1, i.e. the starting playback position is the first frame of the video-on-demand. Since the network side streaming rate is generally higher than the source code rate, a part of data is buffered during the playing process. As shown in fig. 3, when playing to time t, the playing device actually receives more video data than the currently played data, that is, there is a portion of buffered data, and the playing time Δt is required for the portion of buffered data to complete playing. Assuming that an audio track switching instruction is received at time t, the current audio track 1 is indicated to switch to the audio track 2, and because the current timestamp corresponding to the current video frame of the video-on-demand (i.e. the video frame corresponding to the instant t) is later than the start timestamp at this time, the sum of the start playing time of the video-on-demand and the timestamp offset value, which is the absolute value of the difference between the current timestamp and the start timestamp, can be determined as the current playing time of the video-on-demand. Then, the playing device obtains target data with playing time after the current playing time in the video-on-demand from the network side, and plays the target data by utilizing the audio track 2. As can be seen from fig. 3, the playback device starts to receive new data (i.e. target data) from time t, and starts to play back new data using the audio track 2 from time t, where the new data can cover the buffered data, so as to avoid the problem of data delay after the audio track is switched, and thus, achieve a real-time smooth switching effect between the audio track 1 and the audio track 2.
Fig. 4 is a schematic diagram of a multimedia data processing method according to another embodiment of the present application. As can be seen from fig. 4, the playback device starts playing back from the middle of the video-on-demand, i.e. the break-point play-up, using the audio track 1. After playing for a period of time, the user drags the progress bar of the video on demand to the position before the initial playing position for playing, namely the position where the instant t-n is located. The playing device starts playing the video-on-demand from the time t-n, and when the video-on-demand is played to the time t, the video data actually received by the playing device is more than the currently played data, and part of the cache data exists in the playing device, and the part of cache data can be played only after the playing time delta t is needed. Assuming that an audio track switching instruction is received at time t, the current audio track 1 is indicated to switch to the audio track 2, and because the current timestamp corresponding to the current video frame of the video-on-demand (i.e., the video frame corresponding to the time t) is earlier than the start timestamp at this time, the difference between the start playing time of the video-on-demand and the timestamp offset value, which is the absolute value of the difference between the current timestamp and the start timestamp, can be determined as the current playing time of the video-on-demand. Then, the playing device obtains target data with playing time after the current playing time in the video-on-demand from the network side, and plays the target data by utilizing the audio track 2. As can be seen from fig. 4, the playback device starts to receive new data (i.e. target data) from time t, and starts to play back new data using the audio track 2 from time t, where the new data can cover the buffered data, so as to avoid the problem of data delay after the audio track is switched, and thus, achieve a real-time smooth switching effect between the audio track 1 and the audio track 2.
In summary, particular embodiments of the present subject matter have been described. Other embodiments are within the scope of the following claims. In some cases, the actions recited in the claims can be performed in a different order and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing may be advantageous.
The above method for processing multimedia data according to the embodiment of the present application is based on the same concept, and the embodiment of the present application further provides a multimedia data processing device.
Fig. 5 is a schematic block diagram of a multimedia data processing apparatus according to an embodiment of the present application, as shown in fig. 5, the apparatus including:
A first receiving module 51, configured to receive an audio track switching instruction for multimedia data, where the audio track switching instruction includes identification information of a target audio track to be switched;
A first determining module 52, configured to determine a current playing time corresponding to a current playing frame of the multimedia data;
A sending module 53, configured to send an acquisition request for target data to a network side; the target data are data of the multimedia data, wherein the playing time of the data is located after the current playing time, and the obtaining request comprises the current playing time;
A second receiving module 54, configured to receive the target data sent by the network side;
and the playing module 55 is used for playing the target data by utilizing the target audio track according to the identification information of the target audio track.
In one embodiment, the first determination module 52 includes:
the acquisition unit is used for acquiring a current time stamp corresponding to the current playing frame;
And the determining unit is used for determining the current playing time corresponding to the multimedia data according to the current time stamp, the starting playing time of the multimedia data and the starting time stamp.
In an embodiment, the determining unit is further configured to:
Comparing the current timestamp with the starting timestamp to determine whether the current timestamp is earlier than the starting timestamp;
And determining the current playing time corresponding to the multimedia data according to the comparison result.
In an embodiment, the determining unit is further configured to:
calculating a timestamp offset value, the timestamp offset value being an absolute value of a difference between the current timestamp and the start timestamp;
if the current time stamp is earlier than the starting time stamp, determining the difference between the starting playing time and the time stamp deviation value as the current playing time;
And if the current time stamp is later than the starting time stamp, determining the sum of the starting playing time and the time stamp deviation value as the current playing time.
In one embodiment, the apparatus further comprises:
The acquisition module is used for acquiring a playing instruction of the multimedia data before receiving the audio track switching instruction aiming at the multimedia data; the playing instruction comprises identification information of an initial audio track; the initial audio track and the target audio track are different;
And the second determining module is used for playing the multimedia data by utilizing the initial audio track based on the playing instruction and determining the initial playing time and the initial time stamp.
In one embodiment, the transmitting module 53 includes:
And the sending unit is used for sending the acquisition request to the network side based on a real-time streaming protocol RTSP.
In one embodiment, the multimedia data comprises RTSP-based live video and/or on-demand video.
After receiving the audio track switching instruction for the multimedia data, the device of the embodiment of the application sends the acquisition request for the target data to the network side by determining the current playing time corresponding to the current playing frame of the multimedia data, wherein the target data is the data of the multimedia data, the playing time of which is positioned after the current playing time, and receives the target data sent by the network side, and further plays the target data by utilizing the target audio track indicated by the audio track switching instruction. When the audio track is switched, the playing device acquires the data after the current playing time from the network side again, and plays based on the acquired data, and the playing device does not depend on the buffer data before the audio track is switched, so that the phenomenon of asynchronous audio can be avoided, the multimedia data can still be smoothly played when the audio track is switched in the playing process, and the real-time smooth switching effect among multiple audio tracks is realized.
It should be understood by those skilled in the art that the multimedia data processing apparatus of fig. 5 can be used to implement the foregoing multimedia data processing method, and the detailed description thereof should be similar to that of the foregoing method section, so as to avoid complexity and avoid redundancy.
Based on the same thought, the embodiment of the application also provides a multimedia data processing device, as shown in fig. 6. The multimedia data processing apparatus may be of a relatively large variety depending on configuration or performance, may include one or more processors 601 and a memory 602, and may store one or more storage applications or data in the memory 602. Wherein the memory 602 may be transient storage or persistent storage. The application programs stored in the memory 602 may include one or more modules (not shown) each of which may include a series of computer executable instructions for use in a multimedia data processing apparatus. Still further, the processor 601 may be arranged to communicate with the memory 602 and execute a series of computer executable instructions in the memory 602 on a multimedia data processing apparatus. The multimedia data processing apparatus may also include one or more power supplies 603, one or more wired or wireless network interfaces 604, one or more input/output interfaces 605, and one or more keyboards 606.
In particular, in this embodiment, the multimedia data processing apparatus includes a memory, and one or more programs, wherein the one or more programs are stored in the memory, and the one or more programs may include one or more modules, and each module may include a series of computer executable instructions for the multimedia data processing apparatus, and the execution of the one or more programs by the one or more processors comprises computer executable instructions for:
receiving an audio track switching instruction aiming at multimedia data, wherein the audio track switching instruction comprises identification information of a target audio track to be switched;
Determining the current playing time corresponding to the current playing frame of the multimedia data;
Sending an acquisition request for target data to a network side; the target data are data of the multimedia data, wherein the playing time of the data is located after the current playing time, and the obtaining request comprises the current playing time;
Receiving the target data sent by the network side;
and playing the target data by utilizing the target audio track according to the identification information of the target audio track.
After receiving the audio track switching instruction for the multimedia data, the technical scheme of the embodiment of the application sends the acquisition request for the target data to the network side by determining the current playing time corresponding to the current playing frame of the multimedia data, wherein the target data is the data of the multimedia data, the playing time of which is positioned after the current playing time, and receives the target data sent by the network side, and further plays the target data by utilizing the target audio track indicated by the audio track switching instruction. When the audio track is switched, the playing device acquires the data after the current playing time from the network side again, and plays based on the acquired data, and the playing device does not depend on the buffer data before the audio track is switched, so that the phenomenon of asynchronous audio can be avoided, the multimedia data can still be smoothly played when the audio track is switched in the playing process, and the real-time smooth switching effect among multiple audio tracks is realized.
The embodiments of the present application also provide a storage medium storing one or more computer programs, the one or more computer programs including instructions, which when executed by an electronic device including a plurality of application programs, enable the electronic device to perform the processes of the above-described embodiments of the multimedia data processing method, and specifically configured to perform:
receiving an audio track switching instruction aiming at multimedia data, wherein the audio track switching instruction comprises identification information of a target audio track to be switched;
Determining the current playing time corresponding to the current playing frame of the multimedia data;
Sending an acquisition request for target data to a network side; the target data are data of the multimedia data, wherein the playing time of the data is located after the current playing time, and the obtaining request comprises the current playing time;
Receiving the target data sent by the network side;
and playing the target data by utilizing the target audio track according to the identification information of the target audio track.
After receiving the audio track switching instruction for the multimedia data, the technical scheme of the embodiment of the application sends the acquisition request for the target data to the network side by determining the current playing time corresponding to the current playing frame of the multimedia data, wherein the target data is the data of the multimedia data, the playing time of which is positioned after the current playing time, and receives the target data sent by the network side, and further plays the target data by utilizing the target audio track indicated by the audio track switching instruction. When the audio track is switched, the playing device acquires the data after the current playing time from the network side again, and plays based on the acquired data, and the playing device does not depend on the buffer data before the audio track is switched, so that the phenomenon of asynchronous audio can be avoided, the multimedia data can still be smoothly played when the audio track is switched in the playing process, and the real-time smooth switching effect among multiple audio tracks is realized.
The system, apparatus, module or unit set forth in the above embodiments may be implemented in particular by a computer chip or entity, or by a product having a certain function. One typical implementation is a computer. In particular, the computer may be, for example, a personal computer, a laptop computer, a cellular telephone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.
For convenience of description, the above devices are described as being functionally divided into various units, respectively. Of course, the functions of each element may be implemented in the same piece or pieces of software and/or hardware when implementing the present application.
It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
In one typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include volatile memory in a computer-readable medium, random Access Memory (RAM) and/or nonvolatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of computer-readable media.
Computer readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of storage media for a computer include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium, which can be used to store information that can be accessed by a computing device. Computer-readable media, as defined herein, does not include transitory computer-readable media (transmission media), such as modulated data signals and carrier waves.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article or apparatus that comprises the element.
The application may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The application may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices.
In this specification, each embodiment is described in a progressive manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for system embodiments, since they are substantially similar to method embodiments, the description is relatively simple, as relevant to see a section of the description of method embodiments.
The foregoing is merely exemplary of the present application and is not intended to limit the present application. Various modifications and variations of the present application will be apparent to those skilled in the art. Any modification, equivalent replacement, improvement, etc. which come within the spirit and principles of the application are to be included in the scope of the claims of the present application.
Claims (10)
1. A method of multimedia data processing, comprising:
receiving an audio track switching instruction aiming at multimedia data, wherein the audio track switching instruction comprises identification information of a target audio track to be switched;
Determining the current playing time corresponding to the current playing frame of the multimedia data;
Sending an acquisition request for target data to a network side; the target data are data of the multimedia data, wherein the playing time of the data is located after the current playing time, and the obtaining request comprises the current playing time;
Receiving the target data sent by the network side;
and playing the target data by utilizing the target audio track according to the identification information of the target audio track.
2. The method of claim 1, wherein determining a current play time corresponding to a current play frame of the multimedia data comprises:
Acquiring a current time stamp corresponding to the current playing frame;
And determining the current playing time corresponding to the multimedia data according to the current time stamp, the starting playing time of the multimedia data and the starting time stamp.
3. The method according to claim 2, wherein determining the current playing time corresponding to the multimedia data according to the current time stamp and the starting playing time and the starting time stamp of the multimedia data comprises:
Comparing the current timestamp with the starting timestamp to determine whether the current timestamp is earlier than the starting timestamp;
And determining the current playing time corresponding to the multimedia data according to the comparison result.
4. The method according to claim 3, wherein determining the current playing time corresponding to the multimedia data according to the comparison result includes:
calculating a timestamp offset value, the timestamp offset value being an absolute value of a difference between the current timestamp and the start timestamp;
if the current time stamp is earlier than the starting time stamp, determining the difference between the starting playing time and the time stamp deviation value as the current playing time;
And if the current time stamp is later than the starting time stamp, determining the sum of the starting playing time and the time stamp deviation value as the current playing time.
5. The method of claim 2, wherein prior to receiving the track switch instruction for multimedia data, the method further comprises:
acquiring a playing instruction of the multimedia data; the playing instruction comprises identification information of an initial audio track; the initial audio track and the target audio track are different;
and playing the multimedia data by utilizing the initial audio track based on the playing instruction, and determining the initial playing time and the initial time stamp.
6. The method according to claim 1, wherein the sending, to the network side, the acquisition request for the target data includes:
and sending the acquisition request to the network side based on a real-time streaming protocol RTSP.
7. The method according to any of claims 1-6, wherein the multimedia data comprises RTSP-based live video and/or on-demand video.
8. A multimedia data processing apparatus, comprising:
A first receiving module, configured to receive an audio track switching instruction for multimedia data, where the audio track switching instruction includes identification information of a target audio track to be switched;
The first determining module is used for determining the current playing time corresponding to the current playing frame of the multimedia data;
The sending module is used for sending an acquisition request for target data to the network side; the target data are data of the multimedia data, wherein the playing time of the data is located after the current playing time, and the obtaining request comprises the current playing time;
the second receiving module is used for receiving the target data sent by the network side;
and the playing module is used for playing the target data by utilizing the target audio track according to the identification information of the target audio track.
9. A multimedia data processing apparatus comprising a processor and a memory electrically connected to the processor, the memory storing a computer program, the processor being adapted to invoke and execute the computer program from the memory to implement the multimedia data processing method of any of claims 1-7.
10. A storage medium storing a computer program executable by a processor to implement the multimedia data processing method of any one of claims 1-7.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211557922.3A CN118158484A (en) | 2022-12-06 | 2022-12-06 | Multimedia data processing method and device |
| PCT/CN2023/132689 WO2024120167A1 (en) | 2022-12-06 | 2023-11-20 | Multimedia data processing method and apparatus |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211557922.3A CN118158484A (en) | 2022-12-06 | 2022-12-06 | Multimedia data processing method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN118158484A true CN118158484A (en) | 2024-06-07 |
Family
ID=91297598
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211557922.3A Pending CN118158484A (en) | 2022-12-06 | 2022-12-06 | Multimedia data processing method and device |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN118158484A (en) |
| WO (1) | WO2024120167A1 (en) |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102347042B (en) * | 2010-07-28 | 2014-05-07 | Tcl集团股份有限公司 | Audio track switching method, system thereof and audio and video file playing device |
| CN104159136B (en) * | 2014-05-29 | 2016-01-13 | 腾讯科技（深圳）有限公司 | The acquisition methods of interactive information, terminal, server and system |
| CN104967901B (en) * | 2015-06-29 | 2018-05-04 | 腾讯科技（北京）有限公司 | Track switching method and apparatus for media playing in media play |
| CN112866794B (en) * | 2019-11-12 | 2022-06-07 | 腾讯科技（深圳）有限公司 | Audio track switching method and equipment |
| CN114422840A (en) * | 2022-01-04 | 2022-04-29 | 海信视像科技股份有限公司 | Audio track switching method and device |
| CN114567812A (en) * | 2022-03-02 | 2022-05-31 | 北京字节跳动网络技术有限公司 | Audio playing method, device, system, electronic equipment and storage medium |
-
2022
- 2022-12-06 CN CN202211557922.3A patent/CN118158484A/en active Pending
-
2023
- 2023-11-20 WO PCT/CN2023/132689 patent/WO2024120167A1/en unknown
Also Published As
| Publication number | Publication date |
|---|---|
| WO2024120167A1 (en) | 2024-06-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11665378B2 (en) | Establishment and use of time mapping based on interpolation using low-rate fingerprinting, to help facilitate frame-accurate content revision | |
| US9253533B1 (en) | Scene identification | |
| US10362366B2 (en) | Techniques for seamless media content switching during fixed-duration breaks | |
| US20170034263A1 (en) | Synchronized Playback of Streamed Audio Content by Multiple Internet-Capable Portable Devices | |
| US20120315014A1 (en) | Audio fingerprinting to bookmark a location within a video | |
| CN105933732B (en) | A kind of video playing duration statistical method and system | |
| US20200084342A1 (en) | Dynamically adjusting video to improve synchronization with audio | |
| CN111601136B (en) | Video data processing method and device, computer equipment and storage medium | |
| CN104918101B (en) | A kind of method, playback terminal and the system of automatic recording program | |
| US20200084491A1 (en) | Adaptive switching in a whole home entertainment system | |
| CN105828192A (en) | Multi-terminal video continuous playing method and device | |
| CN109756749A (en) | Video data handling procedure, device, server and storage medium | |
| CN108259998B (en) | Player, play control method and device, electronic equipment and play system | |
| CN107690093B (en) | Video playing method and device | |
| CN113301397A (en) | CDN-based audio and video transmission, playing and delay detection method and device | |
| CN110519627B (en) | Audio data synchronization method and device | |
| CN106791994A (en) | A kind of low delay quickly starts broadcasting method and device | |
| CN110012345B (en) | Channel switching method and television terminal | |
| CN105376605A (en) | Method for determining live broadcast time-shifting program and terminal equipment | |
| US20190037251A1 (en) | Playback apparatus, method of controlling playback apparatus, playback method and server apparatus | |
| CN118158484A (en) | Multimedia data processing method and device | |
| CN109547830B (en) | Method and device for synchronous playing of multiple virtual reality devices | |
| CN109194971A (en) | A kind of generation method and device for multimedia file | |
| CN112887786B (en) | Video playing method and device and computer readable medium | |
| CN113225583B (en) | Cloud game progress processing method and device and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication |