CN117676222A

CN117676222A - Multimedia file playing method and device, computing equipment and storage medium

Info

Publication number: CN117676222A
Application number: CN202311633909.6A
Authority: CN
Inventors: 曾梓鹏; 胡清; 钟洪斌; 洪熙; 刘宇民; 周俊文
Original assignee: Zhuhai Kingsoft Digital Network Technology Co Ltd
Current assignee: Zhuhai Kingsoft Digital Network Technology Co Ltd
Priority date: 2023-11-30
Filing date: 2023-11-30
Publication date: 2024-03-08

Abstract

The embodiment of the specification provides a multimedia file playing method and device, computing equipment and storage medium; the method comprises the following steps: acquiring the current drawing progress aiming at the images in the target multimedia file; determining the residual drawing frame number of the multimedia file player under the condition that residual images in the target multimedia file are drawn according to initial drawing times in a target duration based on the image drawing frame rate of the multimedia file player, the audio playing frame rate of the target multimedia file and the current drawing progress; wherein the initial number of draws is determined based on the image drawing frame rate and the audio playing frame rate; determining the drawing times of each residual image of the target multimedia file in the target duration based on the residual drawing frame number; drawing each residual image according to the drawing times of each residual image; the method can improve the playing fluency of the multimedia file.

Description

Multimedia file playing method and device, computing equipment and storage medium

Technical Field

The embodiment of the specification relates to the technical field of computers, in particular to a multimedia file playing method; and also relates to a multimedia file playing device, a computing device and a computer readable storage medium.

Background

With the development of computer technology, various multimedia files are layered endlessly, and the requirements on the smoothness of playing the multimedia files are also higher and higher.

If the multimedia file is a video, when the video is played on the display device, there may be a situation that the images in the video are not synchronous with the audio. For this case, if the display speed of the image is slower than the playing speed of the audio, the frame loss is usually performed to directly draw the image corresponding to the playing progress of the audio; if the display speed of the image is faster than the playing speed of the audio, the current image is usually drawn repeatedly until the image and the audio are synchronized, so that the synchronization of the image and the audio is ensured.

However, in the playing process of the video file, the images are lost or blocked, so that the fluency of the video playing process is poor.

Disclosure of Invention

In view of this, the embodiments of the present disclosure provide a method for playing a multimedia file. One or more embodiments of the present disclosure relate to a multimedia file playing device, a computing device, a computer readable storage medium, and a computer program, which can improve the smoothness of playing a multimedia file.

According to an aspect of embodiments of the present disclosure, there is provided a multimedia file playing method, applied to a multimedia file player, the method including:

Acquiring the current drawing progress aiming at the images in the target multimedia file;

determining the residual drawing frame number of the multimedia file player under the condition that residual images in the target multimedia file are drawn according to initial drawing times in a target duration based on the image drawing frame rate of the multimedia file player, the audio playing frame rate of the target multimedia file and the current drawing progress; wherein the initial number of draws is determined based on the image drawing frame rate and the audio playing frame rate;

determining the drawing times of each residual image of the target multimedia file in the target duration based on the residual drawing frame number;

drawing each residual image according to the drawing times of each residual image

According to still another aspect of the embodiments of the present specification, there is provided a multimedia file playing device including:

the acquisition module is used for acquiring the current drawing progress of the image in the target multimedia file;

the first determining module is used for determining the residual drawing frame number of the multimedia file playing device under the condition that the residual image in the target multimedia file is drawn according to the initial drawing times in the target duration based on the image drawing frame rate of the multimedia file playing device, the audio playing frame rate of the target multimedia file and the current drawing progress; wherein the initial number of draws is determined based on the image drawing frame rate and the audio playing frame rate;

The second determining module is used for determining the drawing times of each residual image of the target multimedia file in the target duration based on the residual drawing frame number;

and the drawing module is used for drawing each residual image according to the drawing times of each residual image.

According to yet another aspect of embodiments of the present specification, there is provided a computing device comprising: a memory and a processor;

the memory is configured to store computer-executable instructions that, when executed by the processor, perform the steps of the method described above.

According to yet another aspect of embodiments of the present description, there is provided a computer-readable storage medium storing computer-executable instructions which, when executed by a processor, implement the steps of the above-described method.

According to a further aspect of the embodiments of the present description, there is provided a computer program, wherein the computer program, when executed in a computer, causes the computer to perform the steps of the above-described method.

In one embodiment of the present disclosure, the remaining drawing frame number of the multimedia file player may be determined when the remaining image in the target multimedia file is drawn according to the initial drawing number of times within the target duration based on the image drawing frame rate of the multimedia file player, the audio playing frame rate of the target multimedia file, and the current drawing progress, and then the drawing number of times of each remaining image may be adjusted based on the remaining drawing frame number. Therefore, the display of the images and the playing of the audio can be synchronized by integrally adjusting the drawing times of the residual images, instead of discarding certain images or repeatedly drawing certain images for a long time, the situation of image blocking or image jumping in the playing process of the multimedia file can be improved, the complete playing of the multimedia file can be ensured, and the smoothness of playing the multimedia file is improved.

Drawings

Fig. 1 is a flowchart of a method for playing a multimedia file according to an embodiment of the present disclosure;

fig. 2 is a flowchart of a video playing method according to an embodiment of the present disclosure;

fig. 3 is a schematic structural diagram of a multimedia file playing device according to an embodiment of the present disclosure;

FIG. 4 is a block diagram of a computing device according to one embodiment of the present disclosure.

Detailed Description

In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present description. This description may be embodied in many other forms than described herein and similarly generalized by those skilled in the art to whom this disclosure pertains without departing from the spirit of the disclosure and, therefore, this disclosure is not limited by the specific implementations disclosed below.

The terminology used in the one or more embodiments of the specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of the specification. As used in this specification, one or more embodiments and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used in one or more embodiments of the present specification refers to and encompasses any or all possible combinations of one or more of the associated listed items. The term "at least one" in one or more embodiments of the present specification refers to "one or more" and "a plurality" refers to "two or more". The term "comprising" is an open description and should be understood as "including but not limited to" and may include other content in addition to what has been described.

It should be understood that although the terms "first," "second," and the like may be used in one or more embodiments of the present description to describe various information, these information should not be limited to these terms. These terms are only used to distinguish one type of information from another. For example, "first" may also be referred to as "second" and, similarly, "second" may also be referred to as "first" without departing from the scope of one or more embodiments of the present description. The word "if" as used herein may be interpreted as "at … …" or "at … …" or "responsive to a determination", depending on the context.

Furthermore, user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, presented data, etc.) referred to in one or more embodiments of the present description are both user-authorized or fully authorized information and data by parties, and the collection, use, and processing of relevant data entails adherence to relevant standards and requirements and provision of corresponding operational portals for user selection of authorization or denial.

With the development of computer technology and display technology, more and more multimedia files (such as video and audio) are layered endlessly, and functions of a multimedia file player are also more and more abundant. By way of example, many self-research engines today may implement the functionality of a multimedia file player, such as an engine may implement the functionality of a player based on ffmpeg (Fast Forward Moving Picture Experts Group) components. The ffmpeg component is a set of open source computer programs that can be used to record and convert digital audio and video, and can convert it into streams. In the following, a multimedia file is taken as an example of video, which includes audio data and image data. The engine can decode the audio data stream and the image data stream in the multimedia file to be played through the ffmpeg component, and transmit the extracted audio data to the system audio playing component of the display device for playing after converting the format of the extracted audio data, and the extracted image data is saved by using a queue. When the image drawing interface of the engine needs to call the image data, the interface can acquire the image information from the queue to draw the picture.

The rendering frame rate of the image by the engine may be different from the audio playing frame rate and the image display frame rate required for playing the multimedia file (the audio playing frame rate and the image display frame rate required for the multimedia file are the same), and the audio playing component may play the audio directly based on the audio playing frame rate. Thus, as the engine draws the image, the actual display frame rate of the image may be different from the image display frame rate required by the multimedia file, and in this case, the playing of the audio may not be synchronous with the display of the image. For this case, the engine typically calculates a relative time difference between the audio and the currently extracted frame of the image, adjusts the extraction rate of the image frame based on the difference, discards the image frame extracted next farther from the audio frame until an image frame close to the audio frame is extracted, and performs drawing based on the image frame to achieve synchronization of the image and the audio. The synchronous mode can lead to the situation that the player loses pictures, so that the playing effect of the multimedia file is not satisfactory.

In the specification, the multimedia file playing method can improve the smoothness of playing the multimedia file, avoid frame loss and ensure the playing integrity of the multimedia file. The present specification relates to a multimedia file playing apparatus, a computing device, and a computer-readable storage medium, and will be described in detail in the following embodiments. The multimedia file playing method can be used for the multimedia file playing device and the computing equipment. For example, the multimedia file playing apparatus and the computing device may be a terminal device having a display function.

Fig. 1 is a flowchart of a method for playing a multimedia file according to an embodiment of the present disclosure, where the method may be used in a multimedia file player, for example, the multimedia file player may be installed in a multimedia file playing device. As shown in fig. 1, the method may include the steps of:

step 102, acquiring the current drawing progress aiming at the images in the target multimedia file.

Alternatively, the multimedia file player may be an engine installed in the display device, which may be an application development engine, such as may be a game application development engine. The engine may be a self-grinding engine, or may be an existing open source engine, which is not limited in the embodiment of the present disclosure.

The drawing frame rate (which may represent the drawing speed) of the image by the multimedia file player in the embodiments of the present disclosure may be changed, for example, the change of the drawing frame rate may be triggered by the user, or the drawing frame rate may be automatically changed by the multimedia file player according to the working requirement. For example, the image drawing frame rate of the multimedia file player can be 200 hertz, that is, drawing of 200 images is performed in one second; the image rendering frame rate of the multimedia file player may also be 100 hz, 60 hz or other values, which is not limited in the embodiment of the present disclosure.

The multimedia file playing device can play the target multimedia file, wherein the target multimedia file is any multimedia file played by the multimedia file playing device. During the playing of the target multimedia file, an audio playing component in the system of the multimedia file playing device may play audio frames in the target multimedia file, such as playing audio frames at an audio playing frame rate set by the target multimedia file. During the playing of the target multimedia file, the playing rate of the audio frames by the audio playing component can be kept unchanged, i.e. the audio frames are continuously played at the audio playing frame rate.

In the playing process of the target multimedia file, the multimedia file player in the multimedia file playing device can draw the image in the target multimedia file. The multimedia file player draws the image at its corresponding image drawing frame rate. In the playing process of the target multimedia file, the multimedia file player can periodically acquire the current drawing progress of the image in the target multimedia file.

For example, the current rendering schedule may be a playing time, that is, a predetermined playing time period between the currently rendered image frame and the first image frame in the target multimedia file. The current rendering schedule may also order the image frames of the currently rendered image frames in the target multimedia file. If the image display frame rate set for the target multimedia file is 30 hz, that is, 30 frames of images are displayed in one second, and the multimedia file player is currently drawing the 15 th frame of images in the first second, the current drawing progress of the images in the target multimedia file can be represented by 0.5 th second or 15 th frame. Any frame of image or any frame of image described later in the embodiment of the present specification refers to an image frame in the target multimedia file, and different frame of image refers to different image frames in the target multimedia file.

In the embodiment of the present disclosure, the multimedia file player may also obtain the current playing progress of the audio in the target multimedia file. The playing progress may refer to the representation mode of the current drawing progress of the image, which is not described herein.

104, determining the residual drawing frame number of the multimedia file player under the condition that the residual image in the target multimedia file is drawn according to the initial drawing times in the target duration based on the image drawing frame rate of the multimedia file player, the audio playing frame rate of the target multimedia file and the current drawing progress; wherein the initial number of draws is determined based on the image drawing frame rate and the audio playing frame rate.

The multimedia file player may acquire and record an audio play frame rate (i.e., an image display frame rate) of the target multimedia file when playing starts for the target multimedia file, so as to facilitate subsequent use. The audio play frame rate of the target multimedia file may be an attribute value of the target multimedia file, and the audio play frame rate may be directly carried in the data of the target multimedia file. The audio play frame rate may be determined at the time of acquisition of the target multimedia file or may be preset or adjusted by a worker.

The multimedia file player may also periodically acquire its current image rendering frame rate, as the image rendering frame rate of the multimedia file player may change. Such as the multimedia file player, can acquire the image rendering frame rate each time the current rendering progress for the image of the target multimedia file is determined. If the image drawing frame rate of the multimedia file player does not change, the multimedia file player can also only acquire the image drawing frame rate once for subsequent use.

The multimedia file player may determine an initial number of draws for the image in the target multimedia file based on the image drawing frame rate and the audio playing frame rate. The initial number of draws may be determined, for example, based on a ratio of an image drawing frame rate to the audio playback frame rate. In the embodiment of the present specification, the number of drawing times of the image in the multimedia file refers to the number of repeated drawing times of the multimedia file player for the frame image. The image drawing frame rate of the multimedia file player is generally greater than or equal to the image display frame rate (i.e. the audio playing frame rate) of the multimedia file, so that the multimedia file player can repeatedly draw certain images in the multimedia file, so as to ensure that the actual display condition of the images has smaller phase difference with the display condition of the target multimedia file requirement. The multimedia file player can recalculate the initial drawing times after each acquisition of the image drawing frame rate.

For example, if the image rendering frame rate of the multimedia file player is 200 hz and the image display frame rate of the target multimedia file is 30 hz, the initial rendering number may be a value obtained by rounding up or rounding down 200/30, and the initial rendering number may be 7 or 6. Thus, each frame of image in the target multimedia file can be initially set, and the multimedia file player can repeatedly draw 7 or 6 times, so that the display condition of the target multimedia file can be closer to the display condition required by the target multimedia file (namely 30 frames of images are displayed in one second).

The image rendering frame rate of the multimedia file player may not be exactly an integer multiple of the image display frame rate of the target multimedia file (as in the case of the above example). In this way, if the multimedia file player adopts the initial drawing times to draw each frame of image in the target multimedia file in the target time, the total drawing times of the multimedia file player are not equal to the total drawing times required by each frame of image in the target multimedia file, and at this time, the remaining drawing frame number of the multimedia file player in the target time can be determined, that is, the remaining drawing times are still remained when the remaining image in the target multimedia file is drawn according to the initial drawing times in the target time. Along the above example, if the target duration is one second and the initial drawing number is 6, the remaining drawing frame number of the multimedia file player in the target duration may be 200-6×30=20 frames. If the initial drawing number is 7, the remaining drawing frame number of the multimedia file player in the target duration may be 200-7×30= -10 frames.

Along with the drawing of the image in the target time length by the multimedia file player, the corresponding residual drawing frame number of the multimedia file player in the target time length can be changed, so that the multimedia file player can determine the corresponding residual drawing frame number based on the current drawing progress. For example, if the multimedia file player repeatedly draws 6 times for the drawn partial image and 7 times for the partial image within the target time period, the remaining drawing frame number of the multimedia file player for the subsequent remaining image within the target time period may be changed.

Still further, in the above example, assuming that the number of initial drawing frames is 6, the current drawing progress in the image drawing process of the target duration is 21 st frame, and 10 frames of images are repeatedly drawn 6 times in the previous 20 frames of images drawn before, and the other 10 frames are repeatedly drawn 7 times, then the remaining drawing frame number corresponding to the multimedia file player in the target duration may no longer be 20, but is the difference between the total remaining frame number of the multimedia file player in the target duration and the frame number required for drawing the remaining image frames with the initial drawing number, that is, (200-6×10-7×10) -6×10=10.

And 106, determining the drawing times of each residual image of the target multimedia file in the target duration based on the residual drawing frame number.

The residual image refers to an image which is not drawn by the multimedia file player in the multi-frame images which need to be drawn in the target duration in the target multimedia file. After the current residual drawing frame number in the target time is determined, the multimedia file player can determine the proper drawing times for each residual image in the target time based on the residual drawing frame number, so as to ensure that the difference between the number of drawn image frames in the target multimedia file in the target time and the audio playing frame rate of the target multimedia file is smaller, and ensure the audio and video synchronization effect.

In the embodiment of the present disclosure, for each current drawing schedule, the number of drawing times of each remaining image in the target duration may be calculated. As the multimedia file player draws the image, the number of subsequent draws determined for some remaining images within the target time period may change, and at this time, the drawing of the remaining images may be performed based on the latest number of draws.

And step 108, drawing each residual image according to the drawing times of each residual image.

Based on step 106, the multimedia file player can determine the number of times that the remaining images to be drawn in each frame of the target multimedia file need to be drawn repeatedly, and further can draw each frame of image according to the corresponding drawing number of times, so as to realize the playing of the target multimedia file.

In the embodiment of the present disclosure, the target multimedia file may be a video, and each playing period of the target multimedia file may correspond to an image and an audio, so that the target multimedia file may be played according to the method for playing a multimedia file provided in the embodiment of the present disclosure in the whole playing process of the target multimedia file. In some implementations, the partial play period of the target multimedia file may correspond to only images or only audio, such as the target multimedia being primarily an audio file with some small video being played in addition to only the partial play period. For such a multimedia file, the playing method provided by the embodiment of the present specification may be performed only in a period in which audio and images need to be played together, so as to achieve audio-video synchronization in the period.

In summary, in the method for playing a multimedia file provided in the embodiments of the present disclosure, the remaining number of drawing frames of the multimedia file player may be determined when the remaining image in the target multimedia file is drawn according to the initial number of drawing times within the target duration based on the image drawing frame rate of the multimedia file player, the audio playing frame rate of the target multimedia file, and the current drawing progress, and then the number of drawing times of each remaining image may be adjusted based on the remaining number of drawing frames. Therefore, the display of the images and the playing of the audio can be synchronized by integrally adjusting the drawing times of the residual images, instead of discarding certain images or repeatedly drawing certain images for a long time, the situation of image blocking or image jumping in the playing process of the multimedia file can be improved, the complete playing of the multimedia file can be ensured, and the smoothness of playing the multimedia file is improved.

The following describes a method for playing a multimedia file provided in the embodiments of the present specification, taking a multimedia file as a video, and taking a multimedia file player as a video player as an example. For multimedia files that are not video, the following method may be referred to for playing accordingly, which is not repeated in the embodiments of the present disclosure. Fig. 2 is a flowchart of a video playing method according to an embodiment of the present disclosure, where the method may be used in a video player, for example, the video player may be installed in a video playing device. As shown in fig. 2, the method may include the steps of:

step 202, acquiring the current drawing progress of the image in the target video.

In the embodiment of the present disclosure, step 202 may refer to the related description about step 102, and the description of the foregoing description is omitted here.

Step 204, determining the image drawing frame number of the video player and the audio playing frame number of the target video in the target duration based on the image drawing frame rate of the video player and the audio playing frame rate of the target video.

Optionally, the target duration may be equal to a unit duration of image drawing performed by the video player and a unit duration of audio playing, where the unit duration is 1 second, and the target duration may be 1 second. Alternatively, the target duration may be 0.5 seconds, 2 seconds, 3 seconds, or other duration. Regarding the image rendering frame rate and the audio playing frame rate of the target video, reference may be made to the relevant descriptions in step 102 and step 104, and will not be repeated here.

The image drawing frame rate refers to the number of frames drawn in one second, and the audio playing frame rate refers to the number of audio frames played in one second. When the target duration is one second, the image drawing frame number of the video player in the target duration is equal to the image drawing frame rate, and the audio playing frame number of the target video in the target duration is equal to the audio playing frame. Under the condition that the target time length is not equal to the unit time length corresponding to the image drawing frame rate, the ratio of the image drawing frame rate to the unit time length can be adopted to obtain the image drawing frame number of the video player in the target time length. Similarly, when the target duration is not equal to the unit duration corresponding to the audio playing frame rate, the ratio of the audio playing frame rate to the unit duration may be multiplied by the target duration to obtain the audio playing frame number in the target duration.

Step 206, determining the initial drawing times of the image in the target duration based on the drawing frame number of the image and the audio playing frame number.

In the embodiment of the present disclosure, step 206 may refer to the related description of step 104, and the description of the foregoing description is omitted here. The number of drawing of an image in a video refers to the number of repeated drawing of the frame image by the video player. The initial drawing times refer to drawing times which are initially set for images to be displayed in a target duration in a target video, and the actual drawing times of the respective images can be subsequently determined based on the initial drawing times.

For example, the video player may round up the ratio of the image drawing frame number k1 to the audio playing frame number k2, resulting in the initial drawing number a1.If k1=200, k2=30, a1=6. Or->If k1=200, k2=30, a1=7.

Step 208, determining the remaining drawing frame number of the video player in the target duration based on the image drawing frame number, the audio playing frame number and the initial drawing times.

In the embodiment of the present disclosure, step 208 may refer to the related description about step 104, and the description of the foregoing description is omitted here.

The remaining drawing frame number of the video player in the target duration refers to: starting from the current drawing progress of the image, if the video player draws the remaining image which is to be drawn but not drawn in the target time according to the initial drawing times, the drawing times of the video player will remain in the target time. The video player may determine a remaining drawing frame number a2 of the video player in the target time period based on the image drawing frame number k1, the audio playing frame number k2, and the initial drawing number a1 corresponding to the target time period.

For example, in the case where the current drawing progress of the image is the start time of the target duration, the video player may determine the first remaining drawing frame number a2, a2=k1—a1×k2 of the video player within the target duration based on the image drawing frame number k1, the audio playing frame number k2, and the initial drawing number a1. The manner of calculation may be understood in connection with the specific numerical example in step 104.

As another example, with the drawing of the image by the video player, in the case where the current drawing progress of the image is the middle time of the target time period, the video player may determine the second remaining drawing frame number a3 of the video player in the target time period at the current drawing progress based on the image drawing frame number k1 and the drawn frame number z of the video player in the target time period and the first remaining drawing frame number a 2. The drawn frame number z may be obtained based on a current drawing progress, which may be directly equal to the drawn frame number if the current drawing progress is expressed in terms of the frame number of the image currently drawn per unit time. The intermediate time refers to a time other than the start time within the target time period. a3 = (k 1-z) -a1 x (a 2-y), y being the remainder of z/a 1. The manner of calculation may be understood in connection with the specific numerical example in step 104.

Under the condition that the current drawing progress of the image is the middle time of the target time length, the video player can also take the remaining time length in the original target time length as the new target time length, and at the moment, the current drawing progress is the starting time of the new target time length, so that the remaining drawing frame number can be determined based on a2=k1-a1×k2.

The above-described remaining drawing frame number (such as the above-described first remaining drawing frame number and second remaining drawing frame number) may be the frame number of the drawing process that should be added to each frame image within the target period. The number of drawing times of partial images in the target duration can be adjusted based on the remaining drawing frame number, so that the partial images are not repeatedly drawn according to the initial drawing times, and the aim that all drawing frames of the video player in the target duration just realize the display of all frame images which should be displayed in the target duration is achieved. Therefore, at least synchronous alignment of image display and audio playing at the end of the target duration can be ensured, and the synchronous degree of image display and audio playing is higher in the process of the target duration. The drawing process of supplementing the remaining drawing frame number into each frame image can be realized by the following steps 210 to 216.

Step 210, determining the target drawing times based on the remaining drawing frame number and the initial drawing times.

In the embodiment of the present disclosure, step 210 may refer to the related description in step 106, and the description of the foregoing description is omitted here.

The video player may determine an adjustment value for the number of drawing times based on the remaining number of drawing frames determined in step 208; and then determining the target drawing times based on the adjustment value and the initial drawing times. The remaining drawing frame number is the total drawing frame number of the drawing process which should be allocated to each frame image, and for a specific image (such as a target image) to be allocated with the drawing frame number, the number of repeated drawing is not the initial drawing number any more, and the number of repeated drawing of the target image (i.e. the number of repeated drawing of the target image) can be obtained by adjusting the number of repeated drawing of the target image based on the initial drawing number.

The target drawing number may be obtained by increasing or decreasing the initial drawing number. The video player may first determine the amount of change to the initial number of drawings, that is, the adjustment value of the number of drawings, based on the total number of drawing frames to be apportioned (i.e., the remaining number of drawing frames). Then, the initial drawing times are adjusted according to the adjustment value, and the target drawing times are obtained.

The adjustment value needs to be smaller than the initial number of drawings. Still with the foregoing example, the initial drawing number a1=6, the video player may set the adjustment value to 1, and the target drawing number determined based thereon may be 7. Alternatively, the video player may set the adjustment value to other values such as 2 or 3.

The target image may be associated with the adjustment value, and the number of target images may be determined based on a ratio of the initial number of draws to the adjustment value. The initial drawing times can be integer times of the adjustment value, the number of the target images is equal to the integer times, the target images can be drawn conveniently by adopting the same target drawing times, and the stable display of each frame of image is ensured.

In the embodiment of the present specification, in the case where the remaining drawing frame number is greater than 0, the video player may determine the number of times obtained by adding one to the initial drawing number a1 as the target drawing number. The number of the remaining drawing frames is larger than 0, which means that some drawing frames are unused when the drawing of each frame image is performed according to the initial drawing times in the target time, and the drawing frames are also required to be shared for use by each frame image. Therefore, the number of drawing times of the partial image can be increased to share the drawing frames. Similarly, in the case where the number of remaining drawing frames is smaller than 0, the number of times obtained by subtracting one from the initial number of drawing is determined as the target number of drawing.

The corresponding adjustment amount in the mode of determining the target drawing times is 1, and the number of target images of the allocated drawing frames is equal to the remaining drawing frame number. Therefore, the residual drawing frame number is distributed to a plurality of target images as much as possible, and the target drawing times of each target image are as close to the initial drawing times as possible, so that the display time difference of each frame of image is ensured to be smaller, namely the display rate of the image is slowly changed, the stable display of each frame of image is ensured, and the sound and picture synchronization effect is further ensured to be better.

In the embodiment of the present specification, the following steps 212 and 214 are taken as examples, and a method of determining a target object for sharing the remaining drawing frame number will be described.

Step 212, determining an image frame interval corresponding to the target drawing times based on the audio playing frame number and the remaining drawing frame number in the target time.

In this embodiment of the present disclosure, the video player may obtain the image frame interval s1 corresponding to the target drawing number based on the ratio of the audio playing frame number k2 to the remaining drawing frame number (for example, the first remaining drawing frame number a 2). If the image frame interval is the absolute value of the ratio, i.e. s1= |k2/a2|. Still following the above example, k2=30, a2=20, s1=1.5.

Step 214, determining a plurality of target images meeting the image frame interval in each remaining image of the target video within the target duration, and determining the drawing times of the target images as target drawing times.

The video player may determine, as the target image, an image corresponding to the number of frames obtained by rounding an integer multiple of the image frame interval in each remaining image of the target video within the target time period. Alternatively, an image corresponding to the number of frames obtained by rounding up an integer multiple of the image frame interval may be determined as the target image, or an image corresponding to the number of frames obtained by rounding down an integer multiple of the image frame interval may be determined as the target image.

Still using the above example, k2=30, a2=20, s1=1.5, the number of target images is 20, and one target image exists every 1.5 frames among 30 frames of images that should be displayed in the target period. The corresponding 1×s1=1.5 value of the first target image is rounded up by the number of frames, so the first target image is the 2 nd frame image. The second target image is the 3 rd frame image, because the number of frames obtained by rounding up the corresponding 2×s1=3 value of the second target image. The corresponding 3×s1=4.5 value of the third target image is rounded up by the number of frames, so the third target image is the 5 th frame image. And so on to obtain all the target images.

For example, the video player may record its number of times it has been drawn b1 for each frame of image in the target video, record the number of frames of image b2 in the target video that have completed drawing within the target time period, and rank c1 in each frame of image that the next target image to be drawn should be drawn in the target time period. The initial value b1=b2=c1=0, b1 gradually increases as the image of the video player is drawn, and when b1 is equal to the number of drawing times corresponding to the image (for example, b1=a1), b1 is reset to the initial value, and drawing of the next frame image is started, b2 is incremented by one. When b2-c1 is not less than 0, c1=c1+s1 is reset to determine the next target image. And repeatedly drawing the target image by adopting the target drawing times. When the target duration is over, b2-c1 is less than 0, c1 can be set to zero to determine the target image in the next target duration.

Alternatively, the video player may set an identification value (as also denoted by b 1) of the current number of drawing times for each frame of image drawn. The initial value of b1 may be 0, and for each drawing of a frame of image, b1 may be increased by one until b1 is equal to the number of drawing times corresponding to the image (for example, b1=a1), b1 is reset to the initial value, and drawing of the next frame of image is started. For an image whose number of drawing is not the initial number of drawing a1, the initial value of b1 corresponding thereto may be changed, and b1 may also be reset to be equal to the initial value when b1=a1. For an image with the drawing times of a1+1, the initial value of b1 corresponding to the image can be changed to-1; for an image having a number of drawing of a1-1, the initial value of b1 corresponding to the image may be changed to 1.

In the embodiment of the specification, the distribution of each target image of the residual drawing frame number is more uniform based on the image frame interval, so that the overall display of each frame image in the target time period can be further ensured to be more stable.

Alternatively, the video player may not determine the target image based on the image frame interval. If the video player determines the target number of target images based on the remaining drawing frame number and the adjustment value, the video player can directly randomly determine the target number of images in each frame image to be displayed in the target duration as target images. As another example, the video player may also set as the target image the number of images of the successive target preceding or following the frame image.

And step 216, determining the drawing times of each residual image of the target video in the target duration based on the residual drawing frame number.

In the embodiment of the present disclosure, step 216 may refer to the related description in step 106, and the description of the foregoing description is omitted herein. The video player may determine the above-described target drawing number as the drawing number of the target image. The initial number of drawing times is determined as the number of drawing times of images other than the target image in the respective remaining images.

Step 218, drawing each residual image according to the drawing times of each residual image.

In the embodiment of the present disclosure, step 218 may refer to the related description in step 108, and the description thereof is omitted herein.

The current drawing progress of the image is represented by the number of frames in the embodiment of the present specification as an example. The current rendering schedule may also be represented by a relative time, such as a predetermined length of play between the currently rendered image frame and the first image frame in the target video. The video player may also determine the current playing progress of the audio, which may also be represented by a relative time, e.g., the relative time is the duration between the current time and the starting playing time of the audio, i.e., the played duration of the audio. The video player determines whether the display of the image and the playing of the audio are synchronous or not according to the current drawing progress of the image and the current playing progress of the audio, and then adjusts the drawing times of the video player on the image under the asynchronous condition so as to enable the display of the image and the playing of the audio to tend to be synchronous. Synchronization of the image display with the audio playback is achieved based on the above steps 204 to 218, as in the case of an unsynchronized.

Alternatively, the relative time identification value d may be set. When the difference value c > d between the current drawing progress of the image and the current playing progress of the audio indicates that the drawing of the image is faster than the playing of the audio, the drawing of the image needs to be slowed, and the repeated drawing times of the subsequent image can be equal to the initial drawing times plus one. When c < - > d, the drawing of the image is slower than the playing of the audio, and the drawing of the image is required to be faster, so that the repeated drawing times of the subsequent image can be equal to the initial drawing times minus one. The video player may periodically determine the difference c and, upon determining that-d.ltoreq.c.ltoreq.d, restore the number of repeated renderings of the image to the initial number of renderings.

In the embodiment of the present disclosure, the extraction rate of the video player for the image frames in the target video may be fixed or may be changed, which only needs to ensure that the extraction rate of the image is faster than or equal to the drawing rate of the image, and ensure that the frame image is already extracted into the queue when a certain frame image needs to be drawn. For example, the rate at which the video player extracts the image frames may be determined based on the value obtained by rounding down the ratio of the image rendering frame rate to the audio playing frame rate. If the value is 6, the video player extracts one image frame for each 6 drawing durations at the extraction rate of the image frame.

Optionally, the video player may also adjust the extraction rate of the images accordingly. When the image display is slower, the extraction rate of the image is also slower, so that excessive occupation of the memory by the extracted image is avoided. Alternatively, the image may be continuously extracted at a faster rate, ensuring that the image must have been extracted when it was rendered.

In the embodiment of the present disclosure, the image drawing frame rate of the video player is variable, so that the actual display effect of the image can be consistent with the required display effect of the video at any frame rate of the engine. When the drawing progress of the image is faster than the playing progress of the audio, the effect of waiting for the audio can be achieved that the image is displayed slightly slowly. When the playing progress of the audio is faster than the drawing progress of the image, the effect that the image slightly quickens the display of the trace audio can be realized. Therefore, the synchronization of the image frame and the audio frame can be realized in both aspects, and the blocking or jumping in the video playing process is reduced, so that the video playing effect is more in line with the user experience.

In summary, in the video playing method provided in the embodiments of the present disclosure, the remaining drawing frame number of the video player may be determined when the remaining image in the target video is drawn according to the initial drawing times in the target duration based on the image drawing frame rate of the video player, the audio playing frame rate of the target video, and the current drawing progress, and then the drawing times of each remaining image may be adjusted based on the remaining drawing frame number. Therefore, the display of the images and the playing of the audio can be synchronized by integrally adjusting the drawing times of the residual images, instead of discarding certain images or repeatedly drawing certain images for a long time, the situation of image blocking or image jumping in the video playing process can be improved, the complete playing of the video can be ensured, and the fluency of video playing is improved.

Corresponding to the above method embodiments, the present disclosure further provides an embodiment of a multimedia file playing device, and fig. 3 is a schematic structural diagram of a multimedia file playing device according to an embodiment of the present disclosure. As shown in fig. 3, the multimedia file playing device includes:

an obtaining module 301, configured to obtain a current drawing progress for an image in a target multimedia file;

a first determining module 302, configured to determine a remaining drawing frame number of the multimedia file playing device when the remaining image in the target multimedia file is drawn according to the initial drawing times within the target duration based on the image drawing frame rate of the multimedia file playing device, the audio playing frame rate of the target multimedia file, and the current drawing progress; wherein, the initial drawing times are determined based on the image drawing frame rate and the audio playing frame rate;

a second determining module 303, configured to determine the number of drawing times of each remaining image of the target multimedia file in the target duration based on the remaining drawing frame number;

and a drawing module 304, configured to draw each of the remaining images according to the drawing times of each of the remaining images.

Optionally, the first determining module 302 is configured to:

determining the image drawing frame number k1 of the multimedia file player and the audio playing frame number k2 of the target multimedia file in the target duration based on the image drawing frame rate of the multimedia file player and the audio playing frame rate of the target multimedia file;

Under the condition that the current drawing progress is the starting time of the target duration, determining initial drawing times a1 based on image drawing frame numbers k1 and audio playing frame numbers k2, and determining first remaining drawing frame numbers a2 of the multimedia file player in the target duration; where a1 is obtained by rounding k1/k2, a2=k1—a1×k2.

Optionally, the first determining module 302 is further configured to:

and under the condition that the current drawing progress is the middle time of the target time, determining the second residual drawing frame number of the multimedia file player in the target time based on the drawn frame number and the first residual drawing frame number of the multimedia file player in the target time.

Optionally, the second determining module 303 is configured to:

the first determining submodule is used for determining target drawing times based on the residual drawing frame number and the initial drawing times;

the second determining submodule is used for determining the drawing times of a plurality of target images as target drawing times in each residual image of the target multimedia file in the target time based on the audio playing frame number and the residual drawing frame number in the target time;

and a third determining sub-module, configured to determine that the number of drawing times of the images other than the target image in the remaining images is the initial number of drawing times.

Optionally, the first determining submodule is configured to:

determining an adjustment value of the number of drawing times based on the remaining number of drawing frames;

the target drawing number is determined based on the adjustment value and the initial drawing number.

Optionally, the first determining submodule is configured to:

when the number of remaining drawing frames is greater than 0, determining the number of initial drawing times plus one as a target drawing time;

in the case where the number of remaining drawing frames is less than 0, the number of times obtained by subtracting one from the initial number of drawing is determined as the target number of drawing.

Optionally, the second determining submodule is configured to:

determining an image frame interval corresponding to the target drawing times based on the audio playing frame number and the residual drawing frame number in the target time length;

and determining a plurality of target images meeting the image frame interval in each residual image of the target multimedia file in the target duration, and determining the drawing times of the target images as target drawing times.

Optionally, the second determining submodule is configured to:

and determining an image corresponding to the frame number obtained by rounding the integral multiple of the image frame interval in each residual image of the target multimedia file in the target time length as a target image.

In summary, in the multimedia file playing device provided in the embodiments of the present disclosure, the remaining drawing frame number of the multimedia file player may be determined when the remaining image in the target multimedia file is drawn according to the initial drawing times within the target duration based on the image drawing frame rate of the multimedia file player, the audio playing frame rate of the target multimedia file, and the current drawing progress, and then the drawing times of each remaining image may be adjusted based on the remaining drawing frame number. Therefore, the display of the images and the playing of the audio can be synchronized by integrally adjusting the drawing times of the residual images, instead of discarding certain images or repeatedly drawing certain images for a long time, the situation of image blocking or image jumping in the playing process of the multimedia file can be improved, the complete playing of the multimedia file can be ensured, and the smoothness of playing the multimedia file is improved.

In this specification, each embodiment is described in a progressive manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for the multimedia file playing device, since it is substantially similar to the multimedia file playing method embodiment, the description is relatively simple, and the relevant points will be referred to in the description of the multimedia file playing method embodiment.

FIG. 4 is a block diagram of a computing device according to one embodiment of the present disclosure. The components of the computing device 400 include, but are not limited to, a memory 410 and a processor 420. Processor 420 is coupled to memory 410 via bus 430 and database 450 is used to hold data.

Computing device 400 also includes access device 440, access device 440 enabling computing device 400 to communicate via one or more networks 460. Examples of such networks include public switched telephone networks (PSTN, public Switched Telephone Network), local area networks (LAN, local Area Network), wide area networks (WAN, wide Area Network), personal area networks (PAN, personal Area Network), or combinations of communication networks such as the internet. The access device 440 may include one or more of any type of network interface, wired or wireless, such as a network interface card (NIC, network interface controller), such as an IEEE802.11 wireless local area network (WLAN, wireless Local Area Network) wireless interface, a worldwide interoperability for microwave access (Wi-MAX, worldwide Interoperability for Microwave Access) interface, an ethernet interface, a universal serial bus (USB, universal Serial Bus) interface, a cellular network interface, a bluetooth interface, near field communication (NFC, near Field Communication).

In one embodiment of the present description, the above-described components of computing device 400, as well as other components not shown in FIG. 4, may also be connected to each other, such as by a bus. It should be understood that the block diagram of the computing device shown in FIG. 4 is for exemplary purposes only and is not intended to limit the scope of the present description. Those skilled in the art may add or replace other components as desired.

Computing device 400 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., tablet, personal digital assistant, laptop, notebook, netbook, etc.), mobile phone (e.g., smart phone), wearable computing device (e.g., smart watch, smart glasses, etc.), or other type of mobile device, or a stationary computing device such as a desktop computer or personal computer (PC, personal Computer). Computing device 400 may also be a mobile or stationary server.

Wherein the processor 420 is configured to execute computer-executable instructions that, when executed by the processor, implement the methods illustrated in fig. 1 or 2 described above.

In this specification, each embodiment is described in a progressive manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for the computing device embodiment, since it is substantially similar to the multimedia file playing method embodiment, the description is relatively simple, and reference is made to the partial description of the multimedia file playing method embodiment for relevant points.

An embodiment of the present disclosure also provides a computer readable storage medium storing computer instructions that, when executed by a processor, implement the steps of the multimedia file playing method described above. The computer instructions include computer program code that may be in source code form, object code form, executable file or some intermediate form, etc. The computer readable storage medium may include: any entity or device capable of carrying the computer program code, a recording medium, a U disk, a removable hard disk, a magnetic disk, an optical disk, a computer Memory, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), an electrical carrier signal, a telecommunications signal, a software distribution medium, and so forth. It should be noted that the content of the computer readable storage medium may be appropriately increased or decreased according to the requirements of jurisdictions in which the computer readable storage medium does not include electrical carrier signals and telecommunication signals, for example, according to jurisdictions in which the computer readable storage medium is not configured.

In this specification, each embodiment is described in a progressive manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for the computer readable storage medium embodiments, since they are substantially similar to the multimedia file playing method embodiments, the description is relatively simple, and the relevant points are found in the partial description of the multimedia file playing method embodiments.

An embodiment of the present disclosure further provides a computer program, where the computer program, when executed in a computer, causes the computer to perform the steps of the above-mentioned multimedia file playing method.

In this specification, each embodiment is described in a progressive manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for the computer program embodiment, since it is substantially similar to the multimedia file playing method embodiment, the description is relatively simple, and the relevant points are only referred to the partial description of the multimedia file playing method embodiment.

The foregoing describes specific embodiments of the present disclosure. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims can be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

It should be noted that the foregoing describes specific embodiments of the present invention. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims can be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous. Further, those skilled in the art will appreciate that the embodiments described in the specification are all preferred embodiments, and that the acts and modules referred to are not necessarily all required for the embodiments described in the specification.

In the foregoing embodiments, the descriptions of the embodiments are emphasized, and for parts of one embodiment that are not described in detail, reference may be made to the related descriptions of other embodiments.

The preferred embodiments of the present specification disclosed above are merely used to help clarify the present specification. Alternative embodiments are not intended to be exhaustive or to limit the invention to the precise form disclosed. Obviously, many modifications and variations are possible in light of the teaching of the embodiments. The embodiments were chosen and described in order to best explain the principles of the embodiments and the practical application, to thereby enable others skilled in the art to best understand and utilize the invention. This specification is to be limited only by the claims and the full scope and equivalents thereof.

Claims

1. A multimedia file playing method, applied to a multimedia file player, the method comprising:

and drawing each residual image according to the drawing times of each residual image.

2. The method according to claim 1, wherein the determining, based on the image rendering frame rate of the multimedia file player, the audio rendering frame rate of the target multimedia file, and the current rendering progress, a remaining rendering frame number of the multimedia file player in a case where the remaining image in the target multimedia file is rendered according to the initial rendering number of times within a target duration includes:

Determining an image drawing frame number k1 of the multimedia file player and an audio playing frame number k2 of the target multimedia file in a target duration based on the image drawing frame rate of the multimedia file player and the audio playing frame rate of the target multimedia file;

under the condition that the current drawing progress is the starting time of the target duration, determining initial drawing times a1 based on the image drawing frame number k1 and the audio playing frame number k2, and determining a first residual drawing frame number a2 of the multimedia file player in the target duration; wherein a1 is obtained by rounding k1/k2, and a2=k1-a1×k2.

3. The method according to claim 2, wherein the determining the remaining drawing frame number of the multimedia file player in the case of drawing the remaining image in the target multimedia file according to the initial drawing times within the target duration based on the image drawing frame rate of the multimedia file player, the audio playing frame rate of the target multimedia file, and the current drawing progress, further comprises:

and under the condition that the current drawing progress is the middle time of the target time length, determining a second residual drawing frame number of the multimedia file player in the target time length based on the drawn frame number of the multimedia file player and the first residual drawing frame number in the target time length.

4. The method according to claim 1, wherein determining the number of times of drawing each remaining image of the target multimedia file within the target time period based on the remaining number of drawing frames includes:

determining a target drawing number based on the remaining drawing frame number and the initial drawing number;

determining the drawing times of a plurality of target images as the target drawing times in each residual image of the target multimedia file in the target time based on the audio playing frame number and the residual drawing frame number in the target time;

and determining the drawing times of the images except the target image in the residual images as the initial drawing times.

5. The method of claim 4, wherein determining a target number of draws based on the remaining number of draw frames and the initial number of draws comprises:

determining an adjustment value of the drawing times based on the remaining drawing frame number;

and determining target drawing times based on the adjustment value and the initial drawing times.

6. The method according to claim 4 or 5, wherein the determining a target number of drawings based on the remaining number of drawings and the initial number of drawings includes:

Determining the initial drawing times plus one obtained times as the target drawing times under the condition that the residual drawing frame number is larger than 0;

and in the case where the number of remaining drawing frames is less than 0, determining the number of times obtained by subtracting one from the initial number of drawing times as the target number of drawing times.

7. The method of claim 4, wherein the determining, based on the number of audio playback frames and the number of remaining rendering frames in the target duration, the number of rendering of the plurality of target images among the remaining images of the target multimedia file in the target duration is the target number of rendering, comprises:

and determining a plurality of target images meeting the image frame interval in each residual image of the target multimedia file in the target duration, and determining the drawing times of the target images as the target drawing times.

8. The method of claim 7, wherein determining a target image that satisfies the image frame interval among the remaining images of the target multimedia file within the target time period comprises:

And determining an image corresponding to the frame number obtained by rounding the integral multiple of the image frame interval in each remaining image of the target multimedia file in the target time length as a target image.

9. A multimedia file playing device, characterized in that the multimedia file playing device comprises:

10. A computing device, comprising: a memory and a processor;

the memory is configured to store computer executable instructions, the processor being configured to execute the computer executable instructions, which when executed by the processor implement the method of any one of claims 1 to 8.

11. A computer readable storage medium, characterized in that computer executable instructions are stored, which when executed by a processor implement the method of any one of claims 1 to 8.