CN114615508A

CN114615508A - Decoding time detection method, decoding method and related equipment

Info

Publication number: CN114615508A
Application number: CN202011407867.0A
Authority: CN
Inventors: 曹洪彬; 金飞剑; 张伟; 王振祥; 楼剑
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2020-12-03
Filing date: 2020-12-03
Publication date: 2022-06-10

Abstract

The disclosure provides a decoding time detection method, a decoding method and related equipment, and relates to the technical field of computers and communication. The decoding time detection method comprises the following steps: determining the bin frame number s of a decoder, wherein s is an integer greater than or equal to 0; according to the bin frame number s, obtaining a first load decoding time of a kth video frame to be decoded in the video to be decoded, which is decoded by the decoder, wherein k is a positive integer greater than 1; according to the kth video frame to be decoded and the kth-1 video frame to be decoded, obtaining second load decoding time of the decoder for decoding the kth video frame to be decoded in the video to be decoded; and determining the target decoding time of the decoder for decoding the kth video frame to be decoded in the video to be decoded according to the first load decoding time and the second load decoding time. The scheme provided by the embodiment of the disclosure can improve the detection accuracy of the single-frame decoding time of the decoder, and can be applied to cloud game scenes realized based on a cloud server.

Description

Decoding time detection method, decoding method and related equipment

Technical Field

The present disclosure relates to the field of computer and communication technologies, and in particular, to a decoding time detection method, a decoding time detection apparatus, a decoding method, a decoding apparatus, an electronic device, and a computer-readable storage medium.

Background

In some application scenarios, it is necessary to accurately estimate the single-frame decoding time consumption of a hardware chip corresponding to a decoder for decoding a video code stream in a terminal device, but the single-frame decoding time consumption estimation manner in the related art does not consider the decoding hoarding and network jitter conditions of the hardware chip, so that the estimated single-frame decoding time consumption is not accurate enough, and the requirements in these application scenarios cannot be met.

Therefore, there is a need for a new decoding time detection method, decoding time detection apparatus, decoding method, decoding apparatus, electronic device, and computer-readable storage medium.

It is to be noted that the information disclosed in the above background section is only for enhancement of understanding of the background of the present disclosure.

Disclosure of Invention

The embodiment of the disclosure provides a decoding time detection method, a decoding time detection device, a decoding method, a decoding device, an electronic apparatus, and a computer-readable storage medium, which can improve the detection accuracy of a single-frame decoding time of a decoder.

Additional features and advantages of the disclosure will be set forth in the detailed description which follows, or in part will be obvious from the description, or may be learned by practice of the disclosure.

The embodiment of the disclosure provides a decoding time detection method, which comprises the following steps: determining the bin frame number s of a decoder, wherein s is an integer greater than or equal to 0; according to the bin frame number s, obtaining first load decoding time of a kth video frame to be decoded in the video to be decoded, which is decoded by the decoder, wherein k is a positive integer greater than 1; according to the kth video frame to be decoded and the kth-1 video frame to be decoded in the video frames to be decoded, obtaining second load decoding time of the decoder for decoding the kth video frame to be decoded in the video to be decoded; and determining the target decoding time of the decoder for decoding the kth video frame to be decoded in the video to be decoded according to the first load decoding time and the second load decoding time.

The embodiment of the present disclosure provides a decoding method, which includes: determining the bin frame number s of a decoder, wherein s is an integer greater than or equal to 0; according to the bin frame number s, obtaining a first load decoding time of a kth video frame to be decoded in the video to be decoded, which is decoded by the decoder, wherein k is a positive integer greater than 1; according to the kth video frame to be decoded and the kth-1 video frame to be decoded in the video frames to be decoded, obtaining second load decoding time of the decoder for decoding the kth video frame to be decoded in the video to be decoded; determining target decoding time for the decoder to decode the kth video frame to be decoded in the video to be decoded according to the first load decoding time and the second load decoding time; and decoding the video to be decoded according to the target decoding time of the kth video frame to be decoded.

The disclosed embodiment provides a decoding time detection device, which comprises: a decoder bin number determining unit for determining a bin number s of the decoder, s being an integer greater than or equal to 0; a first load decoding time obtaining unit, configured to obtain, according to the bin number s, a first load decoding time for the decoder to decode a kth video frame to be decoded in a video to be decoded, where k is a positive integer greater than 1; a second load decoding time obtaining unit, configured to obtain, according to a kth to-be-decoded video frame and a kth-1 to-be-decoded video frame in the to-be-decoded video frame, second load decoding time for the decoder to decode the kth to-be-decoded video frame in the to-be-decoded video; and the target decoding time obtaining unit is used for determining the target decoding time of the decoder for decoding the kth video frame to be decoded in the video to be decoded according to the first load decoding time and the second load decoding time.

In some exemplary embodiments of the present disclosure, the decoder bin number determining unit includes: a test code stream obtaining unit, configured to obtain a test code stream having the same video size as the video to be decoded; the test video frame input unit is used for inputting K frame test video frames in the test code stream to the decoder, K is a positive integer greater than or equal to 1, and the test video frames in the last first preset number in the K frame test video frames are not instant decoding refresh frames; the test decoding frame counting unit is used for counting T frame test decoding frames output by the decoder for decoding the test code stream, wherein T is a positive integer greater than or equal to 1; a bin number obtaining unit, configured to obtain a bin number s of the decoder according to the K-frame test video frame and the T-frame test decoded frame.

In some exemplary embodiments of the present disclosure, the decoder bin number determining unit includes: a preset video frame to be decoded obtaining unit, configured to obtain a second preset number of previous video frames to be decoded in the video to be decoded; a video frame to be decoded input unit, configured to obtain K in the second preset number of video frames to be decoded input to the decoder at the ith time_iFrame of video to be decoded, K_iIs a positive integer greater than or equal to 1, i is a positive integer greater than or equal to 1 and less than or equal to the second preset number; a decoded video frame output unit for recording the T output by the decoder decoding the second preset number of video frames to be decoded at the ith moment_iFrame decoding video frame, T_iIs a positive integer greater than or equal to 1; a bin number obtaining unit for obtaining the number of frames at a predetermined time according to the K_iFrame to be decoded video frame and said T_iFrame decoding video frames to obtain the storage frame quantity of the decoder at the ith moment; a bin number determining unit, configured to determine a bin number s of the decoder according to the bin number of the decoder at the i-th time.

In some exemplary embodiments of the present disclosure, the presetting-to-be-decoded video frame obtaining unit includes: a video frame type obtaining unit, configured to obtain a frame type of a video frame to be decoded in the video to be decoded; the instant decoding refresh frame removing unit is used for removing instant decoding refresh frames and video frames to be decoded with the frame types of the instant decoding refresh frames and the third preset number in front of and behind the instant decoding refresh frames from the video to be decoded; and the second number of video frames to be decoded is used for selecting the second preset number of video frames to be decoded from the rest videos to be decoded.

In this disclosureIn some exemplary embodiments, the video frame to be decoded input unit includes: a slow start unit for turning K at a predetermined speed at the j-th time_jInputting a frame to be decoded video frame to the decoder so that the frame to be decoded video frame does not exist in a buffer area of the decoder; wherein j is a positive integer greater than or equal to 1 and less than or equal to i.

In some exemplary embodiments of the present disclosure, the first load decoding time obtaining unit includes: a first timestamp obtaining unit, configured to obtain, according to the bin number s, a first timestamp that a (k + s) -th to-be-decoded video frame in the to-be-decoded video is input to the decoder; a second timestamp obtaining unit, configured to obtain a second timestamp for the decoder to decode a kth video frame to be decoded in the video to be decoded; and the first load decoding time calculation unit is used for obtaining the first load decoding time according to the first time stamp and the second time stamp.

In some exemplary embodiments of the present disclosure, the second load decoding time obtaining unit includes: a third timestamp obtaining unit, configured to obtain a third timestamp of a kth video frame to be decoded in the video to be decoded, which is decoded by the decoder; a fourth timestamp obtaining unit, configured to obtain a fourth timestamp for the decoder to decode a k-1 th video frame to be decoded in the video to be decoded; and the second load decoding time calculation unit is used for obtaining the second load decoding time according to the third time stamp and the fourth time stamp.

In some exemplary embodiments of the present disclosure, the target decoding time obtaining unit includes: a comparison unit, configured to compare the first load decoding time with the second load decoding time; a target decoding time determining unit, configured to determine a minimum value of the first payload decoding time and the second payload decoding time as a target decoding time for the decoder to decode the kth video frame to be decoded.

In some exemplary embodiments of the present disclosure, the apparatus further comprises: and the cloud game performance determining unit is used for determining whether to issue the cloud game video to the terminal equipment comprising the decoder according to the target decoding time of the kth video frame to be decoded in the video to be decoded by the decoder if the video to be decoded is the cloud game video.

The disclosed embodiment provides a decoding device, which comprises: a decoder bin number determining unit for determining a bin number s of the decoder, s being an integer greater than or equal to 0; a first load decoding time obtaining unit, configured to obtain, according to the bin number s, a first load decoding time for the decoder to decode a kth video frame to be decoded in a video to be decoded, where k is a positive integer greater than 1; a second load decoding time obtaining unit, configured to obtain, according to a kth to-be-decoded video frame and a kth-1 to-be-decoded video frame in the to-be-decoded video frame, second load decoding time for the decoder to decode the kth to-be-decoded video frame in the to-be-decoded video; a target decoding time obtaining unit, configured to determine, according to the first load decoding time and the second load decoding time, a target decoding time for the decoder to decode a kth video frame to be decoded in the video to be decoded; and the video decoding unit to be decoded is used for decoding the video to be decoded according to the target decoding time of the kth video frame to be decoded.

The disclosed embodiments provide a computer-readable storage medium on which a computer program is stored, which when executed by a processor implements the decoding time detection method and the decoding method as described in the above embodiments.

An embodiment of the present disclosure provides an electronic device, including: at least one processor; a storage device configured to store at least one program that, when executed by the at least one processor, causes the at least one processor to implement the decoding time detection method and the decoding method as described in the above embodiments.

Embodiments of the present disclosure provide a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the decoding time detection method and the decoding method provided in the various optional embodiments described above.

In the technical solutions provided in some embodiments of the present disclosure, on one hand, by determining the bin number s of the decoder, the first load decoding time of the decoder for decoding the kth to-be-decoded video frame in the to-be-decoded video can be obtained according to the determined bin number s of the decoder, that is, the detection manner of the single frame decoding time of the decoder in the case of decoding the bin is optimized in the embodiments of the present disclosure, so that the estimation of the single frame decoding time consumption of the decoder can be more accurate; on the other hand, a second load decoding time of the decoder for decoding the kth video frame to be decoded in the video to be decoded is further obtained according to the kth video frame to be decoded and the kth-1 video frame to be decoded in the video to be decoded, so that the second load decoding time is used for adapting scenes, such as uneven frame sending of the decoder, caused by network jitter and other factors. Meanwhile, the target decoding time of the decoder for decoding the kth to-be-decoded video frame in the to-be-decoded video is finally determined according to the first load decoding time and the second load decoding time determined in the manner, the decoder with the frame accumulation decoding condition is optimized, and the deviation of single frame decoding time estimation caused by uneven frame sending due to abnormal conditions such as network jitter is effectively avoided, so that the time consumed by single frame decoding of the decoder can be accurately detected. The scheme provided by the embodiment of the disclosure can be applied to, for example, a cloud game or other video application scenes with high estimation accuracy of the time consumed by decoding a single frame.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and together with the description, serve to explain the principles of the disclosure. It is to be understood that the drawings in the following description are merely exemplary of the disclosure, and that other drawings may be derived from those drawings by one of ordinary skill in the art without the exercise of inventive faculty. In the drawings:

fig. 1 schematically shows a flowchart of a decoding time detection method according to an embodiment of the present disclosure.

Fig. 2 schematically shows a decoder bin number of 3 according to an embodiment of the present disclosure.

Fig. 3 schematically shows a flow chart of step S110 in the embodiment of fig. 1 in an exemplary embodiment.

Fig. 4 schematically illustrates a block diagram of bin number calculation according to an embodiment of the disclosure.

Fig. 5 schematically shows a flow chart of step S110 in the embodiment of fig. 1 in an exemplary embodiment.

Fig. 6 schematically shows a flowchart of step S120 in the embodiment of fig. 1 in an exemplary embodiment.

Fig. 7 schematically shows a flowchart of step S130 in the embodiment of fig. 1 in an exemplary embodiment.

Fig. 8 schematically shows a flowchart of a decoding time detection method according to an embodiment of the present disclosure.

Fig. 9 schematically illustrates a schematic diagram of applying the decoding time detection method provided by the embodiment of the disclosure to a cloud game scene.

Fig. 10 schematically shows a flow chart of a decoding method according to an embodiment of the present disclosure.

Fig. 11 schematically shows a block diagram of a decoding time detection apparatus according to an embodiment of the present disclosure.

Fig. 12 schematically shows a block diagram of a decoding apparatus according to an embodiment of the present disclosure.

FIG. 13 shows a schematic structural diagram of an electronic device suitable for use in implementing embodiments of the present disclosure.

Detailed Description

Example embodiments will now be described more fully with reference to the accompanying drawings. Example embodiments may, however, be embodied in many different forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of example embodiments to those skilled in the art.

Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of embodiments of the disclosure. One skilled in the relevant art will recognize, however, that the subject matter of the present disclosure can be practiced without one or more of the specific details, or with other methods, components, devices, steps, and so forth. In other instances, well-known methods, devices, implementations, or operations have not been shown or described in detail to avoid obscuring aspects of the disclosure.

The block diagrams shown in the figures are functional entities only and do not necessarily correspond to physically separate entities. I.e. these functional entities may be implemented in the form of software, or in at least one hardware module or integrated circuit, or in different networks and/or processor means and/or microcontroller means.

The flow charts shown in the drawings are merely illustrative and do not necessarily include all of the contents and operations/steps, nor do they necessarily have to be performed in the order described. For example, some operations/steps may be decomposed, and some operations/steps may be combined or partially combined, so that the actual execution sequence may be changed according to the actual situation.

The single-frame decoding time-consuming calculation method of the hardware decoder in the related art is to record the time stamp of the video frame input to the decoder and the time stamp of the video frame output after the input video frame is decoded by the decoder, and calculate according to the difference value of the two time stamps, that is, according to the following formula:

in the above-mentioned formula (1),

a time stamp indicating the time when the code stream of the kth frame of video was sent to the decoder,

time stamp, Δ T, representing the decoded output image of the kth frame of video from the decoder (called the out-frame)_kRepresents the decoding time of the k-th frame of the video, k being a positive integer greater than 1.

However, the above-mentioned calculation scheme for calculating the time consumed for decoding a single frame in the related art does not take into consideration the situation that the hardware chip of the decoder may have a frame bin for decoding. The decoding of the bin frame means that the decoder outputs a decoded image only after a certain amount of video frame code streams are sent to the decoder.

For such hardware chips, the calculation of the time consumed by decoding a single frame in the related art is not accurate, and is represented by the following two aspects (here, 3 frames are taken as an example):

first, the calculated decoding time of the k-th frame includes the decoding time of the (k + 1) -th and (k + 2) -th frames.

Second, if the incoming time of the (k + 3) th frame fluctuates due to network jitter or other reasons, the outgoing time of the (k) th frame also fluctuates. Since these jitters may not be estimated, the decoding of the k-th frame is time consuming and computationally inaccurate.

In many video application scenarios, such as cloud games, time-consuming and accurate calculation of single-frame decoding of a hardware chip of a decoder is required to determine whether the hardware chip of a corresponding terminal device can enjoy a good cloud game service experience. The technical solutions in the related art cannot solve the problem of time-consuming accuracy of decoding a single frame of a decoder having a frame accumulation condition, and cannot meet the service requirements of such scenes.

The embodiment of the present disclosure provides a decoding time detection method, so as to solve at least one technical problem in the related art. The decoding time detection method and the decoding method provided by the embodiments of the present disclosure may be executed by any electronic device, such as a terminal device or a server. The terminal device may include, but is not limited to, any one or more of a smartphone, a tablet computer, a desktop computer, a notebook computer, a wearable smart device, a smart television, and the like. The server may be an independent server, a server cluster or a distributed system formed by a plurality of servers, or a cloud server providing basic cloud computing services such as cloud service, a cloud database, cloud computing, a cloud function, cloud storage, network service, cloud communication, middleware service, domain name service, security service, big data and artificial intelligence platforms and the like.

Fig. 1 schematically shows a flowchart of a decoding time detection method according to an embodiment of the present disclosure. The embodiment of fig. 1 is illustrated by taking a terminal device as an example, but the disclosure is not limited thereto. As shown in fig. 1, the method provided by the embodiment of the present disclosure may include the following steps.

In step S110, a bin number S of the decoder is determined, where S is an integer greater than or equal to 0.

In the embodiment of the present disclosure, the decoder is a hardware chip disposed in the terminal device for decoding the video to be decoded, and may have a capability of decoding the frame accumulation.

In the embodiment of the present disclosure, decoding the hoarding frame refers to a phenomenon that a hardware decoder starts to output a corresponding video frame to be decoded only after a certain number of video frames to be decoded in a video to be decoded are input.

For example, the number of bins may be 3 for a video stream of a video to be decoded with 1080p (p is an abbreviation of Progressive scan, representing Progressive scan, and generally a picture resolution of 1080p is 1920 × 1080 pixels) as hardware chips of a partial decoder. For the hardware chip with the storage frame number of 3, the decoded video frame of the 1 st frame to-be-decoded video frame is output from the decoder only after the code stream of the 4 th frame to-be-decoded video frame is input.

For example, as shown in fig. 2, taking the bin number of the decoder as 3 as an example, when the M +3 th to-be-decoded video frame, the M +2 th to-be-decoded video frame, and the input frame data of the M +1 th to-be-decoded video frame are input into the decoder, the decoder outputs the decoded frame data of the M th to-be-decoded video frame after decoding the M th to-be-decoded video frame, at this time, the M +4 th to-be-decoded video frame is not yet input into the decoder, and M is a positive integer greater than or equal to 1.

In the embodiment of the present disclosure, the bin number of the decoder may be obtained by using a test code stream test, or the bin number of the decoder may be obtained by directly using a video to be decoded, which may specifically refer to the embodiments shown in fig. 3 and fig. 5.

In step S120, a first payload decoding time of the decoder for decoding a kth video frame to be decoded in the video to be decoded is obtained according to the bin number S, where k is a positive integer greater than 1.

In the embodiment of the present disclosure, under the influence of external factors such as network jitter, the intervals between frames sent to the decoder are often uneven. In certain cases it may occur that many video frames to be decoded are sent at a time or that no video frames to be decoded are sent for a while.

When a video frame to be decoded is not input into the decoder for a period of time or the speed of inputting the video frame to be decoded is slow, the hardware chip of the decoder can decode the input video frame to be decoded in time, the buffer area of the decoder does not have the video frame to be decoded, and the state that the buffer area does not have the video frame to be decoded is called that the decoder is in a low-load state (or a first load state).

In the embodiment of the present disclosure, the first payload decoding time refers to the time consumed by the decoder in the low payload state to decode a single frame of the kth video frame to be decoded according to the number of frames stored in the decoder.

The specific way of calculating the first payload decoding time may refer to the embodiment of fig. 6 below.

In step S130, a second load decoding time of the decoder for decoding the kth to-be-decoded video frame in the to-be-decoded video is obtained according to the kth to-be-decoded video frame and the kth-1 to-be-decoded video frame in the to-be-decoded video frame.

When a large number of video frames to be decoded are sent at a time, a hardware chip of the decoder is short enough to finish decoding all the input video frames to be decoded, so that the input video frames to be decoded are accumulated in a buffer area of the decoder, the decoding of the video frames to be decoded in the buffer area is not started, the condition is called decoding frame accumulation, and the condition that the decoding frame accumulation occurs in the buffer area of the decoder is called that the decoder is in a high load state (or a second load state).

In the embodiment of the present disclosure, the second load decoding time refers to an estimated time consumed for decoding a single frame of a kth video frame to be decoded in a video to be decoded by a decoder in a high load state.

The specific way of calculating the second payload decoding time may refer to the embodiment of fig. 7 below.

In step S140, a target decoding time for the decoder to decode the kth video frame to be decoded in the video to be decoded is determined according to the first payload decoding time and the second payload decoding time.

In the embodiment of the present disclosure, the target decoding time for the decoder to decode the kth video frame to be decoded in the video to be decoded may be finally determined by comprehensively considering the first load decoding time of the decoder in the low load state and the second load decoding time of the decoder in the high load state obtained by the above calculation.

On one hand, by determining the bin number s of the decoder, the first load decoding time of the decoder for decoding the kth to-be-decoded video frame in the to-be-decoded video can be obtained according to the determined bin number s of the decoder, that is, the detection mode of the single-frame decoding time of the decoder with the bin decoding condition is optimized in the embodiment of the disclosure, so that the estimation of the single-frame decoding time consumption of the decoder can be more accurate; on the other hand, a second load decoding time of the decoder for decoding the kth to-be-decoded video frame in the to-be-decoded video is further obtained according to the kth to-be-decoded video frame and the kth-1 to-be-decoded video frame in the to-be-decoded video frame, so that the second load decoding time is used for adapting scenes, such as uneven frame sending of the decoder, brought by factors such as network jitter. Meanwhile, the target decoding time of the decoder for decoding the kth to-be-decoded video frame in the to-be-decoded video is finally determined according to the first load decoding time and the second load decoding time determined in the manner, the decoder with the frame accumulation decoding condition is optimized, and the deviation of single frame decoding time estimation caused by uneven frame sending due to abnormal conditions such as network jitter is effectively avoided, so that the time consumed by single frame decoding of the decoder can be accurately detected. The scheme provided by the embodiment of the disclosure can be applied to, for example, a cloud game or other video application scenes with high estimation accuracy of the time consumed by decoding a single frame.

Fig. 3 schematically shows a flow chart of step S110 in the embodiment of fig. 1 in an exemplary embodiment. As shown in fig. 3, step S110 in the embodiment of fig. 1 may further include the following steps.

In step S111, a test code stream having the same video size as the video to be decoded is obtained.

For example, if the video size of the video to be decoded is 1080p, the video size of the corresponding test code stream is also 1080p, but the present disclosure is not limited thereto.

In step S112, K frames of test video frames in the test code stream are input to the decoder, where K is a positive integer greater than or equal to 1, and a first preset number of test video frames located at the last in the K frames of test video frames are not instant decoding refresh frames.

In the embodiment of the present disclosure, an Instantaneous Decoding Refresh (IDR) frame is a special I-frame, i.e. all reference frames following the I-frame will only refer to the I-frame. The I frame (intra-frame coding frame) is an independent frame with all information, and can be independently decoded without referring to other images, the I frame and the IDR frame both use intra-frame prediction, and in the coding and decoding, the first I frame is called the IDR frame, so that the coding and decoding processes are conveniently controlled. The purpose of the IDR frame is to refresh immediately so that errors do not propagate, starting from the IDR frame, a new video sequence is recalculated to start encoding or decoding.

In step S113, a T frame test decoding frame output by the decoder decoding the test code stream is counted, where T is a positive integer greater than or equal to 1.

In step S114, a bin number S of the decoder is obtained according to the K frame test video frame and the T frame test decoded frame.

For example, as shown in fig. 4, assuming that a decoder inputs K frames of test video frames and decodes and outputs T frames of test decoded frames, the number of bin frames s of the video size corresponding to the video to be decoded of the decoder can be calculated according to the following formula:

s＝K-T (2)

that is, when calculating the bin number of the hardware chip of the decoder with a specific video size, a certain number (K frames) of test video frames may be sent to the decoder first, and the number of decoded frames (T frames) may be counted, and the bin number of the hardware chip of the decoder may be obtained according to the difference between the two numbers.

This way of counting bin numbers needs to ensure that the frame type of the test video frame that is finally sent to the decoder is not an IDR frame, since all reference frames following the IDR frame will only refer to this frame. Due to the semantics of the IDR frame, most hardware chips with decoding bin will clear the original internal bin after receiving the IDR frame, thus causing the inaccurate number of the calculated bins. For a hardware chip with bin number s, the bin test results of the IDR frame and frames before and after the IDR frame (s-1) are inaccurate and should be excluded.

In the embodiment of the disclosure, every K frames of test video frames in a test code stream are sequentially input to an encoder according to a time sequence, and T frames of test decoded frames output by a corresponding decoder are recorded, for the accuracy of the test result, the last test result can be selected as the final bin frame number of the decoder, that is, the difference value between the K frames of test video frames sent to the decoder last time and the T frames of test decoded frames output by the corresponding decoder is used as the bin frame number s of the decoder, and considering that in most cases, the bin frame number of a hardware chip is 3, so that in the test, it is ensured that the last 3 frames (the first preset number) of the test code stream are not IDR frames. The frame type of each frame of the coded test video frame can be controlled when the test code stream is coded.

The decoding time detection method provided by the embodiment of the disclosure can ensure that the test code stream adopted for testing the bin frame number of the decoder does not include an IDR frame or the last 3 frames are not IDR frames by controlling the frame type of each encoded test video frame, thereby ensuring the accuracy of the calculated bin frame number of the decoder.

Fig. 5 schematically shows a flow chart of step S110 in the embodiment of fig. 1 in an exemplary embodiment. As shown in fig. 5, step S110 in the embodiment of fig. 1 may further include the following steps.

In step S115, a second preset number of previous video frames to be decoded in the video to be decoded are obtained.

In an exemplary embodiment, obtaining a second preset number of video frames to be decoded in the video to be decoded may include: obtaining the frame type of a video frame to be decoded in the video to be decoded; removing the instant decoding refresh frame and the video frames to be decoded with the third preset number before and after the instant decoding refresh frame from the video to be decoded; and selecting the second preset number of video frames to be decoded from the rest videos to be decoded.

In step S116, K in the second preset number of video frames to be decoded input to the decoder at the ith time is obtained_iFrame of video to be decoded, K_iIs a positive integer greater than or equal to 1, and i is a positive integer greater than or equal to 1 and less than or equal to the second preset number.

In an exemplary embodiment, K of the second preset number of video frames to be decoded input to the decoder at the ith time is obtained_iThe frame to be decoded video frame may include: at time j, K is driven at a predetermined speed_jInputting a frame to be decoded video frame to the decoder so that the frame to be decoded video frame does not exist in a buffer area of the decoder; wherein j is a positive integer greater than or equal to 1 and less than or equal to i.

When the real-time detection mode of the embodiment of fig. 5 is adopted, slow start needs to be realized, that is, the first frames of video frames to be decoded cannot be sent to the decoder too fast, so that a hardware chip of the decoder has sufficient time to complete the decoding of each frame, and the phenomenon of code stream data accumulation of the video frames to be decoded caused by incomplete decoding of the decoder cannot occur, so that the calculation of the number of the hoarded frames is more accurate. In the embodiment of the present disclosure, the predetermined speed may be determined according to a specific decoder and a video frame to be decoded, as long as it can satisfy that no video frame to be decoded exists in a buffer of the decoder, that is, no frame accumulation occurs. For example, a threshold speed may be determined based on testing, the threshold speed being such that the decoder does not have an upper speed limit that can be reached by frame accumulation, and the predetermined speed may be selected to be less than or equal to the threshold speed.

In step S117, record T of the output of the decoder decoding the second preset number of video frames to be decoded at the ith time point_iFrame decoding video frame, T_iIs a positive integer greater than or equal to 1.

In step S118, according to the K_iFrame to be decoded video frame and said T_iAnd frame decoding the video frame to obtain the hoarding frame number of the decoder at the ith moment.

In step S119, the bin number S of the decoder is determined according to the bin number of the decoder at the i-th time.

In actual service, if there is no test code stream in the embodiment of fig. 3, the number of frames stored in the decoder can still be obtained by decoding the video to be decoded in real time. For example, the number of incoming frames (the video frames to be decoded are input to the decoder) and outgoing frames (the decoded video frames are output after the video frames to be decoded are decoded by the decoder) at each moment can be recorded in real time in the decoding process of the video to be decoded, the bin frame number at the corresponding moment is obtained according to the difference value of the two corresponding moments, the minimum value of the bin frame numbers at all the moments is taken as the bin frame number s of the decoder, and the real-time updating can be performed according to the decoding process:

s＝min{K_i-T_i} (3)

typically, the test is completed with the first 10 frames (the second predetermined number) of the video stream of the video to be decoded. Similarly, in order to ensure the accuracy of the above calculation result, the obtained first 10 frames of the video frame to be decoded should be the video frame excluding the IDR frame, and considering that the number of the bin frames is 3 in most cases, the calculation of formula (3) excludes the IDR frame and the data of the preceding and following 2 frames (the third predetermined number) of the video frame to be decoded, that is, the IDR frame itself and the 2 frames preceding and following the IDR frame need to be excluded.

The decoding time detection method provided by the embodiment of the disclosure can directly obtain the stocked frame number of the decoder by recording the real-time decoding process of the video frame to be decoded without using an additional test code stream.

Fig. 6 schematically shows a flowchart of step S120 in the embodiment of fig. 1 in an exemplary embodiment. As shown in fig. 6, step S120 in the embodiment of fig. 1 may further include the following steps.

In step S121, a first timestamp of the input of the k + S th to-be-decoded video frame in the to-be-decoded video to the decoder is obtained according to the bin number S.

In step S122, a second timestamp for the decoder to decode the kth video frame to be decoded in the video to be decoded is obtained.

In step S123, the first payload decoding time is obtained according to the first timestamp and the second timestamp.

In the embodiment of the present disclosure, the first time stamp of sending a frame to a decoder (inputting a video frame to be decoded to the decoder) and the second time stamp of outputting the frame may be recorded separately, and the calculation of the first load decoding time may be performed according to a difference between the first time stamp and the second time stamp:

in the above formula (4), Δ T_kRepresents the target decoding time of the decoder for decoding the kth video frame to be decoded in the video to be decoded,

a second timestamp representing a decoding of a kth video frame to be decoded in the video to be decoded by the decoder,

a first timestamp indicating that the (k + s) th to-be-decoded video frame in the to-be-decoded video is input into the decoder, where s is the bin frame number of the hardware chip of the decoder obtained by the above-mentioned embodiment, that is, the first load decoding time (single-frame hardware decoding time or single-frame decoding time) of the (k) th to-be-decoded video frame in the to-be-decoded video is calculated according to the difference between the first timestamp and the second timestamp.

The decoding time detection method provided by the embodiment of the disclosure avoids the influence of the frame accumulation situation of decoding of a hardware chip of a decoder on a calculation result when detecting the single-frame hardware decoding time of the decoder, and can improve the estimation accuracy of the single-frame hardware decoding time. For a hardware chip which does not bin frames, s is 0, and the calculation result is not affected.

Fig. 7 schematically shows a flow chart of step S130 in the embodiment of fig. 1 in an exemplary embodiment. As shown in fig. 7, step S130 in the above embodiment of fig. 1 may further include the following steps.

In step S131, a third timestamp for the decoder to decode the kth video frame to be decoded in the video to be decoded is obtained.

In step S132, a fourth timestamp for the decoder to decode the k-1 th video frame to be decoded in the video to be decoded is obtained.

In step S133, the second payload decoding time is obtained according to the third timestamp and the fourth timestamp.

In the embodiment of the present disclosure, the frame rate of the hardware chip of the decoder may also be calculated in the following manner, where the frame rate is measured by the second payload decoding time. The method comprises the following steps of recording a third time stamp of a last frame (for example, a frame decoded by a k-1 th video frame to be decoded) and a fourth time stamp of a current frame (for example, a frame decoded by a k-1 th video frame to be decoded) of a decoder, and calculating a second load decoding time according to a difference value of the third time stamp and the fourth time stamp:

in the above-mentioned formula,

a fourth timestamp representing the decoding of the k-1 th video frame of the video to be decoded from the decoder,

third time stamp, Δ P, representing the decoding of the kth video frame to be decoded from the decoder_kAnd a second loaded decoding time which represents that the decoder is in a high-load state (or a full-load state) to decode the kth video frame to be decoded is calculated according to the difference value of the third time stamp and the fourth time stamp.

The method of the above formula (5) is suitable for a scenario where the decoder is in a full-load operation, where the full-load state of the decoder means that a certain number of video frames to be decoded still exist in the decoder buffer, and the decoder does not run idle at this time.

In an exemplary embodiment, determining a target decoding time for the decoder to decode a kth video frame to be decoded in the video to be decoded according to the first payload decoding time and the second payload decoding time may include: comparing the first load decoding time with the second load decoding time; determining the minimum value of the first payload decoding time and the second payload decoding time as a target decoding time for the decoder to decode the kth video frame to be decoded.

In the embodiment of the present disclosure, the target decoding time Δ R of the hardware chip of the decoder having the frame accumulation decoding condition can be accurately calculated by combining the two calculation manners of the first load decoding time and the second load decoding time_kThe formula of (a):

ΔR_k＝min{ΔT_k,ΔP_k} (6)

the decoding time detection method provided by the embodiment of the disclosure may not be uniform in the frame transmission interval to the decoder under the influence of external factors such as network jitter. In certain situations it may happen that many frames are sent at a time (the decoder assumes a high load state) or that no frames are sent for a while (the decoder assumes a low load state). When the decoder is in a high load state, because the video frame accumulation to be decoded occurs in the decoder, a plurality of frames are to be decoded, and therefore the decoder always outputs the frames, and the calculation is more accurate by using the formula (5). The improvement using equation (4) above is more accurate when the decoder is in a low load state.

Fig. 8 schematically shows a flowchart of a decoding time detection method according to an embodiment of the present disclosure. The method provided by the embodiment of fig. 8 may be performed by any electronic device, such as a terminal device including a decoder therein, but the present disclosure is not limited thereto.

As shown in fig. 8, the method provided by the embodiment of the present disclosure may include the following steps.

In step S801, the bin number S of the video size of the video to be decoded corresponding to the decoder is determined.

The manner of calculating bin number s can be referred to the above embodiments.

In step S802, a first timestamp of the input of the k + S th video frame to be decoded in the video to be decoded into the decoder is recorded.

The manner in which the first timestamp is calculated may refer to the embodiments described above.

In step S803, a second timestamp of the input of the kth video frame to be decoded into the decoder in the video to be decoded is recorded.

The manner of calculating the second time stamp may refer to the above-described embodiment.

In step S804, a difference between the first timestamp and the second timestamp is calculated as a first payload decoding time for the decoder to decode a kth video frame to be decoded in the video to be decoded.

The manner of calculating the first payload decoding time may refer to the above-described embodiments.

In step S805, a third timestamp for the decoder to decode the kth video frame to be decoded in the video to be decoded is obtained.

The manner of calculating the third time stamp may refer to the above-described embodiment.

In step S806, a fourth timestamp for the decoder to decode the k-1 th video frame to be decoded in the video to be decoded is obtained.

The manner of calculating the fourth time stamp may refer to the above-described embodiments.

In step S807, the second payload decoding time is obtained from the third timestamp and the fourth timestamp.

The manner of calculating the second payload decoding time may refer to the above-described embodiment.

In step S808, when the decoder is in the low load state, the first load decoding time is used as a target decoding time for the decoder to decode the kth video frame to be decoded in the video to be decoded.

In step S809, when the decoder is in the high load state, the second load decoding time is taken as a target decoding time for the decoder to decode the kth video frame to be decoded in the video to be decoded.

The manner of calculating the target decoding time may refer to the above-described embodiments. In the embodiment of the present disclosure, in order to accurately calculate the time consumed by decoding a single frame of a decoder, the first payload decoding time and the second payload decoding time calculated in the following two ways are combined: firstly, a first time stamp of an incoming frame and a second time stamp of an outgoing frame, secondly, the outgoing frame speed of a hardware chip of the decoder, and the time consumption of single frame decoding of the decoder is estimated according to the minimum value of the two modes.

The decoding time detection method provided by the embodiment of the disclosure processes the frame accumulation condition of decoding of the hardware chip, effectively avoids the deviation caused by uneven frame sending due to abnormal conditions such as network jitter and the like, and can accurately calculate the time consumed by single frame decoding of the hardware chip.

In an exemplary embodiment, the method may further include: and if the video to be decoded is the cloud game video, determining whether to issue the cloud game video to the terminal equipment comprising the decoder according to the target decoding time of the kth video frame to be decoded in the video to be decoded by the decoder.

The method provided by the embodiment of the disclosure can be used for accurately estimating the single-frame decoding time (namely target decoding time) of a hardware chip of a decoder, and can be applied to cloud games or other video application scenes, such as real-time audio and video scenes of video call, interactive live broadcast, video conference and the like. The cloud game will be described as an example, but the present disclosure is not limited thereto.

The Cloud technology (Cloud technology) is a hosting technology for unifying series resources such as hardware, software, network and the like in a wide area network or a local area network to realize calculation, storage, processing and sharing of data. The cloud technology is a general term of network technology, information technology, integration technology, management platform technology, application technology and the like applied based on a cloud computing business model, can form a resource pool, is used as required, and is flexible and convenient. Cloud computing technology will become an important support. Background services of the technical network system require a large amount of computing and storage resources, such as video websites, picture-like websites and more web portals. With the high development and application of the internet industry, each article may have its own identification mark and needs to be transmitted to a background system for logic processing, data in different levels are processed separately, and various industrial data need strong system background support and can only be realized through cloud computing.

Among them, Cloud gaming (Cloud gaming) may also be called game on demand (gaming), which is an online game technology based on Cloud computing technology. Cloud game technology enables light-end devices (thin clients) with relatively limited graphics processing and data computing capabilities to run high-quality games. In a cloud game scene, a game is not executed in a player game terminal but is executed in a cloud server, and the cloud server renders the game scene into a video and audio stream which is transmitted to the player game terminal through a network. The player game terminal does not need to have strong graphic operation and data processing capacity, and only needs to have basic streaming media playing capacity and capacity of acquiring player input instructions and sending the instructions to the cloud server.

Referring to fig. 9, the terminal device 920 is connected to the cloud server 910 through a network 930, and the network 930 may be a wide area network or a local area network, or a combination of the two.

A hardware chip provided with a decoder on the terminal device 920 (running a client, such as a game client, an educational learning client, a search client, etc.) may be used to decode the cloud game video to be decoded, acquired from the cloud server 910.

The cloud game plays a game to the cloud server 910 instead of playing the game locally on the terminal device 920 of the user, the cloud game directly sends an operation instruction of a game player to the cloud server 910, the cloud server 910 performs image rendering, the rendered image is encoded into a video stream (a cloud game video to be decoded), and the video stream is sent to the terminal device 920 of the user to perform decoding display. When a user plays a game on the terminal device 920, the video coding quality is high and the time delay is small, so the effect looks the same as that of local rendering.

For the user's terminal device 920, the cloud game is played on the fly and does not need to be downloaded. In the past, when a user plays a game, an installation package needs to be downloaded on the terminal device 920, and the installation package is generally large, so that the user time is wasted, and the storage space of the terminal device 920 is also occupied. In the past, the cost of trying to play a game is high, and the cloud game is adopted, so that a user can play the game anytime and anywhere, and can easily experience various games.

Secondly, the local game needs to be rendered locally, the requirement on hardware configuration of the terminal device 920 of the game player is high, after the cloud game is adopted, the rendering is carried to the cloud server 910, and the requirement on configuration of the terminal device 920 of the user can be greatly reduced. Second, cloud games solve the problem of cross-platform compatibility. The video streaming scheme adopted by the cloud game solves the problem of intercommunication of terminal equipment of each platform.

It should be noted that the method provided by the embodiment of the present disclosure may be applied to a hardware capability detection logic of a terminal device of a cloud game service, and may also be applied to real-time statistics of a decoding process. And judging whether the terminal equipment supports high-quality cloud game service experience or not by acquiring the accurate time consumed for decoding the single-frame hardware. While also providing more accurate decoded data statistics for the decoder.

It should be noted that the above application scenarios are merely illustrated for the convenience of understanding the spirit and principles of the present disclosure, and the embodiments of the present disclosure are not limited in this respect. Rather, embodiments of the present disclosure may be applied to any scenario where an accurate estimate of the decoder's single frame hardware decoding overhead is required.

Fig. 10 schematically shows a flow chart of a decoding method according to an embodiment of the present disclosure. The decoding method provided by the embodiment of the present disclosure may be executed by any electronic device, for example, a terminal device, but the present disclosure is not limited thereto. As shown in fig. 10, a decoding method provided by an embodiment of the present disclosure may include the following steps.

In an exemplary embodiment, obtaining a first payload decoding time for the decoder to decode a kth video frame to be decoded in the video to be decoded according to the bin number s may include: according to the bin frame quantity s, obtaining a first time stamp of inputting a k + s video frame to be decoded in the video to be decoded into the decoder; obtaining a second time stamp of a kth video frame to be decoded in the video to be decoded which is decoded by the decoder; and obtaining the first load decoding time according to the first time stamp and the second time stamp.

In step S130, a second load decoding time of the decoder for decoding the kth video frame to be decoded in the video to be decoded is obtained according to the kth video frame to be decoded and the kth-1 video frame to be decoded in the video frame to be decoded.

In the embodiment of fig. 10, reference may be made to the description of the embodiments of fig. 1 to 9 above regarding a manner of specifically detecting and obtaining a target decoding time for a decoder to decode a kth video frame in the video to be decoded.

In step S1010, the video to be decoded is decoded according to the target decoding time of the kth video frame to be decoded.

According to the decoding method provided by the embodiment of the disclosure, the decoding process of the video to be decoded can be better controlled by acquiring the accurate single-frame decoding time of the decoder.

Fig. 11 schematically shows a block diagram of a decoding time detection apparatus according to an embodiment of the present disclosure. The decoding time detection device provided by the embodiment of the disclosure can be arranged in any electronic equipment, such as terminal equipment. As shown in fig. 11, the decoding time detection apparatus 1100 provided in the embodiment of the present disclosure may include: a decoder bin number determining unit 1110, a first payload decoding time obtaining unit 1120, a second payload decoding time obtaining unit 1130, and a target decoding time obtaining unit 1140.

In the embodiment of the present disclosure, the decoder bin number determining unit 1110 may be configured to determine a bin number s of the decoder, where s is an integer greater than or equal to 0. The first payload decoding time obtaining unit 1120 is configured to obtain a first payload decoding time for the decoder to decode a kth video frame to be decoded in the video to be decoded according to the bin number s, where k is a positive integer greater than 1. The second load decoding time obtaining unit 1130 may be configured to obtain, according to a kth video frame to be decoded and a (k-1) th video frame to be decoded in the video frames to be decoded, a second load decoding time for the decoder to decode the kth video frame to be decoded in the video to be decoded. The target decoding time obtaining unit 1140 may be configured to determine a target decoding time for the decoder to decode a kth video frame to be decoded in the video to be decoded according to the first payload decoding time and the second payload decoding time.

On one hand, by determining the bin number s of the decoder, the decoding time detection apparatus provided by the embodiment of the present disclosure can obtain the first load decoding time of the kth to-be-decoded video frame in the to-be-decoded video decoded by the decoder according to the determined bin number s of the decoder, that is, the detection method of the single frame decoding time of the decoder in the case of decoding the bin is optimized in the embodiment of the present disclosure, so that the estimation of the single frame decoding time consumption of the decoder can be more accurate; on the other hand, a second load decoding time of the decoder for decoding the kth to-be-decoded video frame in the to-be-decoded video is further obtained according to the kth to-be-decoded video frame and the kth-1 to-be-decoded video frame in the to-be-decoded video frame, so that the second load decoding time is used for adapting scenes, such as uneven frame sending of the decoder, brought by factors such as network jitter. Meanwhile, the target decoding time of the decoder for decoding the kth to-be-decoded video frame in the to-be-decoded video is finally determined according to the first load decoding time and the second load decoding time determined in the manner, the decoder with the frame accumulation decoding condition is optimized, and the deviation of single frame decoding time estimation caused by uneven frame sending due to abnormal conditions such as network jitter is effectively avoided, so that the time consumed by single frame decoding of the decoder can be accurately detected. The scheme provided by the embodiment of the disclosure can be applied to, for example, a cloud game or other video application scenes with high estimation accuracy of the time consumed by decoding a single frame.

In an exemplary embodiment, the decoder bin number determining unit 1110 may include: the test code stream obtaining unit can be used for obtaining a test code stream with the same video size as the video to be decoded; the test video frame input unit can be used for inputting K frame test video frames in the test code stream to the decoder, wherein K is a positive integer greater than or equal to 1, and the test video frames in the last first preset number in the K frame test video frames are not instant decoding refresh frames; the test decoding frame counting unit is used for counting T frame test decoding frames output by the decoder for decoding the test code stream, wherein T is a positive integer greater than or equal to 1; a bin number obtaining unit, configured to obtain a bin number s of the decoder according to the K-frame test video frame and the T-frame test decoded frame.

In an exemplary embodiment, the decoder bin number determining unit 1110 may include: the device comprises a preset video frame to be decoded obtaining unit, a decoding unit and a decoding unit, wherein the preset video frame to be decoded obtaining unit can be used for obtaining a second preset number of video frames to be decoded in the video to be decoded; a to-be-decoded video frame input unit, configured to acquire K in the second preset number of to-be-decoded video frames input to the decoder at the ith time_iFrame of video to be decoded, K_iIs a positive integer greater than or equal to 1, i is a positive integer greater than or equal to 1 and less than or equal to the second preset number; a decoded video frame output unit, configured to record T at the ith time when the decoder decodes the second preset number of video frames to be decoded for output_iFrame decoding video frame, T_iIs a positive integer greater than or equal to 1; a bin number obtaining unit at a predetermined time, operable to obtain a bin number based on said K_iFrame to be decoded video frame and said T_iFrame decoding video frames to obtain the storage frame quantity of the decoder at the ith moment; a bin number determining unit, configured to determine a bin number s of the decoder according to the bin number of the decoder at the i-th time.

In an exemplary embodiment, the presetting of the video frame to be decoded obtaining unit may include: a video frame type obtaining unit, configured to obtain a frame type of a video frame to be decoded in the video to be decoded; the instant decoding refresh frame removing unit can be used for removing the instant decoding refresh frames and the video frames to be decoded with the third preset number in front of and behind the instant decoding refresh frames from the video to be decoded; and the second number of video frames to be decoded is used for selecting the second preset number of video frames to be decoded from the rest videos to be decoded.

In an exemplary embodiment, the video frame to be decoded input unit may include: a slow start unit for setting K at a predetermined speed at the j-th time_jThe video frame to be decoded is input to the decoder so that the video frame to be decoded does not exist in a buffer of the decoder. Wherein j is a positive integer greater than or equal to 1 and less than or equal to i.

In an exemplary embodiment, the first load decoding time obtaining unit 1120 may include: a first timestamp obtaining unit, configured to obtain, according to the bin number s, a first timestamp of a k + s-th to-be-decoded video frame in the to-be-decoded video input to the decoder; a second timestamp obtaining unit, configured to obtain a second timestamp for the decoder to decode a kth video frame to be decoded in the video to be decoded; a first payload decoding time calculation unit, configured to obtain the first payload decoding time according to the first timestamp and the second timestamp.

In an exemplary embodiment, the second load decoding time obtaining unit 1130 may include: a third timestamp obtaining unit, configured to obtain a third timestamp for the decoder to decode a kth video frame to be decoded in the video to be decoded; a fourth time stamp obtaining unit, configured to obtain a fourth time stamp when the decoder decodes a (k-1) th video frame to be decoded in the video to be decoded; a second payload decoding time calculation unit, configured to obtain the second payload decoding time according to the third timestamp and the fourth timestamp.

In an exemplary embodiment, the target decoding time obtaining unit 1140 may include: a comparison unit, configured to compare the first load decoding time with the second load decoding time; a target decoding time determining unit, configured to determine a minimum value of the first payload decoding time and the second payload decoding time as a target decoding time for the decoder to decode the kth video frame to be decoded.

In an exemplary embodiment, the decoding time detecting apparatus 1100 may further include: the cloud game performance determining unit may be configured to determine whether to issue the cloud game video to a terminal device including the decoder according to target decoding time of a kth video frame to be decoded in the video to be decoded, which is decoded by the decoder, if the video to be decoded is the cloud game video.

The specific implementation of each unit in the decoding time detection apparatus provided in the embodiment of the present disclosure may refer to the content in the decoding time detection method, and is not described herein again.

Fig. 12 schematically shows a block diagram of a decoding apparatus according to an embodiment of the present disclosure. The decoding device provided by the embodiment of the disclosure can be arranged in any electronic equipment, such as a terminal device. As shown in fig. 12, a decoding apparatus 1200 provided by the embodiment of the present disclosure may include: a decoder bin number determining unit 1110, a first payload decoding time obtaining unit 1120, a second payload decoding time obtaining unit 1130, a target decoding time obtaining unit 1140, and a to-be-decoded video decoding unit 1210.

The decoder bin number determining unit 1110, the first payload decoding time obtaining unit 1120, the second payload decoding time obtaining unit 1130, and the target decoding time obtaining unit 1140 may be implemented as described above with reference to the embodiment of fig. 11. The to-be-decoded video decoding unit 1210 may be configured to decode the to-be-decoded video according to a target decoding time of the kth to-be-decoded video frame.

The specific implementation of each unit in the decoding apparatus provided in the embodiment of the present disclosure may refer to the content in the decoding method, and is not described herein again.

It should be noted that although in the above detailed description several units of the device for action execution are mentioned, this division is not mandatory. Indeed, the features and functions of two or more units described above may be embodied in one unit, in accordance with embodiments of the present disclosure. Conversely, the features and functions of one unit described above may be further divided into embodiments by a plurality of units.

The embodiments of the present disclosure provide a computer-readable storage medium on which a computer program is stored, which when executed by a processor, implements the decoding time detection method as described in the above embodiments.

The embodiments of the present disclosure provide a computer-readable storage medium on which a computer program is stored, which when executed by a processor implements the decoding method as described in the above embodiments.

An embodiment of the present disclosure provides an electronic device, including: at least one processor; a storage device configured to store at least one program that, when executed by the at least one processor, causes the at least one processor to implement the decoding time detection method as described in the above embodiments.

An embodiment of the present disclosure provides an electronic device, including: at least one processor; a storage device configured to store at least one program that, when executed by the at least one processor, causes the at least one processor to implement the decoding method as described in the above embodiments.

Embodiments of the present disclosure provide a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to cause the computer device to perform the decoding time detection method provided in the various alternative embodiments described above.

Embodiments of the present disclosure provide a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to cause the computer device to perform the decoding method provided in the various alternative embodiments described above.

FIG. 13 shows a schematic structural diagram of an electronic device suitable for use in implementing embodiments of the present disclosure. It should be noted that the electronic device 1300 shown in fig. 13 is only an example, and should not bring any limitation to the functions and the scope of the application of the embodiments of the present disclosure.

As shown in fig. 13, an electronic apparatus 1300 includes a Central Processing Unit (CPU)1301 that can perform various appropriate actions and processes in accordance with a program stored in a Read-Only Memory (ROM) 1302 or a program loaded from a storage portion 1308 into a Random Access Memory (RAM) 1303. In the RAM 1303, various programs and data necessary for system operation are also stored. The CPU1301, the ROM 1302, and the RAM 1303 are connected to each other via a bus 1304. An input/output (I/O) interface 1305 is also connected to bus 1304.

The following components are connected to the I/O interface 1305: an input portion 1306 including a keyboard, a mouse, and the like; an output section 1307 including a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, a speaker, and the like; a storage portion 1308 including a hard disk and the like; and a communication section 1309 including a Network interface card such as a LAN (Local Area Network) card, a modem, or the like. The communication section 1309 performs communication processing via a network such as the internet. A drive 1310 is also connected to the I/O interface 1305 as needed. A removable medium 1311 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 1310 as necessary, so that a computer program read out therefrom is mounted into the storage portion 1308 as necessary.

In particular, the processes described below with reference to the flowcharts may be implemented as computer software programs, according to embodiments of the present disclosure. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable storage medium, the computer program containing program code for performing the method illustrated by the flow chart. In such embodiments, the computer program may be downloaded and installed from a network through communications component 1309 and/or installed from removable media 1311. The computer program, when executed by a Central Processing Unit (CPU)1301, performs various functions defined in the methods and/or apparatus of the present application.

It should be noted that the computer readable storage medium shown in the present disclosure may be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having at least one wire, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a Read-Only Memory (ROM), an Erasable Programmable Read-Only Memory (EPROM) or flash Memory), an optical fiber, a portable compact disc Read-Only Memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In contrast, in the present disclosure, a computer-readable signal medium may include a propagated data signal with computer-readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable storage medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable storage medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF (Radio Frequency), etc., or any suitable combination of the foregoing.

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of methods, apparatus and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises at least one executable instruction for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The units described in the embodiments of the present disclosure may be implemented by software or hardware, and the described units may also be disposed in a processor. Wherein the names of the elements do not in some way constitute a limitation on the elements themselves.

As another aspect, the present application also provides a computer-readable storage medium, which may be included in the electronic device described in the above embodiments; or may exist separately without being assembled into the electronic device. The computer-readable storage medium carries one or more programs which, when executed by an electronic device, cause the electronic device to implement the method as described in the embodiments below. For example, the electronic device may implement the steps shown in fig. 1 or fig. 3 or fig. 5 or fig. 6 or fig. 7 or fig. 8 or fig. 10.

Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein may be implemented by software, or by software in combination with necessary hardware. Therefore, the technical solution according to the embodiments of the present disclosure may be embodied in the form of a software product, which may be stored in a non-volatile storage medium (which may be a CD-ROM, a usb disk, a removable hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which may be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the embodiments of the present disclosure.

Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This application is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains.

It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims. It will be understood that the present disclosure is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A decoding time detection method, comprising:

determining the bin frame number s of a decoder, wherein s is an integer greater than or equal to 0;

according to the bin frame number s, obtaining first load decoding time of a kth video frame to be decoded in the video to be decoded, which is decoded by the decoder, wherein k is a positive integer greater than 1;

according to the kth video frame to be decoded and the kth-1 video frame to be decoded in the video frames to be decoded, obtaining second load decoding time of the decoder for decoding the kth video frame to be decoded in the video to be decoded;

and determining the target decoding time of the decoder for decoding the kth video frame to be decoded in the video to be decoded according to the first load decoding time and the second load decoding time.

2. The method of claim 1, wherein determining the bin number s of the decoder comprises:

obtaining a test code stream with the same video size as the video to be decoded;

inputting K frames of test video frames in the test code stream to the decoder, wherein K is a positive integer greater than or equal to 1, and the test video frames in the last first preset number in the K frames of test video frames are not instant decoding refresh frames;

counting T frame test decoding frames output by the decoder for decoding the test code stream, wherein T is a positive integer greater than or equal to 1;

and obtaining the storage frame quantity s of the decoder according to the K frame test video frame and the T frame test decoding frame.

3. The method of claim 1, wherein determining the bin number s of the decoder comprises:

obtaining a second preset number of video frames to be decoded in the video to be decoded;

acquiring K in the second preset number of video frames to be decoded input to the decoder at the ith moment_iFrame of video to be decoded, K_iIs a positive integer greater than or equal to 1, i is a positive integer greater than or equal to 1 and less than or equal to the second preset number;

recording T of the output of the decoder decoding the second preset number of video frames to be decoded at the ith moment_iFrame decoding video frame, T_iIs a positive integer greater than or equal to 1;

according to said K_iFrame to be decoded video frame and said T_iFrame decoding video frames to obtain the storage frame quantity of the decoder at the ith moment;

and determining the bin frame quantity s of the decoder according to the bin frame quantity of the decoder at the ith moment.

4. The method of claim 3, wherein obtaining a second preset number of video frames to be decoded in the video to be decoded comprises:

obtaining the frame type of a video frame to be decoded in the video to be decoded;

removing the instant decoding refresh frame and the video frames to be decoded with the third preset number before and after the instant decoding refresh frame from the video to be decoded;

and selecting the second preset number of video frames to be decoded from the rest videos to be decoded.

5. The method of claim 3, whereinSelecting K in the second preset number of video frames to be decoded input to the decoder at the ith moment_iA frame-to-be-decoded video frame, comprising:

at time j, K is driven at a predetermined speed_jInputting a frame to be decoded video frame to the decoder so that the frame to be decoded video frame does not exist in a buffer area of the decoder;

wherein j is a positive integer greater than or equal to 1 and less than or equal to i.

6. The method of claim 1, wherein obtaining a first payload decoding time for the decoder to decode a kth video frame to be decoded in the video to be decoded according to the bin number s comprises:

according to the bin frame quantity s, obtaining a first time stamp of inputting a k + s video frame to be decoded in the video to be decoded into the decoder;

obtaining a second time stamp of a kth video frame to be decoded in the video to be decoded which is decoded by the decoder;

and obtaining the first load decoding time according to the first time stamp and the second time stamp.

7. The method according to claim 1, wherein obtaining the second payload decoding time for the decoder to decode the kth video frame to be decoded in the video to be decoded according to the kth video frame to be decoded and the kth-1 video frame to be decoded in the video frames to be decoded comprises:

obtaining a third timestamp of the decoder for decoding the kth video frame to be decoded in the video to be decoded;

obtaining a fourth time stamp of a k-1 video frame to be decoded in the video to be decoded by the decoder;

and obtaining the second load decoding time according to the third time stamp and the fourth time stamp.

8. The method of claim 1, wherein determining a target decoding time for the decoder to decode a kth video frame to be decoded in the video to be decoded according to the first payload decoding time and the second payload decoding time comprises:

comparing the first load decoding time with the second load decoding time;

determining the minimum value of the first loaded decoding time and the second loaded decoding time as a target decoding time for the decoder to decode the kth video frame to be decoded.

9. The method of claim 1, further comprising:

and if the video to be decoded is the cloud game video, determining whether to issue the cloud game video to the terminal equipment comprising the decoder according to the target decoding time of the kth video frame to be decoded in the video to be decoded by the decoder.

10. A method of decoding, comprising:

according to the bin frame number s, obtaining a first load decoding time of a kth video frame to be decoded in the video to be decoded, which is decoded by the decoder, wherein k is a positive integer greater than 1;

determining target decoding time for the decoder to decode the kth video frame to be decoded in the video to be decoded according to the first load decoding time and the second load decoding time;

and decoding the video to be decoded according to the target decoding time of the kth video frame to be decoded.

11. The method of claim 10, wherein obtaining a first payload decoding time for the decoder to decode a kth video frame to be decoded in the video to be decoded according to the bin number s comprises:

12. A decoding time detection apparatus, comprising:

a decoder bin number determining unit for determining a bin number s of the decoder, s being an integer greater than or equal to 0;

a first load decoding time obtaining unit, configured to obtain, according to the bin number s, a first load decoding time for the decoder to decode a kth video frame to be decoded in a video to be decoded, where k is a positive integer greater than 1;

a second load decoding time obtaining unit, configured to obtain, according to a kth to-be-decoded video frame and a kth-1 to-be-decoded video frame in the to-be-decoded video frame, second load decoding time for the decoder to decode the kth to-be-decoded video frame in the to-be-decoded video;

and the target decoding time obtaining unit is used for determining the target decoding time of the decoder for decoding the kth video frame to be decoded in the video to be decoded according to the first load decoding time and the second load decoding time.

13. A decoding apparatus, comprising:

a target decoding time obtaining unit, configured to determine, according to the first load decoding time and the second load decoding time, a target decoding time for the decoder to decode a kth video frame to be decoded in the video to be decoded;

and the video decoding unit to be decoded is used for decoding the video to be decoded according to the target decoding time of the kth video frame to be decoded.

14. An electronic device, comprising:

at least one processor;

a storage device configured to store at least one program that, when executed by the at least one processor, causes the at least one processor to implement the decoding time detection method according to any one of claims 1 to 9 or the decoding method according to any one of claims 10 to 11.

15. A computer-readable storage medium, in which a computer program is stored, which, when being executed by a processor, implements the decoding time detection method according to any one of claims 1 to 9 or the decoding method according to any one of claims 10 to 11.