CN116801010A - Video stream playing method, device, equipment and storage medium - Google Patents

Abstract

The embodiment of the invention provides a video stream playing method, a device, equipment and a storage medium, which are applied to a player of a Web terminal and comprise the following steps: responding to a video playing request, and acquiring a video stream to be decoded corresponding to the video playing request, wherein the video stream to be decoded comprises a video frame to be decoded and an audio frame to be decoded; according to the video stream to be decoded, establishing a plurality of threads, wherein each thread corresponds to a video decoder; decoding the video frames to be decoded through a plurality of video decoders to obtain video frame data corresponding to the video streams to be decoded; decoding an audio frame to be decoded by an audio decoder to obtain audio frame data corresponding to a video stream to be decoded; video frame data and audio frame data are played. According to the scheme, under the condition that the safety requirements of video decoders which can only correspond to one single thread decoding for each thread are met, a plurality of video decoders are used for decoding video frames to be decoded, so that the decoding efficiency of the video frames is effectively improved, and the fluency of video playing at the Web end is ensured.

Description

Video stream playing method, device, equipment and storage medium

Technical Field

The present invention relates to the field of internet technologies, and in particular, to a video stream playing method, apparatus, device, and storage medium.

Background

Streaming media refers to a media format that is played on the internet in a streaming manner, such as a video stream. In order to reduce the transmission and storage costs of the video stream, the original video file corresponding to the video stream is generally encoded to compress redundant information in the video file; after the encoded video stream is obtained, it is decoded, i.e. decompressed, according to its corresponding encoding format to obtain a video file available for display. Among these, common codec formats include: advanced video coding (Advanced Video Coding, AVC for short), h.264, high efficiency video coding (High Efficiency Video Coding, HEVC for short), h.265, etc. When decoding, the video stream can be decoded by hardware through the video acceleration function of the display card, and can also be decoded by software through a decoder running on the CPU.

When the player of the Web terminal decodes the video stream, the Web terminal may not provide an application programming interface corresponding to the hardware decoding of the corresponding encoding format, but cannot perform hardware decoding, and only can decode the video stream by adopting a software decoding mode. When the video stream is software decoded, the decoding of video frames in the video stream and the decoding of audio frames in the video stream are included. Generally, the decoding speed of the player for the audio frames is faster, so that the player can meet the requirements of various playing scenes, but the decoding speed for the video frames is slower. When decoding video frames in a video stream, in general, a thread established by a player at a Web end is only allowed to call a single-thread decoded video decoder, so that the decoding efficiency of the single-thread decoded video decoder is limited, the decoding requirement of the video stream with higher frame rate cannot be met, and the user experience is affected, for example: video jamming and other conditions can occur in a double-speed playing scene.

Disclosure of Invention

The embodiment of the invention provides a video stream playing method, a device, equipment and a storage medium, which are used for improving the decoding efficiency of a Web end player on video frames in a video stream.

In a first aspect, an embodiment of the present invention provides a video stream playing method, which is applied to a player of a Web terminal, where the method includes:

responding to a video playing request, and acquiring a video stream to be decoded corresponding to the video playing request, wherein the video stream to be decoded comprises a video frame to be decoded and an audio frame to be decoded;

establishing a plurality of threads according to the video stream to be decoded, wherein each thread corresponds to a video decoder;

decoding the video frames to be decoded through a plurality of video decoders so as to obtain video frame data corresponding to the video streams to be decoded;

decoding the audio frame to be decoded through an audio decoder to obtain audio frame data corresponding to the video stream to be decoded;

and playing the video frame data and the audio frame data.

In a second aspect, an embodiment of the present invention provides a video stream playing device, which is applied to a player at a Web end, where the device includes:

the acquisition module is used for responding to a video playing request and acquiring a video stream to be decoded corresponding to the video playing request;

the processing module is used for establishing a plurality of threads according to the video stream to be decoded, and each thread corresponds to one video decoder respectively; decoding the video frames to be decoded through a plurality of video decoders so as to obtain video frame data corresponding to the video streams to be decoded; decoding the audio frame to be decoded through an audio decoder to obtain audio frame data corresponding to the video stream to be decoded;

and the playing module is used for playing the video frame data and the audio frame data.

In a third aspect, an embodiment of the present invention provides an electronic device, including: a memory, a processor, a communication interface; wherein the memory has executable code stored thereon, which when executed by the processor, causes the processor to at least implement the video stream playing method according to the first aspect.

In a fourth aspect, embodiments of the present invention provide a non-transitory machine-readable storage medium having executable code stored thereon, which when executed by a processor of an electronic device, causes the processor to at least implement a video stream playing method as described in the first aspect.

In the scheme provided by the embodiment of the invention, a player of a Web terminal responds to a video playing request to obtain a video stream to be decoded, which corresponds to the video playing request, wherein the video stream to be decoded comprises a video frame to be decoded and an audio frame to be decoded; then, according to the video stream to be decoded, establishing a plurality of threads, wherein each thread corresponds to one video decoder respectively, and decoding video frames to be decoded through the plurality of video decoders so as to obtain video frame data corresponding to the video stream to be decoded; decoding an audio frame to be decoded by an audio decoder to obtain audio frame data corresponding to a video stream to be decoded; finally, playing the video frame data and the audio frame data. In the scheme, the player of the Web end establishes a plurality of threads, so that under the safety requirement of the video decoder which can only correspond to one single-thread decoding for each thread, the video frame to be decoded in the video stream to be decoded can be decoded by using the video decoders which can be decoded by the plurality of single-thread decoding, thereby effectively improving the decoding efficiency of the video frame and ensuring the fluency of video playing of the Web end.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

Fig. 1 is a flowchart of a video stream playing method according to an embodiment of the present invention;

fig. 2 is a schematic diagram of a playing process of a video stream according to an embodiment of the present invention;

fig. 3 is a schematic diagram of another video stream playing process according to an embodiment of the present invention;

fig. 4 is a flowchart of another video stream playing method according to an embodiment of the present invention;

fig. 5 is a flowchart of another video stream playing method according to an embodiment of the present invention;

fig. 6 is a schematic diagram of still another video stream playing process according to an embodiment of the present invention;

fig. 7 is a schematic structural diagram of a video stream playing device according to an embodiment of the present invention;

fig. 8 is a schematic structural diagram of an electronic device corresponding to the video stream playing apparatus provided in the embodiment shown in fig. 7.

Detailed Description

For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments of the present invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.

It should be noted that, the user information (including but not limited to user equipment information, user personal information, etc.) and the data (including but not limited to data for analysis, stored data, presented data, etc.) related to the embodiments of the present invention are information and data authorized by the user or fully authorized by each party, and the collection, use and processing of the related data need to comply with the related laws and regulations and standards of the related country and region, and provide corresponding operation entries for the user to select authorization or rejection.

In addition, the sequence of steps in the method embodiments described below is only an example and is not strictly limited.

The video stream playing method provided by the embodiment of the invention can be executed by an electronic device, and the electronic device can be a terminal device such as a PC, a notebook computer, a smart phone and the like, and can also be a server. The server may be a physical server comprising a separate host, or may be a virtual server, or may be a cloud server or a server cluster.

Fig. 1 is a flowchart of a video stream playing method provided by an embodiment of the present invention, which is applied to a server of a Web terminal, as shown in fig. 1, and may include the following steps:

101. and responding to the video playing request, and acquiring a video stream to be decoded corresponding to the video playing request, wherein the video stream to be decoded comprises video frames to be decoded and audio frames to be decoded.

102. According to the video stream to be decoded, a plurality of threads are established, and each thread corresponds to one video decoder respectively.

103. And decoding the video frames to be decoded by a plurality of video decoders corresponding to a plurality of threads to obtain video frame data corresponding to the video stream to be decoded.

104. And decoding the audio frame to be decoded by an audio decoder to obtain audio frame data corresponding to the video stream to be decoded.

105. Video frame data and audio frame data are played.

Wherein, the player of the Web end refers to a player which can run in a Web browser. According to the difference of terminal equipment for installing the Web browser, the player of the Web terminal comprises: the type of the player at the Web end in the embodiment is not limited, for example, a player at the Web end of a computer and a player at the Web end of a mobile phone, such as a player at the Web end of a mobile phone. In addition, for convenience of description, the web-side player will be hereinafter abbreviated as a player.

In this embodiment, the video decoders mentioned without specific description are all single-thread decoding video decoders, and the single-thread decoding video decoders perform decoding operation by only one thread when decoding video frames.

Fig. 2 is a schematic diagram of a playing process of a video stream according to an embodiment of the present invention. As shown in fig. 2, when the player plays the video stream, the procedure can be summarized as follows: first, a video stream acquisition process, comprising: and responding to a video playing request of a user, acquiring a coded and compressed video stream, namely a video stream to be decoded, wherein the video stream to be decoded comprises video frames to be decoded and audio frames to be decoded. The video stream to be decoded is usually encapsulated into a corresponding video format for streaming after being encoded and compressed, for example: may be packaged in MP4, FLV, AVI, etc. formats. Second, a demultiplexing process comprising: the player disassembles the obtained encapsulated video stream to be decoded through the demultiplexer, so as to obtain a video frame (i.e. a video frame to be decoded) and an audio frame (i.e. an audio frame to be decoded) of which one frame is compressed. Third, the decoding process includes: the player is correspondingly provided with an audio decoder and a video decoder, and decodes the audio frames to be decoded through the audio decoder to obtain uncompressed audio frame data (hereinafter referred to as audio frame data) of one frame; for a video frame to be decoded, decoding is performed by a video decoder to obtain uncompressed video frame data (hereinafter referred to as video frame data) of one frame. When decoding a video frame to be decoded, specifically, the player establishes a thread, that is, starts a worker thread in the player, and calls a video decoder under the thread to decode the video frame to be decoded. Fourth, a playing process, including: transmitting uncompressed video frame data and uncompressed audio frame data to hardware associated with video playback, such as: a graphics processor (graphics processing unit, GPU) or the like for video playback.

The present solution focuses on the decoding process of the video frame to be decoded in the video stream playing process shown in fig. 2, and other contents related to video stream playing, such as: the demultiplexing process, the decoding process of the audio frame to be decoded, the playing process after decoding, etc. can refer to the related technology, and the description of this scheme is omitted.

In the related art, when decoding a video frame, in order to ensure security, a server corresponding to a Web end generally only allows to call a single-threaded decoded video decoder for one thread established by a player. It is readily appreciated that the decoding capabilities of single-threaded decoded video decoders are limited, and as video encoding techniques evolve and video playback requirements increase, the decoding capabilities of single-threaded decoded video decoders cannot accommodate the decoding requirements of some specific encoding formats or trick-play scenarios. Such as: the decoding capability of a video decoder corresponding to a player of a web end of some mobile phones cannot meet the video frame decoding requirement of a video stream in an H.265 coding format, so that the player cannot play the video stream coded in the H.265 coding format; and, for example: under the fast play, namely double-speed play scene, the decoding capability of a video decoder corresponding to the player is insufficient, so that the situation of clamping and the like of video play is caused, and the user experience is influenced.

In order to solve at least one of the above technical problems, an embodiment of the present invention provides a video stream playing method shown in fig. 1. The following describes the video stream playing method shown in fig. 1 with reference to fig. 3, and fig. 3 is a schematic diagram of another video stream playing process according to an embodiment of the present invention.

As shown in fig. 3, first, in response to a video play request of a user, a video stream to be decoded corresponding to the video play request is acquired. Then, according to the video stream to be decoded, n threads, namely thread 1, thread 2, … and thread n are established. Wherein each thread corresponds to a video decoder, respectively, such as: thread 1 corresponds to video decoder 1, i.e., thread 1 calls video decoder 1, similarly thread 2 corresponds to video decoder 2, …, and thread n corresponds to video decoder n. Then, the n video decoders corresponding to the n threads decode the video frames to be decoded in the obtained video stream to be decoded to obtain video frame data 1, video frame data 2, … and video frame data n. And determining video frame data corresponding to the video stream to be decoded according to the video frame data 1, the video frame data 2 and … and the video frame data n. And decoding the audio frames to be decoded in the video stream to be decoded through an audio decoder so as to obtain audio frame data corresponding to the video stream to be decoded. Finally, the video frame data decoded by the n video decoders and the audio frame data decoded by the audio decoders are synchronously played.

Wherein, optionally, the number of threads to be established can be determined according to the frame rate corresponding to the video stream to be decoded, wherein the frame rate is the number of video frames displayed in unit time. For example, when a frame rate corresponding to a video stream to be decoded is less than a first threshold, a first number of threads are established; when the frame rate corresponding to the video stream to be decoded is greater than or equal to a first threshold value and smaller than a second threshold value, establishing a second number of threads; and when the frame rate corresponding to the video stream to be decoded is greater than or equal to a second threshold value, establishing a third number of threads and the like. The first threshold value and the second threshold value and the corresponding thread number can be set in a self-defined mode.

It should be noted that, in this embodiment, the programming language corresponding to the player is different from the programming language corresponding to the decoder, for example: the player is implemented in JavaScript language, the decoder is implemented in c++, and in order to ensure that a thread built or started in the player can call the decoder, the call of the thread built in the player to the decoder is typically implemented by webempty, i.e. wasm.

If a player using the JavaScript language is regarded as a JS terminal and a wasm and a decoder using the C++ language are regarded as wasm terminals, in the related art, the JS terminal only establishes one thread, and only one video decoder can be called corresponding to the wasm terminal for safety; the video stream playing method provided in this embodiment may be understood as establishing multiple threads on the JS end, so that although each thread still only calls one video decoder, the multiple threads established on the JS end may implement parallel call of multiple video decoders on the wasm end, thereby improving the video decoding performance corresponding to the player.

In summary, when the video stream to be decoded is decoded, a plurality of threads are established according to the video stream to be decoded, and video frames to be decoded in the video stream to be decoded are decoded through a plurality of video decoders corresponding to the threads one by one, so that safety is considered, each thread is guaranteed to only call one video decoder and the video decoder is single-thread decoding, the video frames to be decoded in the same video stream to be decoded are simultaneously decoded through the plurality of threads, video decoding capacity of the corresponding decoder of the player is improved, video decoding efficiency is guaranteed, and the web player can adapt to various scene requirements of video stream playing.

In the practical application process, in the process of playing the video stream, the number of threads for decoding the video stream to be decoded can be determined by combining with the hardware environment corresponding to the player. As shown in fig. 4, fig. 4 is a flowchart of another video stream playing method according to an embodiment of the present invention, which is applied to a player at a Web end, and may include the following steps:

401. and responding to the video playing request, and acquiring a video stream to be decoded corresponding to the video playing request, wherein the video stream to be decoded comprises video frames to be decoded and audio frames to be decoded.

402. Establishing a test thread, wherein the test thread corresponds to a video decoder; and decoding a test video frame in a preset video stream to be tested by a video decoder corresponding to the test thread so as to determine the video decoding capability corresponding to the video decoder in the current hardware environment.

403. And establishing a plurality of threads according to the decoding capacity corresponding to the video decoder and the video stream to be decoded in the current hardware environment, wherein each thread corresponds to one video decoder.

404. Decoding the video frames to be decoded through a plurality of video decoders corresponding to a plurality of threads to obtain video frame data corresponding to the video streams to be decoded; and decoding the audio frame to be decoded by an audio decoder to obtain audio frame data corresponding to the video stream to be decoded.

405. Video frame data and audio frame data are played.

The specific implementation process of step 401, step 404 and step 405 may refer to the foregoing embodiments, and in this embodiment, a detailed description is omitted.

It will be appreciated that the video decoder, as a software program running on the CPU, has a video decoding capability that is related to the running state of the CPU, in other words, the video decoding capability of the video decoder will vary with the hardware environment in which it is located. For example, video decoders running on a certain CPU may exhibit different video decoding capabilities at different times, or the same video decoder running on a different CPU may exhibit different video decoding capabilities due to differences in CPUs.

In practical application, when a video frame to be decoded in a video stream to be decoded is required to be decoded, if the established threads are too many, resource waste is caused; if the number of threads to be built is too small, the video decoding requirements corresponding to the video stream to be decoded cannot be met.

In the implementation, in order to meet the video decoding requirement of the video stream to be decoded and avoid resource waste, before a plurality of threads are built for the video stream to be decoded, a test thread is built, and a video decoder is called by the test thread to decode a preset test video frame in the video stream to be tested, so that the video decoding capability corresponding to the video decoder in the current hardware environment is determined; then, based on the determined video decoding capability, the number of threads to be established for decoding video frames to be decoded in the video stream to be decoded is determined. The preset video stream to be tested may be a video stream containing a set number of test video frames, for example: a video stream comprising 15 test video frames. Wherein the test video frame has been encoded compressed as a video frame to be decoded.

Optionally, a plurality of video streams to be tested corresponding to a plurality of encoding formats may be stored in advance, when determining the video decoding capability corresponding to the video decoder in the current hardware environment, a target video stream to be tested with the same encoding format as that of the video stream to be decoded is determined from the plurality of video streams to be tested, video decoding is performed on the test video frames in the target video stream to be tested, and then, the video decoding capability corresponding to the video decoder in the current hardware environment is determined according to the video decoding result of the target video stream to be tested.

Alternatively, the decoding capability of a video decoder may be characterized by the number of video frames decoded per unit time; alternatively, the number of video frame data obtained by decoding is characterized.

Optionally, the video decoding requirement of the video stream to be decoded may be characterized by a frame rate corresponding to the video stream to be decoded. Accordingly, after determining the video decoding capability of the video decoder in the current hardware environment, the number of threads to be matched may be determined and established according to the video decoding capability of the video decoder in the current hardware environment and the frame rate corresponding to the video stream to be decoded. For example, assuming that the video decoder can decode m video frames in a unit time in the current hardware environment, and the frame rate corresponding to the video stream to be decoded is that 2m video frames are displayed in the unit time, 2 threads may be established for calling the 2 video decoders to decode the video stream to be decoded.

In this embodiment, when a plurality of threads are established for a video stream to be decoded, by testing the video decoding capability of a video decoder in the current hardware environment, a number of threads corresponding to the hardware environment and the video decoding requirement are established, so that the decoding requirement of the video stream to be decoded can be met without wasting resources.

Fig. 5 is a flowchart of another video stream playing method provided by an embodiment of the present invention, which is applied to a player at a Web end, and may include the following steps:

501. and responding to the video playing request, and acquiring a video stream to be decoded corresponding to the video playing request, wherein the video stream to be decoded comprises video frames to be decoded and audio frames to be decoded.

502. According to the video stream to be decoded, a plurality of threads are established, and each thread corresponds to one video decoder respectively.

503. And determining a plurality of target video frame queues to be decoded corresponding to the video stream to be decoded according to the number of threads corresponding to the plurality of threads, wherein the plurality of target video frame queues to be decoded correspond to the plurality of threads one by one.

504. And respectively decoding the video frames to be decoded contained in the target video frame queues to be decoded through the video decoders so as to obtain target video frame data respectively corresponding to the target video frame queues to be decoded.

505. And determining video frame data corresponding to the video stream to be decoded according to the target video frame data.

506. And decoding the audio frame to be decoded by an audio decoder to obtain audio frame data corresponding to the video stream to be decoded.

507. Video frame data and audio frame data are played.

The specific implementation process of step 501, step 502, step 506 and step 507 may refer to the foregoing embodiments, and the details of this embodiment are not repeated.

In practical applications, when a video stream to be decoded is transmitted, a video frame to be decoded is usually transmitted in units of a Group of pictures (GOP), where a Group of pictures is a Group of consecutive pictures in the video stream to be decoded, and a Group of pictures includes a plurality of video frames to be decoded.

It can be understood that, when the player acquires the video stream to be decoded, the player actually acquires the groups of pictures corresponding to the video stream to be decoded one by one. To ensure real-time decoding, the video decoder decodes the currently acquired group of pictures. In video decoding, decoding is performed in units of picture groups, so that determining a plurality of target to-be-decoded video frame queues corresponding to a to-be-decoded video stream may also be understood as determining a plurality of to-be-decoded picture group queues. Each time a player acquires a group of pictures, the player determines the target video frame queue to be decoded to which the player belongs, so that the video decoder can perform video decoding on the target video frame queue. In the implementation process, the number of the target video frame queues to be decoded is matched with the number of the established threads, each target video frame queue to be decoded corresponds to one thread, and each target video frame queue to be decoded comprises a plurality of video frames to be decoded by taking a picture group as a unit.

In an alternative embodiment, decoding, by a plurality of video decoders, a plurality of video frames to be decoded included in a frame group in a plurality of target video frame queues to be decoded respectively, to obtain target video frame data corresponding to the plurality of target video frame queues to be decoded respectively, includes: and determining target to-be-decoded video frame queues to which a plurality of picture groups contained in the to-be-decoded video stream respectively belong, and respectively decoding a plurality of to-be-decoded video frames contained in the picture groups in the target to-be-decoded video frame queues through a plurality of video decoders so as to obtain target video frame data corresponding to the target to-be-decoded video frame queues respectively.

Optionally, after a certain frame group is acquired by the player, the target to-be-decoded video frame queue to which the currently acquired frame group belongs may be determined according to the number of un-decoded frame groups respectively included in the current target to-be-decoded video frame queues.

For ease of understanding, this is illustrated in connection with fig. 6. Fig. 6 is a schematic diagram of a video stream playing process according to another embodiment of the present invention, as shown in fig. 6, assuming that a video stream to be decoded includes a plurality of groups of pictures, including GOP1, GOP2, GOP3, etc., when a player acquires the video stream to be decoded, the player sequentially acquires the plurality of groups of pictures according to the sequence of GOP1, GOP2, GOP3 …. Assuming that 2 threads are established according to the video stream to be decoded, correspondingly determining 2 target video frame queues to be decoded, namely a target video frame queue 1 to be decoded and a target video frame queue 2 to be decoded. Wherein, the target video frame queue 1 to be decoded corresponds to thread 1, and thread 1 calls video decoder 1; the target video frame queue 2 to be decoded corresponds to thread 2, thread 2 invoking video decoder 2.

When the player acquires the GOP1, since neither the target video frame queue 1 to be decoded nor the target video frame queue 2 to be decoded contains any group of pictures that are not decoded, the GOP1 may be assigned to the target video frame queue 1 or the GOP1 may be assigned to the target video frame queue 2 to be decoded, and in the case illustrated in fig. 6, the GOP1 is assigned to the target video frame queue 1 to be decoded.

When the player acquires the GOP2, since the target video frame queue 1 to be decoded already contains the GOP1, the GOP2 is assigned to the target video frame queue 2 to be decoded. Then, GOP3 is assigned to the target video frame queue 1 to be decoded, and so on. That is, after the player acquires a plurality of picture groups corresponding to the video stream to be decoded, the player alternately belongs the plurality of picture groups to the determined target video frame queues to be decoded according to the acquisition order.

For the group of pictures, GOP1, GOP3, etc., in the target video frame queue 1 to be decoded, it is decoded by the video decoder 1 to obtain target video frame data 1. For a group of pictures such as GOP2 in the target video frame queue 2 to be decoded, it is decoded by the video decoder 2 to obtain target video frame data 2. And then, according to the target video frame data 1 and the target video frame data 2, determining video frame data corresponding to the video stream to be decoded. And finally, playing video frame data corresponding to the video stream to be decoded and audio frame data obtained by decoding the audio frames to be decoded in the video stream to be decoded through an audio decoder.

In the scheme, a plurality of threads are established according to the video stream to be decoded, and a plurality of target video frame queues corresponding to the video stream to be decoded are determined according to the number of the established threads. When the video stream to be decoded is decoded, the video decoder corresponding to each thread decodes the picture group in the target video frame queue to be decoded corresponding to the thread, so that parallel decoding of a plurality of picture groups can be realized, video decoding efficiency is ensured, and the web player can adapt to various scene requirements of video stream playing.

A video stream playback apparatus of one or more embodiments of the present invention will be described in detail below. Those skilled in the art will appreciate that these means may be configured by the steps taught by the present solution using commercially available hardware components.

Fig. 7 is a schematic structural diagram of a video stream playing device according to an embodiment of the present invention, which is applied to a player at a Web end, as shown in fig. 7, and the device includes: an acquisition module 11, a processing module 12 and a playing module 13.

The obtaining module 11 is configured to obtain, in response to a video playing request, a video stream to be decoded corresponding to the video playing request, where the video stream to be decoded includes a video frame to be decoded and an audio frame to be decoded.

A processing module 12, configured to establish a plurality of threads according to the video stream to be decoded, where each thread corresponds to a video decoder; decoding the video frames to be decoded through a plurality of video decoders so as to obtain video frame data corresponding to the video streams to be decoded; and decoding the audio frame to be decoded through an audio decoder to obtain audio frame data corresponding to the video stream to be decoded.

A playing module 13, configured to play the video frame data and the audio frame data.

Optionally, the processing module 12 is specifically configured to establish a test thread, where the test thread corresponds to a video decoder; decoding a test video frame in a preset video stream to be tested by a video decoder corresponding to the test thread to determine the video decoding capability corresponding to the video decoder in the current hardware environment; and establishing a plurality of threads according to the video decoding capability and the video stream to be decoded.

Optionally, the processing module 12 is further specifically configured to establish a plurality of threads according to the video decoding capability and a frame rate corresponding to the video stream to be decoded.

Optionally, the processing module 12 is further specifically configured to determine a plurality of target to-be-decoded video frame queues corresponding to the to-be-decoded video stream according to the number of threads corresponding to the plurality of threads, where the plurality of target to-be-decoded video frame queues are in one-to-one correspondence with the plurality of threads; respectively decoding the video frames to be decoded contained in the target video frame queues to be decoded through a plurality of video decoders so as to obtain target video frame data respectively corresponding to the target video frame queues to be decoded; and determining video frame data corresponding to the video stream to be decoded according to the target video frame data.

Optionally, the video stream to be decoded includes a plurality of picture groups, each picture group includes a plurality of video frames to be decoded, and the processing module 12 is further specifically configured to determine a target video frame queue to be decoded to which the plurality of picture groups belong; and respectively decoding a plurality of video frames to be decoded contained in the picture groups in the target video frame queues to be decoded through a plurality of video decoders so as to obtain target video frame data respectively corresponding to the target video frame queues to be decoded.

Optionally, the processing module 12 is further specifically configured to determine, according to the number of non-decoded picture groups in the target to-be-decoded video frame queues, a target to-be-decoded video frame queue to which the currently acquired picture group belongs.

Optionally, the player uses a first programming language, the decoder uses a second programming language, and the thread established by the player invokes the decoder through WebAssembly.

The apparatus shown in fig. 7 may perform the steps described in the foregoing embodiments, and detailed execution and technical effects are referred to in the foregoing embodiments and are not described herein.

In one possible design, the structure of the video stream playing device shown in fig. 7 may be implemented as an electronic device, as shown in fig. 8, where the electronic device may include: memory 21, processor 22, communication interface 23. Wherein the memory 21 has stored thereon executable code which, when executed by the processor 22, causes the processor 22 to at least implement the video stream playing method as provided in the previous embodiments.

In addition, embodiments of the present invention provide a non-transitory machine-readable storage medium having executable code stored thereon, which when executed by a processor of an electronic device, causes the processor to at least implement a video stream playing method as provided in the foregoing embodiments.

The apparatus embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separate. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present invention without undue burden.

From the above description of the embodiments, it will be apparent to those skilled in the art that the embodiments may be implemented by adding necessary general purpose hardware platforms, or may be implemented by a combination of hardware and software. Based on such understanding, the foregoing aspects, in essence and portions contributing to the art, may be embodied in the form of a computer program product, which may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and are not limiting; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims (10)

1. A video stream playing method is characterized by being applied to a player of a Web terminal and comprising the following steps:

responding to a video playing request, and acquiring a video stream to be decoded corresponding to the video playing request, wherein the video stream to be decoded comprises a video frame to be decoded and an audio frame to be decoded;

establishing a plurality of threads according to the video stream to be decoded, wherein each thread corresponds to a video decoder;

decoding the video frames to be decoded through a plurality of video decoders so as to obtain video frame data corresponding to the video streams to be decoded;

decoding the audio frame to be decoded through an audio decoder to obtain audio frame data corresponding to the video stream to be decoded;

and playing the video frame data and the audio frame data.

2. The method of claim 1, wherein the establishing a plurality of threads from the video stream to be decoded comprises:

establishing a test thread, wherein the test thread corresponds to one video decoder;

decoding a test video frame in a preset video stream to be tested by a video decoder corresponding to the test thread to determine the video decoding capability corresponding to the video decoder in the current hardware environment;

and establishing a plurality of threads according to the video decoding capability and the video stream to be decoded.

3. The method of claim 2, wherein the establishing a plurality of threads based on the video decoding capability and the video to be decoded comprises:

and establishing a plurality of threads according to the video decoding capability and the frame rate corresponding to the video stream to be decoded.

4. The method of claim 1, wherein decoding the video frames to be decoded by a plurality of video decoders to obtain video frame data corresponding to the video stream to be decoded comprises:

determining a plurality of target video frame queues to be decoded corresponding to the video stream to be decoded according to the number of threads corresponding to the plurality of threads, wherein the plurality of target video frame queues to be decoded are in one-to-one correspondence with the plurality of threads;

respectively decoding the video frames to be decoded contained in the target video frame queues to be decoded through a plurality of video decoders so as to obtain target video frame data respectively corresponding to the target video frame queues to be decoded;

and determining video frame data corresponding to the video stream to be decoded according to the target video frame data.

5. The method according to claim 4, wherein the video stream to be decoded includes a plurality of groups of pictures, each group of pictures includes a plurality of video frames to be decoded, and the decoding, by a plurality of video decoders, of the video frames to be decoded included in the plurality of target video frame queues to be decoded, respectively, to obtain target video frame data corresponding to the plurality of target video frame queues to be decoded, respectively, includes:

determining a target video frame queue to be decoded to which the plurality of picture groups belong;

and respectively decoding a plurality of video frames to be decoded contained in the picture groups in the target video frame queues to be decoded through a plurality of video decoders so as to obtain target video frame data respectively corresponding to the target video frame queues to be decoded.

6. The method of claim 5, wherein said determining a target video frame queue to decode to which said plurality of groups of pictures belong comprises:

and determining the target video frame queues to be decoded, to which the currently acquired picture groups belong, according to the number of the picture groups which are not decoded in the target video frame queues to be decoded.

7. The method of any one of claims 1 to 6, wherein the player uses a first programming language and the decoder uses a second programming language, and wherein the thread established by the player invokes the decoder through WebAssembly.

8. A video stream playing device, characterized by being applied to a player at a Web end, comprising:

the device comprises an acquisition module, a video playing module and a decoding module, wherein the acquisition module is used for responding to a video playing request and acquiring a video stream to be decoded corresponding to the video playing request, and the video stream to be decoded comprises a video frame to be decoded and an audio frame to be decoded;

the processing module is used for establishing a plurality of threads according to the video stream to be decoded, and each thread corresponds to one video decoder respectively; decoding the video frames to be decoded through a plurality of video decoders so as to obtain video frame data corresponding to the video streams to be decoded; decoding the audio frame to be decoded through an audio decoder to obtain audio frame data corresponding to the video stream to be decoded;

and the playing module is used for playing the video frame data and the audio frame data.

9. An electronic device, comprising: a memory, a processor, a communication interface; wherein the memory has stored thereon executable code which, when executed by the processor, causes the processor to perform the video stream playback method of any one of claims 1 to 7.

10. A non-transitory machine-readable storage medium having stored thereon executable code which, when executed by a processor of an electronic device, causes the processor to perform the video stream playback method of any one of claims 1 to 7.