CN117082258A

CN117082258A - Video decoding method, device, equipment and storage medium

Info

Publication number: CN117082258A
Application number: CN202311042207.0A
Authority: CN
Inventors: 赵志立
Original assignee: Tencent Technology Beijing Co Ltd
Current assignee: Tencent Technology Beijing Co Ltd
Priority date: 2023-08-17
Filing date: 2023-08-17
Publication date: 2023-11-17

Abstract

A video decoding method, apparatus, device and storage medium are applied to an asymmetric processor system comprising a plurality of processor cores; in the application, the following components are added: grouping the processor cores based on the computing power of each of the plurality of processor cores to obtain at least one processor resource group; the computing power of the processor cores belonging to the same processor resource group is located in the same computing power interval; distributing corresponding processor resource groups for each video frame to be decoded according to respective decoding sequence information of each video frame to be decoded in the video to be decoded and a computing power interval corresponding to at least one processor resource group; the decoding sequence information characterizes the decoding sequence of the video frames to be decoded and the dependence information; and invoking at least one processor resource group to decode the video frame to be decoded. Aiming at an asymmetric processor system, video parallel decoding is provided, the service efficiency of a processor core is improved, the stability of decoding speed is ensured, and video playing experience is improved.

Description

Video decoding method, device, equipment and storage medium

Technical Field

The present application relates to the field of computer technologies, and in particular, to a video decoding method, apparatus, device, and storage medium.

Background

Because of the large data volume of video content, various video encoding compression techniques have been developed in order to reduce the network bandwidth occupation for transmitting video, and to reduce the storage space occupation for storing video.

With the evolution of video coding technology, current coding standards have increasingly high demands on computational power, including video coding computational power and video decoding computational power. For the computational power problem of video decoding, hardware decoding is widely adopted to solve the problem. At this time, the video decoding is updated along with the upgrade of the hardware equipment, but the hardware equipment is fixed when leaving the factory, and the hardware equipment is diversified and fragmented, so that the compatibility problem exists between the video decoding and the hardware equipment, and therefore, the processor software decoding is used as the supplement of the hardware decoding and is widely applied.

Processor software decoding is mainly affected by the main frequency of a processor, but as the main frequency of the processor is more and more difficult to raise, the current processor system is developed towards the multi-core direction. And for performance balancing and power consumption issues, an asymmetric processor system is proposed that includes multiple processor cores. Meanwhile, in order to fully utilize the multi-core capability and solve the problem of high computational power of video decoding, a video parallel decoding method is proposed.

Currently, when video is decoded in parallel by an asymmetric processor system comprising a plurality of processor cores, the asymmetric processor system randomly selects the processor cores to decode the video frames. At this time, the processor system with the asymmetric processor system with the larger computing power selects the last video frame in the plurality of video frames with the dependency relationship to decode, and the processor system with the smaller computing power selects the first video frame in the plurality of video frames with the dependency relationship to decode; finally, the processor cores with larger computation power in the asymmetric processor system are in idle states for a long time, so that the use efficiency of the processor cores in the asymmetric processor system is reduced, and the speed of decoding the video by the asymmetric processor system is reduced.

Therefore, for an asymmetric processor system including a plurality of processor cores, how to realize parallel decoding of video, improve the use efficiency of the processor cores, and ensure the stability of the video decoding speed is a technical problem to be solved at present.

Disclosure of Invention

The embodiment of the application provides a video decoding method, a device, equipment and a storage medium, which are used for providing a video parallel decoding method aiming at an asymmetric processor system comprising a plurality of processor cores, improving the service efficiency of the processor cores, ensuring the stability of video decoding speed and improving video playing experience.

In a first aspect, an embodiment of the present application provides a video decoding method applied to an asymmetric processor system including a plurality of processor cores, the method including:

grouping the processor cores based on the computing power of each of the plurality of processor cores to obtain at least one processor resource group; the computing power of the processor cores belonging to the same processor resource group is located in the same computing power interval;

according to the respective decoding order information of each video frame to be decoded in the video to be decoded and the respective calculation power interval of at least one processor resource group, distributing a corresponding processor resource group for each video frame to be decoded; the decoding sequence information characterizes the decoding sequence and the dependence information of the corresponding video frames to be decoded;

and invoking at least one processor resource group to decode the video frame to be decoded.

In a second aspect, an embodiment of the present application provides a video decoding apparatus applied to an asymmetric processor system including a plurality of processor cores, the apparatus comprising:

a grouping unit, configured to perform grouping processing on each processor core based on respective computing capabilities of the plurality of processor cores, to obtain at least one processor resource group; the computing power of the processor cores belonging to the same processor resource group is located in the same computing power interval;

The distribution unit is used for distributing corresponding processor resource groups for each video frame to be decoded according to respective decoding sequence information of each video frame to be decoded in the video to be decoded and respective calculation power intervals of at least one processor resource group; the decoding sequence information characterizes the decoding sequence and the dependence information of the corresponding video frames to be decoded;

and the decoding unit is used for calling at least one processor resource group and decoding the video frame to be decoded.

In one possible implementation, the allocation unit is specifically configured to:

determining a number of decoded video frames at a current time based on the at least one set of processor resources;

selecting at least one target video frame from the respective video frames to be decoded based on the first order of decoding time stamps and the number of decoded video frames; wherein the first order characterizes a monotonically increasing decoding timestamp;

and allocating a corresponding processor resource group for each target video frame in the at least one target video frame based on the respective computational power intervals of the at least one processor resource group.

In one possible implementation, if the number of decoded video frames is the number of groups of the processor resource group, and selecting a target video frame that is consistent with the number of decoded video frames; the allocation unit is specifically for:

Arranging all target video frames based on a first sequence of decoding time stamps to obtain a first video frame sequence;

arranging the at least one processor resource group based on a second order of the computational intervals of the at least one processor resource group to obtain a processor resource group sequence; the second sequence represents monotonically decreasing calculation force interval;

and respectively distributing each target video frame in the first video frame sequence to one processor resource group matched with the target video frame position in the processor resource group sequence.

In one possible implementation, if the number of decoded video frames is determined based on the number of groups of processor resource groups and the number of cores of the processor cores in each processor resource group, and selecting a target video frame that is consistent with the number of decoded video frames; the allocation unit is specifically for:

acquiring target video frames consistent with the core number from the first video frame sequence in sequence based on a second sequence of the calculation power intervals of at least one processor resource group and the core number of the processor cores in the processor resource group;

And correspondingly distributing one target video frame in the acquired target video frames consistent with the number of cores to one processor core in the matched processor resource group.

In one possible implementation, if a target video frame is selected; the allocation unit is specifically for:

dividing a target video frame into a plurality of pixel areas, and sequencing decoding priorities of the pixel areas to obtain an area decoding sequence;

arranging the at least one processor resource group based on a second order of the computational intervals of the at least one processor resource group to obtain a processor resource group sequence;

and allocating corresponding processor resource groups for each pixel region in the plurality of pixel regions based on the region decoding sequence and the processor resource group sequence.

In one possible implementation, if it is detected that there is a decoded video frame associated with the video to be decoded at the current time; the allocation unit is specifically for:

selecting at least one target video frame from the respective video frames to be decoded based on the first order of decoding time stamps and the number of decoded video frames;

Determining a processor resource group associated with each target video frame in at least one target video frame at the last moment, and acquiring an association result;

and respectively distributing corresponding processor resource groups for each target video frame in the at least one target video frame based on the respective calculation power interval of the at least one processor resource group and the association result.

In one possible implementation, there is one processor resource group in the association result to associate at least two target video frames; the number of the decoded video frames is determined based on the number of groups of the processor resource groups and the number of cores of the processor cores in each processor resource group, and a target video frame consistent with the number of the decoded video frames is selected; the distribution unit is specifically used for:

acquiring target video frames consistent with the core number from the first video frame sequence in sequence based on a second order of computing power of at least one processor resource group and the core number of the processor cores in the processor resource group;

and respectively distributing each obtained target video frame in the target video frames to one processor core in the processor resource group matched with the target video frame according to the association result.

if the target video frame is associated with one processor core in the matched processor resource group at the last moment, distributing the target video frame to the processor core associated with the last moment;

if the target video frame is not associated with one processor core in the matched processor resource group at the last moment, distributing the target video frame to the appointed processor core; the processor cores are designated as: and in the matched processor resource group, processor cores which are not associated with all the acquired target video frames at the last moment are not associated with each other.

In a third aspect, embodiments of the present application provide a computing device comprising: a memory and a processor, wherein the memory is used for storing a computer program; and a processor for executing a computer program to implement the steps of the video decoding method provided by the embodiment of the application.

In a fourth aspect, an embodiment of the present application provides a computer readable storage medium storing a computer program, which when executed by a processor, implements the steps of the video decoding method provided by the embodiment of the present application.

In a fifth aspect, embodiments of the present application provide a computer program product comprising a computer program stored in a computer readable storage medium; when a processor of a computing device reads the computer program from the computer-readable storage medium, the processor executes the computer program, so that the computing device performs the steps of the video decoding method provided by the embodiment of the present application.

The application has the following beneficial effects:

the embodiment of the application provides a video decoding method, a device, equipment and a storage medium, relates to the technical field of computers, in particular to a video decoding technology, and is applied to an asymmetric processor system comprising a plurality of processor cores. Taking into account the difference in computing power of the plurality of processor cores in the asymmetric processor system and the decoding order of the video frames to be decoded; in the embodiment of the application, firstly, based on the computing capacity of each of a plurality of processor cores, the processor cores with the computing capacity in the same computing capacity interval are assigned to the same processor resource group, so as to obtain at least one processor resource group; then, according to the respective decoding order information of each video frame to be decoded and the respective calculation power interval of at least one processor resource group in the video frames to be decoded, the respective processor resource group is allocated to each video frame to be decoded, the decoding order information characterizes the decoding order and the dependence information of the respective video frames to be decoded, and in the video decoding process, the processor resource group is allocated to the video frames to be decoded according to the decoding order and the dependence relation of the video frames to be decoded and the calculation power interval, instead of randomly selecting a processor core to perform video frame decoding; and finally, at least one processor resource group is called to respectively decode the corresponding video frames to be decoded.

The decoding sequence and the dependence information of the video frames to be decoded and the computing power of the processor cores are considered in the process of the processor resource allocation, so that the high-computing-power processor resource can be allocated to the video frames to be decoded with high priority, the situation that the high-computing-power processor cores are allocated to the video frames with low priority for decoding and the low-computing-power processor cores are allocated to the video frames with high priority for decoding due to random allocation is avoided, and the problems that the use efficiency of the high-computing-power processor cores is low and the video decoding speed is low due to the fact that the high-computing-power processor cores are in an idle state for a long time are further avoided; therefore, in the embodiment of the application, when the plurality of processor resource groups decode the video frames to be decoded simultaneously, the parallel decoding of the video in the asymmetric processor system comprising the plurality of processor cores is realized, the service efficiency of the processor cores and the video decoding speed are improved, the stability of the video decoding speed is ensured, and the video playing experience is improved.

Additional features and advantages of the application will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the application. The objectives and other advantages of the application will be realized and attained by the structure particularly pointed out in the written description and claims thereof as well as the appended drawings.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the description of the embodiments will be briefly described below, it will be apparent that the drawings in the following description are only some embodiments of the present application, and that other drawings can be obtained according to these drawings without inventive effort for a person skilled in the art.

FIG. 1 is a schematic diagram of frame-level multithreaded parallel decoding in an asymmetric processor system in accordance with the related art;

fig. 2 is a schematic diagram of an application scenario provided in an embodiment of the present application;

fig. 3 is a flowchart of a video decoding method according to an embodiment of the present application;

FIG. 4 is a schematic diagram illustrating a processor resource group partition according to an embodiment of the present application;

fig. 5 is a schematic diagram of video frame ordering to be decoded according to an embodiment of the present application;

fig. 6 is a schematic diagram of a first video frame allocation according to an embodiment of the present application;

fig. 7 is a schematic diagram of a second video frame allocation according to an embodiment of the present application;

fig. 8 is a schematic diagram of third video frame allocation according to an embodiment of the present application;

fig. 9 is a schematic diagram of a first video frame division provided in an embodiment of the present application;

Fig. 10 is a schematic diagram of a second video frame division provided in an embodiment of the present application;

FIG. 11 is a schematic diagram of a third video frame division provided in an embodiment of the present application;

fig. 12 is a schematic diagram of a fourth video frame allocation according to an embodiment of the present application;

FIG. 13 is a flowchart of another video decoding method according to an embodiment of the present application;

fig. 14 is a schematic diagram of a fifth video frame allocation according to an embodiment of the present application;

fig. 15 is a block diagram of a video decoding apparatus according to an embodiment of the present application;

FIG. 16 is a block diagram of a computing device according to an embodiment of the present application.

Detailed Description

In order to make the objects, technical solutions and advantageous effects of the present application more apparent, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application, and it is apparent that the described embodiments are only some embodiments, but not all embodiments of the present application. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.

Some terms in the embodiments of the present application are explained below to facilitate understanding by those skilled in the art.

An asymmetric processor system is also called an asymmetric multi-core processor system, and comprises a plurality of processor cores, wherein the computing capacities of the plurality of processor cores are not completely consistent; that is, an asymmetric processor system integrates one or more low power consumption low performance small processor cores while using one or more high power consumption high performance large processor cores. Asymmetric processor systems typically use a small number of general-purpose processor cores to handle functions requiring high software flexibility, while using a large number of hardware accelerators to handle as many tasks as possible.

The processor core is also called as a processor core and a processor core, and is the most important component part of the processor; is made of monocrystalline silicon by a certain production process. All computations, accept/store commands, process data, etc. of the asymmetric processor system are performed by the core. Various processor cores have fixed logic structures, and logic units such as a first-level cache, a second-level cache, an execution unit, an instruction-level unit, a bus interface and the like have scientific layout.

The computing power refers to the computing power of the computing device to achieve a specific result output by processing the data.

The video frame to be decoded is a decoded unfinished video frame, which includes: video frames that are not decoded, and video frames that are decoded but not completed.

Hardware decoding is to decode video by hardware. Hardware decoding is performed by a graphics processor (Graphics Processing Unit, GPU), which can reduce processor workload and power consumption. The graphic processor is also called as a display core, a visual processor and a display chip, and is a microprocessor which is specially used for image operation on personal computers, workstations, game machines and some mobile devices (such as tablet computers, smart phones and the like).

Processor software decoding is performed by a processor occupied by the software itself, and because processor software decoding increases processor workload, excessive mobile processor resources are occupied, and if the processor capacity is insufficient, the software is also affected.

The processor main frequency is the clock frequency of the processor and is also the operating frequency of the processor. The formula is: main frequency = external frequency x double frequency, wherein the external frequency is the bus clock frequency, and double frequency is the multiple of the phase difference between the external frequency and the main frequency of the processor. Generally, the number of instructions completed in one clock cycle is fixed, so the higher the main frequency, the faster the processor speed.

Motion compensation is an efficient way to predict, compensate for, the current local picture from the previous local picture, and it is an efficient way to reduce the frame sequence redundancy information. Motion compensation is a method of describing the difference between adjacent frames (adjacent here means adjacent in coding, not necessarily adjacent in playing order), and specifically how each tile of the previous frame (here means previous in coding, not necessarily previous in playing order to the current frame) moves to a certain position in the current frame. This approach is often used by video compression/video codecs to reduce spatial redundancy in video sequences.

The word "exemplary" is used hereinafter to mean "serving as an example, embodiment, or illustration. Any embodiment described as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments.

The terms "first," "second," and the like herein are used for descriptive purposes only and are not to be construed as either explicit or implicit relative importance or to indicate the number of technical features indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include one or more features, and in the description of embodiments of the application, unless otherwise indicated, the meaning of "a plurality" is two or more.

The data volume of video content is enormous, such as 1920x1080 resolution, 30 frames per second of video, and up to 178 megabytes per second of data. In order to reduce the network bandwidth occupation of transmitting video and the storage space occupation of storing video, various video coding compression techniques have been developed to compress the volume of video files to 1/1000 or more of the original volume.

With the evolution of video coding technology, current coding standards have increasingly high demands on computational power, including video coding computational power and video decoding computational power. For the computational problem of video decoding, hardware decoding is widely adopted on terminal equipment to solve the computational problem. However, the video hardware decoding is updated along with the hardware updating in the terminal equipment, that is to say, the video decoding is updated only after the hardware updating in the terminal equipment, but the hardware is fixed when the terminal equipment leaves the factory, so that the iteration speed of the video decoding is slower; and hardware diversification fragmentation causes compatibility problems between video decoding and hardware devices. Therefore, processor software decoding is an essential complement to hardware decoding and is widely used.

Processor software decoding is mainly affected by the main frequency of a processor, and as the main frequency of the processor is more and more difficult to increase, a processor system is evolved towards multiple cores. Firstly, a symmetrical processor system comprising multi-core processor cores is proposed, and in order to fully utilize multi-core capability and solve the problem of high computation power of video decoding, a video parallel decoding method is further proposed; the video parallel decoding method comprises the following steps: frame-level multithreading parallel decoding and intra-frame multithreading parallel decoding; wherein intra-multithreaded parallel decoding includes, but is not limited to: multiple slice (slice) parallel decoding, multiple tile (tile) parallel decoding, wavefront (wave front) parallel decoding.

Meanwhile, for the purpose of performance balancing and power consumption issues, an asymmetric processor system including a multi-core processor core is further proposed. Specifically, a System On Chip (SOC) integrates a plurality of low-power-consumption low-performance small cores, and one or more high-power-consumption high-performance large cores.

Furthermore, the video parallel decoding method performed on the symmetric processor system is applied to the asymmetric processor system. However, video parallel decoding works well on symmetric processor systems, is inefficient on asymmetric processors, and even has situations where parallel decoding speeds are slower than single-threaded speeds.

Referring now to fig. 1, a schematic diagram of frame-level multithreading parallel decoding in an asymmetric processor system in the related art is presented using a processor core as a central processing unit (Central Processing Unit, CPU) core; an asymmetric CPU system comprising 4 CPU cores is exemplarily presented in fig. 1, wherein CPU core 1 and CPU core 2 are low-power-consumption low-performance small cores, and CPU core 3 and CPU core 4 are high-power-consumption high-performance large cores; at this time, 4 CPU cores decode 4 video frames simultaneously, each CPU core decodes one video frame, wherein CPU core 1 decodes video frame 1, CPU core 2 decodes video frame 2, CPU core 3 decodes video frame 3, and CPU core 4 decodes video frame 4.

In the related art, in order to achieve a high compression ratio in a video encoding process, a video compression algorithm generally adopts motion compensation to reduce temporal redundancy, and when the motion compensation algorithm causes video frame decoding, a dependency relationship exists between video frames. For example, in fig. 1, video frame 2 depends on video frame 1, video frame 3 depends on video frame 1 and video frame 2, and video frame 4 depends on video frame 1 and video frame 2.

When a dependency relationship exists between video frames, video frame decoding is limited by the dependency relationship, as in fig. 1, the decoding progress of video frame 2 is limited by the progress of video frame 1, when video frame 1 is decoded to a certain position, video frame 2 can continue to go downwards, and the progress of video frame 2 is followed by video frame 1; similarly, the decoding progress of video frame 3 and video frame 4 is limited by the progress of video frame 1 and video frame 2. At this time, because of the difference in the sizes of the CPU cores, the decoding speed of the CPU core 3 and the CPU core 4 is faster than that of the CPU core 1 and the CPU core 2, but because of the limitation of motion compensation, the CPU core 3 and the CPU core 4 wait for the CPU core 1 and the CPU core 2 to decode the first two video frames before the decoding operation, and the CPU core 3 and the CPU core 4 are in an idle state for a long time. Thus reducing the efficiency of use of the processor cores in the asymmetric processor system and reducing the speed at which the asymmetric processor system decodes the video.

Therefore, in the video parallel decoding mode in the related art, in the asymmetric multi-core processor system, the video decoding efficiency and the processor utilization rate are not high. And because of the randomness of the scheduling, the problem of unstable decoding speed exists, and the video playing is not smooth due to the unstable decoding speed, so that the video playing experience is reduced.

In view of this, embodiments of the present application provide a video decoding method, apparatus, device, and storage medium, which are applied to an asymmetric processor system including a plurality of processor cores, and are used for providing a video parallel decoding method for an asymmetric processor system including a plurality of processor cores, so as to improve the use efficiency of the processor cores, ensure the stability of the video decoding speed, and improve the video playing experience.

In the embodiment of the application, the difference of the computing power of a plurality of processor cores in an asymmetric processor system and the decoding sequence of video frames to be decoded are considered; in the embodiment of the application, firstly, based on the computing capacity of each of a plurality of processor cores, the processor cores with the computing capacity in the same computing capacity interval are assigned to the same processor resource group, so as to obtain at least one processor resource group; then, according to the respective decoding order information of each video frame to be decoded and the respective calculation power interval of at least one processor resource group in the video frames to be decoded, the respective processor resource group is allocated to each video frame to be decoded, the decoding order information characterizes the decoding order and the dependence information of the respective video frames to be decoded, and in the video decoding process, the processor resource group is allocated to the video frames to be decoded according to the decoding order and the dependence relation of the video frames to be decoded and the calculation power interval, instead of randomly selecting a processor core to perform video frame decoding; and finally, at least one processor resource group is called to respectively decode the corresponding video frames to be decoded.

The application scenario set up by the present application will be briefly described below. It should be noted that the following scenario is only for illustrating the embodiments of the present application, and is not limiting. In the specific implementation, the technical scheme provided by the embodiment of the application can be flexibly applied according to actual needs.

Referring to fig. 2, fig. 2 is a schematic diagram of an application scenario provided in an embodiment of the present application. The application scenario includes a terminal device 210 and a server 220, where the terminal device 210 and the server 220 may communicate through a communication network.

In an alternative embodiment, the communication network may be a wired network or a wireless network. Accordingly, the terminal device 210 and the server 220 may be directly or indirectly connected through wired or wireless communication. For example, the terminal device 210 may be indirectly connected to the server 220 through a wireless access point, or the terminal device 210 may be directly connected to the server 220 through the internet, which is not limited herein.

The terminal device 210 includes, but is not limited to, a mobile phone, a tablet computer, a notebook computer, a desktop computer, an electronic book reader, an intelligent voice interaction device, an intelligent home appliance, a vehicle-mounted terminal, and the like; the terminal device can be provided with various clients, and the clients can be online platforms, application programs (such as a browser, game software, shopping software, a video player and the like) and web pages, applets and the like which support application video decoding functions such as video playing, video clipping and the like; video includes, but is not limited to: long video, short video, live. For example, a client instructs video playback through which video content can be viewed online. Before the video content is displayed in the client, firstly, decoding the acquired video content, decoding video frame images of the acquired video content, continuously displaying the images, and finally, playing the video.

The server 220 is a backend server corresponding to a client installed in the terminal apparatus 210. The server 220 may be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN), basic cloud computing services such as big data and artificial intelligence platforms, and the like.

In one possible implementation scenario, the terminal device 210 communicates with the server 220, acquires the video compression content from the server 220, and decodes the video compression content by using the video decoding method provided by the embodiment of the present application after the terminal device 210 acquires the video compression content. It should be noted that the terminal device 210 provided in the embodiment of the present application applies an asymmetric processor system.

It should be noted that, the number of the terminal devices 210 and the servers 220 shown in fig. 2 is merely illustrative, and the number is not limited in practice, and is not particularly limited in the embodiment of the present application. In the embodiment of the present application, when the number of servers 220 is plural, plural servers 220 may be formed into a blockchain, and the servers 220 are nodes on the blockchain.

In order to further explain the technical solution provided by the embodiments of the present application, a terminal device is taken as an example to perform video decoding, and the video decoding method provided by the exemplary embodiment of the present application is described below with reference to the accompanying drawings. It should be noted that the above application scenario is only shown for the convenience of understanding the spirit and principle of the present application, and the embodiments of the present application are not limited in any way. Furthermore, although the operations of the methods of the present application are depicted in the drawings in a particular order, this is not required to or suggested that these operations must be performed based on the particular order or that all of the illustrated operations must be performed in order to achieve desirable results. Additionally or alternatively, certain steps may be omitted, multiple steps combined into one step to perform, and/or one step decomposed into multiple steps to perform.

Referring to fig. 3, fig. 3 is a flowchart of a video decoding method according to an embodiment of the present application, including the following steps:

step S300, based on the computing power of each processor core, grouping the processor cores to obtain at least one processor resource group; wherein the computing power of the processor cores belonging to the same processor resource group is located in the same computing power interval.

Since the computing power of the multiple processor cores in the asymmetric processor system is not considered when the video is decoded in parallel in the related art, a random allocation manner is still adopted, which results in low use efficiency of the processor cores and low video decoding speed. Therefore, in the video decoding method provided by the application, the situation that the computing capacities of a plurality of processor cores in an asymmetric processor system are different is considered, and according to the computing capacities of the processor cores, grouping processing is carried out on each processor core, and the processor cores with the computing capacities in the same computing power interval are divided into the same processor resource group to obtain at least one processor resource group.

The CPU resource group division is described by taking a processor core as an example. Referring to fig. 4, fig. 4 is a schematic diagram of processor resource group division according to an embodiment of the present application. The asymmetric processor system comprises 7 CPU cores, wherein each CPU core corresponds to one computing capacity; for example, the calculation capability of the CPU core 1 is 96, the calculation capability of the CPU core 2 is 95, the calculation capability of the CPU core 3 is 83, the calculation capability of the CPU core 4 is 72, the calculation capability of the CPU core 5 is 96, the calculation capability of the CPU core 6 is 82, the calculation capability of the CPU core 7 is 71, and the set calculation capability section includes: [100, 90), [90, 80), [80, 70), and so forth. At this time, since the computing capacities of the CPU core 1, the CPU core 2, and the CPU core 5 are located in the same computing power section [100,90 ], the CPU core 1, the CPU core 2, and the CPU core 5 are divided into the same CPU resource group; the computing power of the CPU core 3 and the CPU core 6 are located in the same computing power interval [90,80 ], so that the CPU core 3 and the CPU core 6 are divided into the same CPU resource group; the computing power of the CPU core 4 and the CPU core 7 are located in the same power interval 80,70, and thus the CPU core 4 and the CPU core 7 are divided into the same CPU resource group.

It should be noted that the allocation manner shown in fig. 4 is merely an exemplary illustration, and the calculation force interval may be set according to the actual situation in the actual application, and the calculation force interval may be set to a fixed value. When the calculation power interval is set to a fixed value, CPU cores having the same calculation power are divided into the same CPU resource group.

Step S301, according to the respective decoding order information of each video frame to be decoded in the video to be decoded and the respective calculation interval of at least one processor resource group, distributing a corresponding processor resource group for each video frame to be decoded; the decoding order information characterizes the decoding order of the corresponding video frame to be decoded and the dependency information.

In the embodiment of the application, each video frame to be decoded in the video to be decoded is provided with a decoding sequence information, and the decoding sequence information is generally represented by a decoding time stamp (Decoding Time Stamp, DTS); the decoding sequence information characterizes the decoding sequence of the video frames to be decoded and the dependence information; the decoding sequence of the video frames to be decoded is ordered in a monotonically increasing mode according to the decoding time stamp; the dependence information of the video frames to be decoded is that the video frames to be decoded with large decoding time stamps depend on the video frames to be decoded with small decoding time stamps, and the video frames to be decoded with small decoding time stamps do not depend on the video frames to be decoded with large decoding time stamps.

In consideration of the fact that decoding time can be reduced and decoding speed can be improved by adopting a motion compensation mode based on decoding sequence information in the video decoding process, and the computing capacity of a processor core also seriously affects the decoding speed of video decoding. In the embodiment of the application, a mode of distributing corresponding processor resources for a video frame to be decoded based on decoding sequence information and a calculation power interval of a processor resource group is provided.

Step S302, at least one processor resource group is called to decode the video frame to be decoded.

In the embodiment of the application, after the corresponding processor resource group is allocated to the video frame to be decoded, the processor core in at least one processor resource group is called to respectively decode the corresponding video frame to be decoded.

And simultaneously decoding at least one video frame to be decoded by the processor cores in the plurality of processor resource groups, so that parallel decoding of the video to be decoded is realized under an asymmetric processor system.

Next, referring to steps A1 to A3, the allocation of the processor resource group to the video frame to be decoded in step S301 will be described in detail.

Step A1, determining the number of decoded video frames at the current time based on the at least one set of processor resources.

In one possible implementation, the number of processor resource groups is taken as the number of decoded video frames at the current moment; for example, as shown in fig. 4 with 3 processor resource groups, the number of decoded video frames is 3.

In another possible implementation, the number of decoded video frames at the current time is determined based on the number of groups of processor resource groups and the number of cores of the processor cores in each processor resource group; for example, as shown in fig. 4, there are 3 processor resource groups, where one processor resource group has 3 processor cores, and the other two processor resource groups have 2 processor cores, where the number of decoded video frames at the current time is determined to be 7; that is, the number of processor cores in the asymmetric processor system is taken as the number of decoded video frames at the current time.

And step A2, selecting at least one target video frame from the video frames to be decoded according to the first sequence of the decoding time stamps and the number of the decoded video frames.

In the embodiment of the application, the first sequence characterizes monotonically increasing decoding time stamps; i.e. at least one target video frame is selected from the individual video frames to be decoded in such a way that the decoding time stamps associated with the video frames to be decoded monotonically increase, and the number of decoded video frames.

Illustratively, firstly, sorting video frames to be decoded according to a mode that decoding time stamps associated with the video frames to be decoded monotonically increase to obtain a video frame sequence to be decoded; then, selecting at least one target video frame from the video frames to be decoded according to the video frame sequence to be decoded and the number of the decoded video frames.

It should be noted that, the decoding time stamp only determines the decoding order of the video frames to be decoded, but is not used for determining the playing order of the video frames to be decoded, so that the positions of the video frames in the video frame sequence to be decoded are not completely consistent with the positions of the video frames in the playing order; referring to fig. 5, fig. 5 is a schematic diagram of a video frame ordering to be decoded according to an embodiment of the present application, and as can be seen from fig. 5, a decoding time of a partial priority playing video frame is located after a post playing video frame.

In the embodiment of the application, when at least one target video frame is selected from all the video frames to be decoded according to the video frame sequence to be decoded and the number of the video frames to be decoded, whether the number of the video frames to be decoded in the video frame sequence to be decoded is larger than or equal to the number of the video frames to be decoded is firstly determined, if so, the target video frames consistent with the number of the video frames to be decoded are acquired according to the video frame sequence to be decoded, and otherwise, the target video frames less than the number of the video frames to be decoded are acquired.

And step A3, allocating corresponding processor resource groups for each target video frame in the at least one target video frame based on the respective calculation power intervals of the at least one processor resource group.

In the embodiment of the application, the number of the selected target video frames influences the mode of allocating the resource group for the target video frames. The following is a specific description of the specific case.

Case one: a target video frame is selected that corresponds to the number of groups of the processor resource groups.

When the number of the decoded video frames is the number of the processor resource groups and the target video frames consistent with the number of the decoded video frames are selected; that is, a target video frame is selected that corresponds to the number of groups of the processor resource groups.

When selecting target video frames consistent with the number of groups of the processor resource groups, each target video frame is allocated a processor resource group.

Illustratively, the target video frames are ordered according to a mode that decoding time stamps associated with the target video frames monotonically increase, so as to obtain a first video frame sequence; sequencing the processor resource groups according to a monotonically decreasing manner of the calculation intervals of the processor resource groups to obtain a processor resource group sequence; then, each target video frame in the first video frame sequence is respectively allocated to one processor resource group matched with the target video frame position in the processor resource group sequence.

Referring to fig. 6, fig. 6 is a schematic diagram of a first video frame allocation according to an embodiment of the present application; as can be seen from fig. 6:

based on the schematic diagrams shown in fig. 4 and fig. 5, it is determined that there are 3 CPU resource groups based on fig. 4, a decoding order of video frames to be decoded is determined based on fig. 5, at this time, 3 video frames to be decoded are obtained as target video frames, which are respectively a video frame 1 to be decoded, a video frame 3 to be decoded and a video frame 2 to be decoded, and decoding time stamps of the 3 target video frames are determined, at this time, it is determined that the first video frame sequence is: video frame 1 to be decoded, video frame 3 to be decoded, video frame 2 to be decoded;

the CPU resource group sequences obtained by sequencing the CPU resource groups according to the calculation intervals are as follows: a CPU resource group 1 including a CPU core 1, a CPU core 2 and a CPU core 5, a CPU resource group 2 including a CPU core 3 and a CPU core 6, and a CPU resource group 3 including a CPU core 4 and a CPU core 7;

then, the video frame 1 to be decoded is allocated to the CPU resource group 1, the video frame 3 to be decoded is allocated to the CPU resource group 2, and the video frame 2 to be decoded is allocated to the CPU resource group 3.

Considering that each processor resource group contains at least one processor core, if a video frame to be decoded is allocated to one processor core in the processor resource group for video decoding processing, other processor cores in the processor resource group are in an idle state, which further results in resource waste. Therefore, when one video frame to be decoded is allocated to one processor resource group and the processor resource group includes at least two processor cores, in order to reduce resource waste and improve video decoding speed, in the embodiment of the present application, an implementation manner of allocating one video frame to be decoded to all the processor cores in the processor resource group for decoding is proposed.

Illustratively, when a video frame to be decoded is assigned to all processor cores in a processor group for decoding processing: firstly, dividing a video frame to be decoded into a plurality of pixel areas; then, different pixel areas are allocated to different processor cores for decoding processing. Referring to fig. 7, fig. 7 is a schematic diagram of a second video frame allocation according to an embodiment of the present application.

It should be noted that, when a plurality of processors in a processor resource group check a video frame to be decoded to decode, the decoding manner may refer to an implementation manner of decoding a video frame to be decoded by a plurality of processor resource groups, and the principle is similar, and details are not described herein.

And a second case: a target video frame is selected that corresponds to the number of cores of the processor core.

When the number of decoded video frames is determined based on the number of groups of processor resource groups and the number of cores of the processor cores in each processor resource group, and a target video frame consistent with the number of decoded video frames is selected; that is, a target video frame is selected that corresponds to the number of cores of the processor core.

When selecting target video frames which are consistent with the number of cores of the processor cores, one processor core is allocated to each target video frame.

Illustratively, the target video frames are ordered according to a mode that decoding time stamps associated with the target video frames monotonically increase, so as to obtain a first video frame sequence; according to the monotonically decreasing mode of the computing power interval of the processor resource group and the number of cores of the processor cores in the processor resource group, acquiring target video frames consistent with the number of cores from the first video frame sequence in sequence; and correspondingly distributing one target video frame in the acquired target video frames consistent with the number of cores to one processor core in the matched processor resource group.

Referring to fig. 8, fig. 8 is a schematic diagram of third video frame allocation according to an embodiment of the present application; as can be seen from fig. 8:

based on the schematic diagrams shown in fig. 4 and fig. 5, it is determined that 7 CPU cores exist based on fig. 4, a decoding order of video frames to be decoded is determined based on fig. 5, 7 video frames to be decoded are acquired as target video frames at this time, namely, video frame 1 to be decoded, video frame 3 to be decoded, video frame 2 to be decoded, video frame 4 to be decoded, video frame 6 to be decoded, video frame 5 to be decoded, and video frame 7 to be decoded, and decoding time stamps of the respective 7 target video frames are determined, and at this time, it is determined that the first video frame sequence is: video frame to be decoded 1, video frame to be decoded 3, video frame to be decoded 2, video frame to be decoded 4, video frame to be decoded 6, video frame to be decoded 5, and video frame to be decoded 7;

then, based on 3 CPU cores in the CPU resource group 1, obtaining a video frame 1 to be decoded, a video frame 3 to be decoded and a video frame 2 to be decoded from a first video frame sequence; based on 2 CPU cores in the CPU resource group 2, obtaining a video frame 4 to be decoded and a video frame 6 to be decoded from a first video frame sequence; based on 2 CPU cores in the CPU resource group 3, obtaining a video frame 5 to be decoded and a video frame 7 to be decoded from a first video frame sequence;

and finally, distributing the acquired video frames to be decoded to different CPU cores in the matched CPU resource group.

In the embodiment of the application, when the obtained target video frame is distributed to different processor cores in the matched processor resource group, the computing capacity of the processor cores can be considered, namely, the video frame to be decoded with small decoding time stamp is distributed to the processor cores with large computing capacity; taking allocation in the CPU resource group 1 as an example, the video frame 1 to be decoded is allocated to the CPU core 1, the video frame 3 to be decoded is allocated to the CPU core 5, and the video frame 2 to be decoded is allocated to the CPU core 2; similarly, the video frame 4 to be decoded is distributed to the CPU core 3, and the video frame 6 to be decoded is distributed to the CPU core 6; the video frames 5 to be decoded are allocated to the CPU core 4, and the video frames 7 to be decoded are allocated to the CPU core 7.

And a third case: a target video frame is selected.

When the number of video frames to be decoded in the video frame sequence to be decoded is smaller than the number of video frames to be decoded, acquiring target video frames which are smaller than the number of video frames to be decoded from the video frame sequence to be decoded; at this time, a case will occur in which the same target video frame is allocated to at least two processor resource groups, or the same target video frame is allocated to at least two processor cores.

Thus, taking as an example the selection of a target video frame, the allocation of a set of processor resources to the target video frame will be described.

In the embodiment of the application, after a target video frame is acquired, the target video frame is first divided into a plurality of pixel areas. Illustratively, the target video frame is divided into a plurality of pixel regions according to a video encoding format; for example: dividing a target video frame into smaller slices, referring to fig. 9, fig. 9 is a schematic diagram of dividing pixel areas of a first video frame according to an embodiment of the present application; referring to fig. 10, fig. 10 is a schematic diagram showing a second video frame dividing pixel area according to an embodiment of the present application; referring to fig. 11, fig. 11 is a schematic diagram of dividing a target video frame into pixel regions according to a third embodiment of the present application. In another example, each object included in the target video frame is identified, and the target video frame is divided into a plurality of pixel areas according to an area where the object is located; for example, the target video frame includes birds, flowers, grasslands, sky, etc., the identified birds are in the target video frame as a pixel area, the identified flowers are in the target video frame as a pixel area, the identified grasslands are in the target video frame as a pixel area, the identified sky is in the target video frame as a pixel area, etc.

After a target video frame is divided into a plurality of pixel areas, carrying out decoding priority sequencing on the plurality of pixel areas to obtain an area decoding sequence; arranging at least one processor resource group according to a monotonically decreasing manner of the computing power interval of the processor resource group to obtain a processor resource group sequence; and then, based on the region decoding sequence and the processor resource group sequence, respectively distributing corresponding processor resource groups for each pixel region in the region decoding sequence.

In one possible implementation, the multiple pixel regions may be prioritized for decoding based on dependencies between the pixel regions to obtain a region decoding sequence.

Illustratively, a pixel region is allocated to one processor core in a set of processor resources; referring to fig. 12, fig. 12 is a schematic diagram of a fourth video frame allocation according to an embodiment of the present application, and as can be seen from fig. 12:

based on the schematic diagram shown in fig. 4, fig. 4 includes 7 CPU cores, where the target video frame is divided into 7 pixel areas, and the dependency relationship between the 7 pixel areas is determined as follows: pixel region 2 depends on pixel region 1, pixel region 3 depends on pixel region 2 … …, and pixel region 7 depends on pixel region 6; thus, the region decoding sequence is: pixel region 1, pixel region 2 … …, pixel region 7;

Sequencing the CPU resource groups according to the calculation intervals, wherein the obtained CPU resource group sequences are as follows: a CPU resource group 1 including a CPU core 1, a CPU core 2 and a CPU core 5, a CPU resource group 2 including a CPU core 3 and a CPU core 6, and a CPU resource group 3 including a CPU core 4 and a CPU core 7;

then, based on 3 CPU cores in the CPU resource group 1, acquiring a pixel area 1, a pixel area 2 and a pixel area 3 from the area decoding sequence; acquiring a pixel region 4 and a pixel region 5 from the region decoding sequence based on 2 CPU cores in the CPU resource group 2; acquiring a pixel region 6 and a pixel region 7 from the region decoding sequence based on 2 CPU cores in the CPU resource group 3;

and finally, distributing the acquired pixel areas to different CPU cores in the matched CPU resource group, wherein one CPU core processes one pixel area.

In the embodiment of the application, when the acquired pixel areas are distributed to different processor cores in the matched processor resource group, the computing capacity of the processor cores can be considered, and the pixel areas which are preferentially decoded can be distributed to the processor cores with large computing capacity; for example: continuing with fig. 4, at this time, pixel region 1 is allocated to CPU core 1, pixel region 2 is allocated to CPU core 5, pixel region 3 is allocated to CPU core 2, pixel region 4 is allocated to CPU core 3, pixel region 5 is allocated to CPU core 6, pixel region 6 is allocated to CPU core 4, and pixel region 7 is allocated to CPU core 7.

It should be noted that, in the embodiment of the present application, the number of pixel areas obtained by dividing the target video frame may also be greater than the number of processor cores in the asymmetric processor system, where whether a dependency relationship exists between the pixel areas is identified, when the dependency relationship is determined, a pixel area consistent with the number of processor cores is obtained from the pixel areas based on the dependency relationship, and is assigned to one processor core in the above manner, and when it is determined that the dependency relationship does not exist, a pixel area consistent with the number of processor cores is randomly obtained from the pixel areas, and is assigned to one processor core in the above manner.

In one possible implementation, one pixel region may also be allocated to one processor resource group.

In another possible implementation, after decoding of a pixel region is completed by a processor resource group or a processor core, the processor resource is reassigned to the remaining pixel regions that are not decoded; the redistribution method refers to an implementation manner of redistributing processor resources for the video frame to be encoded, and the principle is similar, and will not be described herein.

It should be noted that, the above-mentioned manner of selecting one target video frame and performing CPU resource allocation on the target video frame is only one implementation manner, and the same applies to the manner of selecting two target video frames. For example, when two target video frames are selected based on the asymmetric CPU system shown in fig. 4, the target video frames to be preferentially decoded may be allocated to the CPU resource group 1 and the CPU resource group 2, and the target video frames to be subsequently decoded may be allocated to the CPU resource group 3, and at this time, pixel regions are divided for the target video frames to be preferentially decoded, and CPU cores are allocated for the pixel regions. Therefore, the technical scheme of the application can be infinitely extended according to actual conditions, and is not repeated here.

In the embodiment of the present application, in order to increase the speed of video decoding, an implementation manner is proposed that the video to be decoded is redistributed to the processor resource group or the processor core according to the decoding progress of the processor resource group or the processor core. Therefore, before each video frame to be decoded is respectively allocated with a corresponding processor resource group based on the respective decoding order information of each video frame to be decoded in the video to be decoded and the respective calculation power interval of at least one processor resource group, judging whether the decoded video frame associated with the video to be decoded exists at the current moment, if so, reallocating the corresponding processor resource group for the video frame to be decoded according to the decoding order information, the calculation power interval and the decoding progress of each processor resource group at the last moment, otherwise, allocating the corresponding processor resource group for the video frame to be decoded according to the decoding order information and the calculation power interval.

Referring to fig. 13, fig. 13 is a flowchart of another video decoding method according to an embodiment of the present application, including the following steps:

step S1300, obtaining respective decoding order information of each video frame to be decoded in the video frames to be decoded.

It should be noted that, the decoding order information is self-contained in the video frame to be decoded, and is generally indicated by a decoding timestamp.

Step S1301, performing packet processing on each processor core based on the computing capability of each of the plurality of processor cores, to obtain at least one processor resource group; wherein the computing power of the processor cores belonging to the same processor resource group is located in the same computing power interval.

Considering that the computing power of not every processor core is different in an asymmetric processor system, often several high-power high-performance large cores have the same or similar computing power and several low-power low-performance small cores have the same or similar computing power; thus, in embodiments of the present application, processor cores are grouped into groups according to computing power, with the computing power of the processor cores within each group being the same or similar. Specifically, refer to the relevant content of step S300, which is not described herein.

Step S1302, determining whether there is a decoded video frame associated with the video to be decoded at the current time, if yes, executing step S1303, otherwise executing step S1304.

Step S1303 reallocates the corresponding processor resource groups for the video frame to be decoded based on the decoding sequence information, the calculation interval, and the decoding progress of each processor resource group at the previous time.

In the embodiment of the application, when the decoded video frame associated with the video to be decoded exists at the current moment, a corresponding processor resource group is allocated to the video frame to be decoded in a mode of step B1 to step B4.

And step B1, determining the number of decoded video frames at the current moment based on at least one processor resource group.

Referring specifically to the step A1, the detailed description is not repeated here.

Step B2, selecting at least one target video frame from the video frames to be decoded based on the first sequence of the decoding time stamps and the number of the decoded video frames; wherein the first order characterizes a monotonically increasing decoding timestamp.

Referring specifically to the step A2, the detailed description is not repeated here.

And B3, determining a processor resource group associated with each target video frame in at least one target video frame at the last moment, and acquiring an association result.

Illustratively, the number of decoded video frames is determined based on the number of groups of processor resource groups and the number of cores of the processor cores in each of the processor resource groups, and a target video frame that corresponds to the number of decoded video frames is selected for illustration.

Based on the schematic diagram shown in fig. 8, when the decoding of the video frame 1 to be decoded is completed by the CPU core 1, but none of the video frames 2 to 7 to be decoded is completed, the current time continues to select the video frame to be decoded currently from the video frames to be decoded as: and determining whether the currently selected video frames 2-8 to be decoded contain the video frames decoded at the previous moment, determining a CPU resource group associated with the video frames decoded at the previous moment and a CPU core in the CPU resource group, and acquiring a corresponding association result.

Thus, the correlation results are: at the last moment, the video frame to be decoded 2 and the video frame to be decoded 3 are respectively associated with the CPU core 2 and the CPU core 5 in the CPU resource group 1, the video frame to be decoded 4 and the video frame to be decoded 4 are respectively associated with the CPU core 3 and the CPU core 6 in the CPU resource group 2, and the video frame to be decoded 5 and the video frame to be decoded 7 are respectively associated with the CPU core 4 and the CPU core 7 in the CPU resource group 3.

And B4, respectively distributing corresponding processor resource groups for each target video frame in the at least one target video frame based on the respective calculation power interval of the at least one processor resource group and the association result.

In the embodiment of the application, when one processor resource group exists in the association result to associate at least two target video frames, the processor resource group is allocated to the video frames to be encoded in the following manner.

Illustratively, the target video frames are ordered according to a mode that decoding time stamps associated with the target video frames monotonically increase, so as to obtain a first video frame sequence; according to the monotonically decreasing mode of the computing power interval of the processor resource group and the number of cores of the processor cores in the processor resource group, acquiring target video frames consistent with the number of cores from the first video frame sequence in sequence; and finally, respectively distributing each target video frame in the acquired target video frames to one processor core in the processor resource group matched with the target video frame according to the association result.

When each obtained target video frame in the target video frames is respectively distributed to one processor core in the processor resource group matched with the target video frame according to the association result, firstly, determining whether the target video frame is associated with one processor core in the processor resource group matched with the target video frame at the previous moment according to the association result, and if the association is determined, directly distributing the target video frame to the processor core associated with the previous moment; otherwise, distributing the target video frame to the appointed processor core; the processor cores are designated as: and in the matched processor resource group, processor cores which are not associated with all the acquired target video frames at the last moment are not associated with each other.

Referring to fig. 14, fig. 14 is a schematic diagram of a fifth video frame allocation according to an embodiment of the present application. As can be seen from fig. 14, after the video frame 1 is decoded, the to-be-decoded video frame needs to be redistributed to the CPU core 1, at this time, 7 target video frames are obtained, which are respectively the target video frames 2 to 8, and the first video frame sequence is determined according to the decoding timestamps associated with the target video frames: target video frame 3, target video frame 2, target video frame 4, target video frame 6, target video frame 5, target video frame 8;

then, based on 3 CPU cores in the CPU resource group 1, acquiring a target video frame 3, a target video frame 2 and a target video frame 4 from the first video frame sequence; based on 2 CPU cores in the CPU resource group 2, acquiring a target video frame 6 and a target video frame 5 from the first video frame sequence; based on 2 CPU cores in the CPU resource group 3, acquiring a target video frame 7 and a target video frame 8 from the first video frame sequence;

And finally, distributing the obtained target video frames to different CPU cores in the matched CPU resource group according to the association result. Since the target video frame 2 and the target video frame 3 are associated with the CPU core 2 and the CPU core 5 in the CPU resource group 1 at the previous time, the target video frame 2 is continuously allocated to the CPU core 2, the target video frame 3 is continuously allocated to the CPU core 5, and then the target video frame 4 is allocated to the CPU core 1; similarly, the target video frame 5 is allocated to the CPU core 3, and the target video frame 6 is allocated to the CPU core 6; the target video frame 8 is allocated to the CPU core 4, and the target video frame 7 is allocated to the CPU core 7.

For this purpose, the CPU cores of the target video frame 2, the target video frame 3, the target video frame 6, and the target video frame 7 are unchanged.

In the application, after one frame is decoded, the video frame to be decoded is redistributed according to the rule of reducing the extra consumption, so that the context switching overhead caused by total redistribution can be avoided.

In step S1304, a processor resource group is allocated to each video frame to be decoded based on the decoding order information and the respective computation intervals of the at least one processor resource group.

Here, the implementation manner of the step S1304 is identical to the implementation manner of the step S301, specifically refer to the relevant content of the step S301, and the detailed description is not repeated here.

In the application, in the process of the allocation of the processor resources of video decoding, the decoding sequence, the dependence information and the computing power of the processor cores of the video frames to be decoded are considered, the processor resources with high computing power can be allocated to the video frames to be decoded with high priority, the situation that the processor cores with high computing power are allocated to the video frames with low priority for decoding and the processor cores with low computing power are allocated to the video frames with high priority for decoding caused by random allocation is avoided, and the problems of low use efficiency of the processor cores with high computing power and low video decoding speed caused by long-time idle state of the processor cores with high computing power are further avoided; therefore, in the asymmetric processor system comprising a plurality of processor cores, the embodiment of the application realizes parallel decoding of the video, improves the service efficiency of the processor cores and the video decoding speed when decoding the video frames to be decoded, ensures the stability of the video decoding speed, avoids the problems of unsmooth playing, even clamping and the like caused by unstable decoding speed and untimely decoding, and improves the video playing experience.

Based on the same inventive concept, an embodiment of the present application provides a video decoding apparatus applied to an asymmetric processor system including a plurality of processor cores, referring to fig. 15, fig. 15 is a video decoding apparatus 1500 provided in an embodiment of the present application, the apparatus includes:

A grouping unit 1501, configured to perform grouping processing on each processor core based on respective computing capabilities of the plurality of processor cores, to obtain at least one processor resource group; the computing power of the processor cores belonging to the same processor resource group is located in the same computing power interval;

an allocation unit 1502, configured to allocate a corresponding processor resource group to each video frame to be decoded according to respective decoding order information of each video frame to be decoded in the video to be decoded and respective computation intervals of at least one processor resource group; the decoding sequence information characterizes the decoding sequence and the dependence information of the corresponding video frames to be decoded;

the decoding unit 1503 is configured to invoke at least one processor resource group to decode the video frame to be decoded.

In one possible implementation, the allocation unit 1502 is specifically configured to:

In one possible implementation, if the number of decoded video frames is the number of groups of the processor resource group, and selecting a target video frame that is consistent with the number of decoded video frames; the allocation unit 1502 is specifically configured to:

In one possible implementation, if the number of decoded video frames is determined based on the number of groups of processor resource groups and the number of cores of the processor cores in each processor resource group, and selecting a target video frame that is consistent with the number of decoded video frames; the allocation unit 1502 is specifically configured to:

In one possible implementation, if a target video frame is selected; the allocation unit 1502 is specifically configured to:

In one possible implementation, if it is detected that there is a decoded video frame associated with the video to be decoded at the current time; the allocation unit 1502 is specifically configured to:

In one possible implementation, there is one processor resource group in the association result to associate at least two target video frames; the number of the decoded video frames is determined based on the number of groups of the processor resource groups and the number of cores of the processor cores in each processor resource group, and a target video frame consistent with the number of the decoded video frames is selected; the allocation unit 1502 is specifically configured to:

It should be noted that although several units (or modules) of the apparatus are mentioned in the above detailed description, this division is merely exemplary and not mandatory. Indeed, the features and functions of two or more units (or modules) described above may be embodied in one unit (or module) in accordance with embodiments of the application. Conversely, the features and functions of one unit (or module) described above may be further divided into a plurality of units (or modules) to be embodied. Of course, in implementing the present application, the functions of each unit (or module) may be implemented in the same piece or pieces of software or hardware.

Having described the video decoding method and apparatus of an exemplary embodiment of the present application, another exemplary embodiment of the present application computing device is described next.

Those skilled in the art will appreciate that the various aspects of the application may be implemented as a system, method, or program product. Accordingly, aspects of the application may be embodied in the following forms, namely: an entirely hardware embodiment, an entirely software embodiment (including firmware, micro-code, etc.) or an embodiment combining hardware and software aspects may be referred to herein as a "circuit," module "or" system.

In one possible implementation, a computing device provided by an embodiment of the present application may include at least a processor and a memory. Wherein the memory stores a computer program which, when executed by the processor, causes the processor to perform any of the steps of the video decoding method of various exemplary embodiments of the present application.

In one possible implementation, the computing device may be a terminal device, and the structure of the computing device may be as shown in fig. 16, including: communication component 1610, memory 1620, display unit 1630, camera 1640, sensor 1650, audio circuitry 1660, bluetooth module 1670, processor 1680, and the like.

The communication component 1610 is for communicating with a server. In some embodiments, a circuit wireless fidelity (Wireless Fidelity, wiFi) module may be included, the WiFi module belonging to a short-range wireless transmission technology, through which the computing device may help the user to send and receive information.

Memory 1620 may be used to store software programs and data. The processor 1680 performs various functions of the terminal device and data processing by executing software programs or data stored in the memory 1620. The memory 1620 may include high-speed random access memory, and may also include non-volatile memory, such as at least one type of disk storage device, flash memory device, or other volatile solid-state storage device. The memory 1620 stores an operating system that enables the terminal device to operate. The memory 1620 may store an operating system and various application programs, and may also store codes for executing the multimedia information recommendation method according to the embodiment of the present application.

The display unit 1630 may also be used to display information input by a user or information provided to the user and a graphical user interface (graphical user interface, GUI) of various menus of the terminal device. Specifically, the display unit 1630 may include a display screen 1632 disposed on the front surface of the terminal device. The display 1632 may be configured in the form of a liquid crystal display, light emitting diodes, or the like.

The display unit 1630 may also be used to receive input numeric or character information, generate signal inputs related to user settings and function control of the terminal device, and in particular, the display unit 1630 may include a touch screen 1631 disposed on the front of the terminal device, and may collect touch operations on or near the user, such as clicking buttons, dragging scroll boxes, and the like.

The touch screen 1631 may cover the display screen 1632, or the touch screen 1631 may be integrated with the display screen 1632 to implement input and output functions of the terminal device, and after integration, the touch screen may be abbreviated as touch screen. The display unit 1630 may display an application program, multimedia information played through the application program, and corresponding operation steps.

The camera 1640 may be used to capture still images. The camera 1640 may be one or a plurality of cameras. The object generates an optical image through the lens and projects the optical image onto the photosensitive element. The photosensitive element may be a charge coupled device (charge coupled device, CCD) or a Complementary Metal Oxide Semiconductor (CMOS) phototransistor. The photosensitive elements convert the optical signals to electrical signals, which are then passed to the processor 1680 for conversion to digital image signals.

The terminal device may further include at least one sensor 1650, such as an acceleration sensor 1651, a distance sensor 1652, a fingerprint sensor 1653, a temperature sensor 1654. The terminal device may also be configured with other sensors such as gyroscopes, barometers, hygrometers, thermometers, infrared sensors, light sensors, motion sensors, and the like.

Audio circuitry 1660, speakers 1661, and microphone 1662 may provide an audio interface between the user and the terminal device. The audio circuit 1660 may transmit the received electrical signal converted from audio data to the speaker 1661, and convert the electrical signal into an audio signal by the speaker 1661 to be output. The terminal device may also be configured with a volume button for adjusting the volume of the sound signal. On the other hand, the microphone 1662 converts the collected sound signals into electrical signals, which are received by the audio circuit 1660 and converted into audio data, which are output to the communication component 1610 for transmission to, for example, another terminal device, or to the memory 1620 for further processing.

The bluetooth module 1670 is used to exchange information with other bluetooth devices having bluetooth modules through bluetooth protocols. For example, the terminal device may establish a bluetooth connection with a wearable computing device (e.g., a smart watch) that also has a bluetooth module via bluetooth module 1670 for data interaction.

The processor 1680 is a control center of the terminal device, connects various parts of the entire terminal using various interfaces and lines, and performs various functions of the terminal device and processes data by running or executing software programs stored in the memory 1620 and calling data stored in the memory 1620. In some embodiments, the processor 1680 may include one or more processing units; the processor 1680 may also integrate an application processor that primarily handles operating systems, user interfaces, applications, etc., and a baseband processor that primarily handles wireless communications. It will be appreciated that the baseband processor described above may not be integrated into the processor 1680. The processor 1680 of the present application may run an operating system, an application, a user interface display, and a touch response, and a multimedia information recommendation method according to an embodiment of the present application. In addition, a processor 1680 is coupled to the display unit 1630.

In some possible embodiments, aspects of the video decoding method provided by the present application may also be implemented in the form of a computer program product comprising a computer program for causing a computing device to carry out the steps of the video decoding method according to the various exemplary embodiments of the application as described in the present specification when the program product is run on the computing device.

The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium would include the following: an electrical connection having one or more wires, a portable disk, a hard disk, random Access Memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

The program product of embodiments of the present application may employ a portable compact disc read only memory (CD-ROM) and comprise a computer program and may run on a computing device. However, the program product of the present application is not limited thereto, and in this document, a readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with a command execution system, apparatus, or device.

The readable signal medium may comprise a data signal propagated in baseband or as part of a carrier wave in which a readable computer program is embodied. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A readable signal medium may also be any readable medium that is not a readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with a command execution system, apparatus, or device.

A computer program embodied on a readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

Computer programs for performing the operations of the present application may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer program may execute entirely on the user's computing device, partly on the user's equipment, as a stand-alone software package, partly on the user's computing device, partly on a remote computing device, or entirely on the remote computing device or server. In the case of remote computing devices, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., connected via the Internet using an Internet service provider).

It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having a computer-usable computer program embodied therein.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program commands may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable device to produce a machine, such that the commands executed by the processor of the computer or other programmable device produce a means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program commands may also be stored in a computer readable memory that can direct a computer or other programmable apparatus to function in a particular manner, such that the commands stored in the computer readable memory produce an article of manufacture including command means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

While preferred embodiments of the present application have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. It is therefore intended that the following claims be interpreted as including the preferred embodiments and all such alterations and modifications as fall within the scope of the application.

It will be apparent to those skilled in the art that various modifications and variations can be made to the present application without departing from the spirit or scope of the application. Thus, it is intended that the present application also include such modifications and alterations insofar as they come within the scope of the appended claims or the equivalents thereof.

Claims

1. A video decoding method for application to an asymmetric processor system comprising a plurality of processor cores, the method comprising:

grouping the processor cores based on the computing power of each processor core to obtain at least one processor resource group; the computing power of the processor cores belonging to the same processor resource group is located in the same computing power interval;

distributing corresponding processor resource groups for each video frame to be decoded according to respective decoding sequence information of each video frame to be decoded in the video to be decoded and respective calculation power intervals of the at least one processor resource group; the decoding sequence information characterizes the decoding sequence of the video frames to be decoded and the dependence information;

and calling the at least one processor resource group to decode the video frame to be decoded.

2. The method of claim 1, wherein the decoding order information is a decoding time stamp; the allocating a corresponding processor resource group for each video frame to be decoded according to the respective decoding order information of each video frame to be decoded in the video to be decoded and the respective computation interval of the at least one processor resource group, includes:

selecting at least one target video frame from the video frames to be decoded according to the first sequence of the decoding time stamps and the number of the decoded video frames; wherein the first order characterizes the monotonically increasing decoding timestamps;

3. The method of claim 2, wherein if the number of decoded video frames is the number of groups of processor resources and selecting a target video frame that is consistent with the number of decoded video frames;

then assigning a respective set of processor resources to each of the at least one target video frame based on the respective computational power intervals of the at least one set of processor resources, comprising:

Arranging the target video frames based on the first sequence of the decoding time stamps to obtain a first video frame sequence;

arranging the at least one processor resource group based on a second order of computational intervals of the at least one processor resource group to obtain a processor resource group sequence; wherein the second order characterizes the monotonically decreasing calculation interval;

4. The method of claim 2, wherein if the number of decoded video frames is determined based on the number of groups of processor resources and the number of cores of the processor cores in each of the groups of processor resources, and selecting a target video frame that corresponds to the number of decoded video frames;

Based on a second sequence of the calculation intervals of the at least one processor resource group and the number of cores of the processor cores in the processor resource group, acquiring target video frames consistent with the number of cores from the first video frame sequence in sequence;

and correspondingly distributing one target video frame in the target video frames which are obtained and are consistent with the number of the cores to one processor core in the matched processor resource group.

5. The method of claim 2, wherein if a target video frame is selected;

dividing the target video frame into a plurality of pixel areas, and sorting decoding priorities of the pixel areas to obtain an area decoding sequence;

arranging the at least one processor resource group based on a second order of computational intervals of the at least one processor resource group to obtain a processor resource group sequence;

and allocating a corresponding processor resource group for each pixel region in the plurality of pixel regions based on the region decoding sequence and the processor resource group sequence.

6. The method according to any of claims 1-5, wherein if it is detected that the decoded video frame associated with the video to be decoded exists at the current time instant;

the allocating a corresponding processor resource group for each video frame to be decoded according to the respective decoding order information of each video frame to be decoded in the video to be decoded and the respective calculation power interval of the at least one processor resource group, includes:

selecting at least one target video frame from the respective video frames to be decoded based on the first order of the decoding time stamps and the number of decoded video frames;

determining a processor resource group associated with each target video frame in the at least one target video frame at the last moment, and acquiring an association result;

and allocating a corresponding processor resource group for each target video frame in the at least one target video frame based on the respective calculation power interval of the at least one processor resource group and the association result.

7. The method of claim 6, wherein there is one processor resource group associated with at least two target video frames in the association result; the number of the decoded video frames is determined based on the number of groups of the processor resource groups and the number of cores of the processor cores in each processor resource group, and a target video frame consistent with the number of the decoded video frames is selected;

The allocating a corresponding processor resource group for each target video frame in the at least one target video frame based on the respective computing power intervals of the at least one processor resource group and the association result includes:

based on a second order of computing power of the at least one processor resource group and the number of cores of the processor cores in the processor resource group, sequentially acquiring target video frames consistent with the number of cores from the first video frame sequence;

and respectively distributing each target video frame in the acquired target video frames to one processor core in a processor resource group matched with the target video frame according to the association result.

8. The method of claim 7, wherein assigning each of the acquired target video frames to one of the set of processor resources that match the target video frame, respectively, based on the correlation result, comprises:

if the target video frame is associated with one processor core in the matched processor resource group at the last moment, distributing the target video frame to the processor core associated at the last moment;

If the target video frame is not associated with one processor core in the matched processor resource group at the last moment, distributing the target video frame to the appointed processor core; the specified processor core is: and in the matched processor resource group, processor cores which are not associated with all the acquired target video frames at the last moment are not associated with each other.

9. A video decoding apparatus for application to an asymmetric processor system comprising a plurality of processor cores, the apparatus comprising:

a grouping unit, configured to perform grouping processing on the respective processor cores based on respective computing capabilities of the plurality of processor cores, to obtain at least one processor resource group; the computing power of the processor cores belonging to the same processor resource group is located in the same computing power interval;

the distribution unit is used for distributing corresponding processor resource groups for each video frame to be decoded according to respective decoding sequence information of each video frame to be decoded in the video to be decoded and respective calculation power intervals of the at least one processor resource group; the decoding sequence information characterizes the decoding sequence of the corresponding video frame to be decoded and the dependence information;

and the decoding unit is used for calling the at least one processor resource group and decoding the video frame to be decoded.

10. The apparatus according to claim 9, wherein the distribution unit is specifically configured to:

11. The apparatus according to claim 9 or 10, wherein if the presence of the decoded video frame associated with the video to be decoded at the current time instant is detected;

the dispensing unit has means for:

12. The apparatus of claim 11, wherein there is one processor resource group associated with at least two target video frames in the association result; the number of the decoded video frames is determined based on the number of groups of the processor resource groups and the number of cores of the processor cores in each processor resource group, and a target video frame consistent with the number of the decoded video frames is selected;

the dispensing unit has means for:

based on a second order of computing power of the at least one processor resource group and the number of cores of the processor cores in the processor resource group, sequentially acquiring target video frames consistent with the number of cores from the first video frame sequence; wherein the second order characterizes the monotonically decreasing calculation interval;

13. The apparatus of claim 12, wherein the distribution unit has means for:

14. A computing device, the computing device comprising: a processor and a memory, wherein:

the memory is used for storing a computer program;

the processor being adapted to execute the computer program for implementing the method of any one of claims 1 to 8.

15. A computer readable storage medium, characterized in that the computer readable storage medium stores a computer program which, when executed by a processor, implements the method of any one of claims 1-8.

16. A computer program product comprising a computer program which, when executed by a processor, implements the method of any one of claims 1 to 8.