CN116567319A

CN116567319A - Hardware decoding capability detection method and device, storage medium and computer equipment

Info

Publication number: CN116567319A
Application number: CN202210101675.XA
Authority: CN
Inventors: 杨小祥; 曹洪彬; 陈思佳; 黄永铖; 张佳; 曹健
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2022-01-27
Filing date: 2022-01-27
Publication date: 2023-08-08

Abstract

The application discloses a hardware decoding capability detection method, a device, a storage medium and computer equipment, which can be applied to various scenes such as games, game basic technologies, data processing and the like. The method comprises the following steps: detecting whether the terminal has hardware decoding capability according to the static capability and the native API capability of the multimedia programming interface; if the terminal has the hardware decoding capability, starting a hardware decoder to decode the target video; and if the terminal does not have the hardware decoding capability, starting a software decoder to decode the target video. According to the embodiment of the application, the hardware decoding capability can be accurately detected by detecting the static capability and the native API capability of the multimedia programming interface, and the coverage rate of the decoding capability of the detection terminal is improved.

Description

Hardware decoding capability detection method and device, storage medium and computer equipment

Technical Field

The present invention relates to the field of image processing technologies, and in particular, to a method and apparatus for detecting decoding capability of hardware, a storage medium, and a computer device.

Background

Along with the development of technology and the high-speed development of communication infrastructure, the requirements of people on video quality are higher and higher. Current video applications fall into three general categories: video call class, video player class, screen sharing class. Video telephony applications are directed to low latency scenes, with no special requirements for high resolution and high frame rate, video resolution is typically 480p-720p, and video frame rate is 30fps. Video player applications are directed to high resolution scenes, and video frame rates only reach 30fps, so long as the decoded output frame rate is guaranteed to be stable. Screen sharing related applications are directed to high resolution and low latency scenes where the decoded frame rate is typically required to be 15-30fps. For some high resolution and high frame rate application scenarios (such as cloud games), the requirements on the decoder are relatively high, the video resolution is generally above 720p, and the decoding frame rate is generally required to reach 60fps. In general, for some low resolution, low frame rate scenes, the use of a software decoder may suffice; but for high resolution and high frame rate usage scenarios, it is often necessary to enable a hardware decoder to meet the low latency and high fluency user experience.

Current video hardware decoding techniques have difficulty detecting hardware decoding capabilities.

Disclosure of Invention

The embodiment of the application provides a method, a device, a storage medium and computer equipment for detecting hardware decoding capability, which can accurately detect the hardware decoding capability.

In one aspect, a method for detecting hardware decoding capability is provided, the method comprising: detecting whether the terminal has hardware decoding capability according to the static capability of the multimedia programming interface and the API capability of the original application programming interface; if the terminal has the hardware decoding capability, starting a hardware decoder to decode the target video; and if the terminal does not have the hardware decoding capability, starting a software decoder to decode the target video.

In another aspect, there is provided a hardware decoding capability detection apparatus, the apparatus comprising: the detection unit is used for detecting whether the terminal has hardware decoding capability according to the static capability of the multimedia programming interface and the API capability of the original application programming interface; the first decoding unit is used for starting a hardware decoder to decode the target video if the terminal has hardware decoding capability; and the second decoding unit is used for starting the software decoder to decode the target video if the terminal does not have the hardware decoding capability.

In another aspect, a computer readable storage medium is provided, the computer readable storage medium storing a computer program adapted to be loaded by a processor for performing the steps in the hardware decoding capability detection method according to any of the embodiments above.

In another aspect, a computer device is provided, the computer device including a processor and a memory, the memory storing a computer program, the processor being configured to perform the steps in the hardware decoding capability detection method according to any of the embodiments above by calling the computer program stored in the memory.

The embodiment of the application detects whether the terminal has hardware decoding capability according to the static capability and the native API capability of the multimedia programming interface; if the terminal has the hardware decoding capability, starting a hardware decoder to decode the target video; and if the terminal does not have the hardware decoding capability, starting a software decoder to decode the target video. According to the embodiment of the application, the hardware decoding capability can be accurately detected by detecting the static capability and the native API capability of the multimedia programming interface, and the coverage rate of the decoding capability of the detection terminal is improved.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the description of the embodiments will be briefly introduced below, it being obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

Fig. 1 is a schematic view of an application scenario provided in an embodiment of the present application.

Fig. 2 is a schematic structural diagram of a hardware decoding capability detection system according to an embodiment of the present application.

Fig. 3 is a first flowchart of a method for detecting a hardware decoding capability according to an embodiment of the present application.

Fig. 4 is a second flowchart of a hardware decoding capability detection method according to an embodiment of the present application.

Fig. 5 is a first structural schematic diagram of a hardware decoding capability detection device according to an embodiment of the present application.

Fig. 6 is a second schematic structural diagram of a hardware decoding capability detection device according to an embodiment of the present application.

Fig. 7 is a schematic structural diagram of a computer device according to an embodiment of the present application.

Detailed Description

The following description of the embodiments of the present application will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are only some, but not all, of the embodiments of the present application. All other embodiments, which can be made by those skilled in the art based on the embodiments herein without making any inventive effort, are intended to be within the scope of the present application.

The embodiment of the application provides a hardware decoding capability detection method, a device, a storage medium and computer equipment. Specifically, the method for detecting the hardware decoding capability of the embodiment of the application may be executed by a computer device, where the computer device may be a device such as a terminal or a server. The terminal can be intelligent terminal equipment such as a smart phone, a tablet personal computer, a notebook computer, a desktop computer, an intelligent television, an intelligent sound box, a wearable intelligent device, an intelligent vehicle-mounted terminal, a palm game host and the like, and the terminal can also comprise a client, wherein the client can be a cloud game client, a client applet, a video client, a browser client or an instant messaging client and the like. The server may be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN), basic cloud computing services such as big data and artificial intelligent platforms, and the like.

The embodiment of the application can be applied to various scenes such as games, game basic technologies, data processing and the like.

First, partial terms or terminology appearing in the course of describing the embodiments of the present application are explained as follows:

artificial intelligence (Artificial Intelligence, AI) is the theory, method, technique and application system that uses a digital computer or a machine controlled by a digital computer to simulate, extend and extend human intelligence, sense the environment, acquire knowledge and use the knowledge to obtain optimal results. In other words, artificial intelligence is an integrated technology of computer science that attempts to understand the essence of intelligence and to produce a new intelligent machine that can react in a similar way to human intelligence. Artificial intelligence, i.e. research on design principles and implementation methods of various intelligent machines, enables the machines to have functions of sensing, reasoning and decision.

Cloud technology (Cloud technology) refers to a hosting technology for integrating hardware, software, network and other series resources in a wide area network or a local area network to realize calculation, storage, processing and sharing of data. The cloud technology is based on the general names of network technology, information technology, integration technology, management platform technology, application technology and the like applied by the cloud computing business mode, can form a resource pool, and is flexible and convenient as required. Cloud computing technology will become an important support. Background services of technical networking systems require a large amount of computing, storage resources, such as video websites, picture-like websites, and more portals. Along with the high development and application of the internet industry, each article possibly has an own identification mark in the future, the identification mark needs to be transmitted to a background system for logic processing, data with different levels can be processed separately, and various industry data needs strong system rear shield support and can be realized only through cloud computing.

Cloud gaming (Cloud gaming), which may also be referred to as game on demand, is an online gaming technology based on Cloud computing technology. Cloud gaming technology enables lightweight devices (thin clients) with relatively limited graphics processing and data computing capabilities to run high quality games. In a cloud game scenario, the game is not run in a player game terminal, but is run in a cloud server, and the game scenario is rendered into a video-audio stream by the cloud server and transmitted to the player game terminal through a network. The player game terminal does not need to have strong graphic operation and data processing capability, and only needs to have basic streaming media playing capability and the capability of acquiring player input instructions and sending the player input instructions to the cloud server.

Video coding: the manner in which a file in the original video format is converted into a file in another video format by compression techniques may be referred to as a bitstream.

Video decoding: reverse process of video coding.

FPS: the number of frames per second (Frames Per Second, FPS), which is a definition in the image arts, refers to the number of frames per second transmitted for a picture, and colloquially to the number of pictures for an animation or video. The FPS measures the amount of information used to save and display the dynamic video. The more frames per second, the smoother the displayed motion.

DirectX: microsoft multimedia programming interface (DX for short).

DXVA: microsoft DirectX video hardware acceleration (DirectX Video Acceleration).

MPEG: a motion picture expert group (Moving Picture Experts Group).

WMV: windows media video format (Windows Media Video).

VLD: variable word length decoding (Variable Length Decoding).

IDCT: inverse discrete cosine transform (Inverse Discrete Cosine Transform).

MoComp: motion compensation (Motion Compensation).

CPU: a central processing unit (Central Processing Unit).

GPU: a graphics processor (Graphics Processing Unit).

API: an application programming interface (Application Programming Interface).

QSV: quick Video synchronization technology (Quick Sync Video).

CUDA: unified computing device architecture (Compute Unified Device Architecture).

The cloud server is a server running a game in the cloud, and has functions such as video enhancement (pre-coding processing) and video coding.

The intelligent terminal is a device which has rich man-machine interaction modes, has the capability of accessing the Internet, is generally provided with various operating systems and has stronger processing capability. The intelligent terminal comprises a smart phone, a living room television, a tablet personal computer, a vehicle-mounted terminal, a palm game host and the like.

In order to enable the cloud server and the intelligent terminal to perform better cooperation and cooperation, joint optimization is achieved, video coding configuration can be reasonably allocated according to hardware capacity and real-time performance of the intelligent terminal in the cloud game process, computing tasks such as video image processing and video content analysis are reasonably allocated, and cloud game image quality experience is further improved under limited cloud server resources.

Video coding synergy: and selecting optimal coding and decoding configuration and coding and decoding strategies according to the coding and decoding capability of the intelligent terminal and combining the game type and the user network type.

Video rendering collaboration: and reasonably dividing video rendering tasks according to the graphic processing capability of the intelligent terminal, so that the cloud server and the intelligent terminal can effectively cooperate to improve video image quality. Including rendering region coordination, rendering task coordination, video analysis and processing coordination, and the like.

Terminal state cooperation: and according to the real-time performance of the operation of the intelligent terminal, the coding cooperative task and the rendering cooperative strategy of the cloud server and the intelligent terminal are dynamically adjusted, and the optimal experience of the user is ensured in real time.

The cloud server and intelligent terminal collaborative architecture mainly comprises: cloud server, end cloud cooperation strategy, end cloud cooperation protocol, intelligent terminal cooperation interface, software and hardware cooperation module, etc.

The end cloud cooperation strategy comprises the following steps: video coding collaborative policies, video rendering collaborative policies, terminal state collaborative policies. And the cloud server formulates an optimal video coding rendering collaborative strategy according to the equipment capability reported by the intelligent terminal and by combining the game type and the user network environment. Meanwhile, the cloud server acquires the operation performance of the intelligent terminal in real time through terminal state coordination, and dynamically adjusts the terminal cloud coordination strategy.

The cloud end cooperative protocol refers to a unified protocol for data interaction between a cloud server and an intelligent terminal.

The intelligent terminal cooperative interface is an intelligent terminal software and hardware module interface, and can effectively interact with the intelligent terminal through the interface, configure video coding and rendering parameters and acquire real-time operation performance of hardware.

The decoding protocols comprise video coding and decoding protocols such as H264, H265, AV1 and the like, and Profile and Level supported by the intelligent terminal under each coding and decoding protocol. Decoding performance refers to the highest decoding frame rate and single frame decoding delay supported for a given video size under a particular decoding protocol.

Wherein, the video size is defined as follows: 360p,576p,720p,1080p,2k,4k. The video frame rate is defined as follows: 30fps,40fps,50fps,60fps,90fps,120fps.

The decoding performance supported by the intelligent terminal is given in a triplet form, the first element is an enumeration definition of video resolution, the second element is an enumeration definition of video frame rate, the third element is a single frame decoding delay at video resolution and video frame rate, such as H264 decoding of device a, and the single frame decoding delay at 720p@60fps is 10ms.

The cloud server determines a coding function set to be started according to the game type and the network condition, and then determines the optimal coding configuration of the current equipment through the equipment type and the coding capability reported by the intelligent terminal.

The decoding capability data structure of the intelligent terminal is as shown in table 1:

table 1 intelligent terminal decoding capability data structure requirements

And the cloud server determines the optimal encoding and decoding configuration such as the decoding protocol, the decoding resolution, the video frame rate and the like of the current equipment, and encoding and decoding strategies such as the number of video encoding reference frames, SVC starting and the like according to the decoding capability of the intelligent terminal and by combining the game type and the user network condition.

A schematic diagram of a video rendering collaborative optimization connection flow as shown in fig. 1:

1. And the cloud server initiates a request to the intelligent terminal through the START client to acquire decoding capability information of the intelligent terminal. The request protocol field includes a protocol version number, a specific codec protocol query.

2. When receiving the capability information acquisition request, the intelligent terminal returns a state mark (success 0, failure is a specific error code), a supported protocol version number and terminal equipment capability information. If the intelligent terminal only supports part of decoding protocol, the supported decoding protocol information is returned; if the intelligent terminal does not support the decoding protocol, returning to the codes=0; and if the intelligent terminal capability information request fails, returning a specific error code.

3. After receiving the terminal decoding capability information, the cloud server combines the game type and the user network condition to determine the optimal encoding and decoding configuration such as the decoding protocol, the decoding resolution, the video frame rate and the like of the current terminal equipment, and encoding and decoding strategies such as the number of video encoding reference frames, SVC starting and the like.

4. And after receiving the optimal decoding configuration, the intelligent terminal decodes the video stream.

The START client is a cloud game client and is installed on the intelligent terminal.

The frequency coding collaborative optimization connection protocol is exemplified as follows:

1. Video decoding capability request:

2. video decoding capability response (support all query decoding protocols):

3. video decoding capability response (only partial query decoding protocol is supported):

4. video decoding capability request (no decoding protocol supported):

5. video decoding capability request (protocol request failure):

Video decoding can be generally divided into two types, one is software decoding with a CPU as a core, and the other is hardware accelerated decoding with a GPU as a core, which is currently mainstream. The advantage of the software decoding is that the third party software decoder is complete and has high universality, and almost all video decoding can be satisfied by using the CPU as a decoding core. However, on a platform with weaker CPU performance, especially when viewing high-rate high-definition video, the CPU is prone to problems such as video picture blocking, video cogging, and video slowness due to insufficient efficiency caused by the number of threads and self-architecture. The hardware accelerated decoding has the advantages that the mainstream video card can well support the mainstream video format decoding at present, and particularly, after the graphic core adopts a unified architecture, the parallel architecture design of the graphic core is utilized, so that the video decoding method has excellent supporting effect on high-definition videos in H.264 and VC-1 formats.

Almost all video cards on the market currently have a certain video hardware acceleration capability. Microsoft corporation has specifically formulated the DXVA specification for this purpose (including both DXVA1.0 and DXVA 2.0 versions). In the DXVA specification, different display cards have different video hardware acceleration capacities, and have more acceleration items supported by the display cards and have less support by the display cards. The DXVA1.0 specification originally defines the decoding capability of MPEG2 and WMV hardware, and the H.264 and VC-1 hardware decoding capability is added in the DXVA 2.0. According to the specifications of DXVA, hardware accelerated decoding can be divided into four levels, which are respectively from high to low: VLD (processing in terms of bitstream) > IDCT (intra-frame compression processing) > moscomp (inter-frame compression processing) > PostProc (post-display processing). As can be readily seen from the above distribution of grades, VLD accelerates most, so it contains IDCT, mocp, and PostProc; IDCT is accelerated, including MoComp and PostProc; finally the MoComp acceleration contains only PostProc.

Successful utilization of video hardware decoding capability requires coordination of several links. Firstly, the display card hardware needs to support a specific video coding format; secondly, the graphics driver needs this ability to report the graphics hardware correctly; finally, video decoders need to be able to enable this capability.

The prior technical proposal has the following disadvantages: 1. a particular video encoding format cannot be supported. DXVA is an interface specification that a decoder supports, and not representative of, all combinations of functions within the DXVA specification may or may not be utilized. 2. And failing to correctly inquire the hardware decoding capability supported by the display card of the terminal. Due to the display card driving version problem or the user using environment problem (such as dynamic library loss), the user fails to create the decoder, so that the decoding capability of the hardware supported by the original terminal display card cannot be correctly queried through the DXVA interface. 3. The validity verification of the decoding capability is lacking. Although a certain graphics card indicates that a certain capability is supported, if the support for this capability is not stable enough, it is easy to cause a system crash or error, or performance is low.

The embodiment of the application describes an adaptive detection method for hardware decoding capability. The method is mainly based on the hardware decoding capability of the DXVA protocol detection Windows platform, and improves the coverage rate of the decoding capability of the detection terminal by adding a mode of multimedia programming interface detection and multimedia programming interface repair and combining with the detection of a native API interface.

The embodiment of the application can be applied to video services (such as cloud games) with high resolution, high frame rate and low delay. The product side needs to detect the hardware decoding capability of the terminal in advance, and decides whether to use a hardware decoder or a software decoder according to the detection result.

Referring to fig. 2, fig. 2 is a schematic structural diagram of a hardware decoding capability detection system according to an embodiment of the present application. The hardware decoding capability detection system comprises a terminal 10, a cloud server 20 and the like; the terminal 10 and the cloud server 20 are connected to each other through a network, for example, a wired or wireless network connection.

Wherein the terminal 10 may be used to display a graphical user interface. The terminal is used for interacting with a user through a graphical user interface, for example, the terminal downloads and installs a corresponding client and operates, for example, the terminal invokes a corresponding applet and operates, for example, the terminal presents a corresponding graphical user interface through a login website, and the like. In the embodiment of the application, the terminal for displaying the cloud game picture can be a terminal for a user, wherein the cloud game picture can contain HDR video or standard dynamic range SDR video with HDR image effect.

The cloud server 20 may be a cloud server, or a platform that provides a service of the cloud server.

Taking a cloud game as an example, when playing the game, a cloud game client in the user operation terminal 10 sends an operation instruction to the cloud server 20, the cloud server 20 runs the game according to the operation instruction, codes and compresses data such as a game picture and returns the data to the cloud game client in the terminal 10 through a network, and finally, based on the hardware decoding capability detection method provided by the embodiment of the present application, the cloud game client in the terminal 10 decodes and outputs the game picture. The game screen includes a target video.

In the embodiment of the present application, when decoding the target video, the terminal 10 receives the target video from the cloud server 20; then the terminal 10 detects whether the system environment of the multimedia programming interface in the terminal meets the expected conditions, if the system environment of the multimedia programming interface does not meet the expected conditions, the user is prompted to perform the repair operation of the multimedia programming interface, and then whether the terminal has hardware decoding capability or not can be further detected according to the static capability and the native API capability of the multimedia programming interface no matter whether the system environment of the multimedia programming interface meets the expected conditions, if the terminal has the hardware decoding capability, the hardware decoder is started to decode the target video, and if the terminal does not have the hardware decoding capability, the software decoder is started to decode the target video. The embodiment of the application can accurately detect the decoding capability of the hardware and verify the validity of the decoding capability through detecting the environment of the multimedia programming interface system.

The following will describe in detail. It should be noted that the following description order of embodiments is not a limitation of the priority order of embodiments.

Referring to fig. 3 to fig. 4, fig. 3 to fig. 4 are flow diagrams of a hardware decoding capability detection method according to an embodiment of the present application. The method may be applied to a terminal as described in fig. 2, the method comprising:

step 110, detecting whether the terminal has hardware decoding capability according to the static capability and the native API capability of the multimedia programming interface.

When the hardware decoding capability detection is performed, the method mainly comprises the steps of detecting the static capability of a multimedia programming interface and detecting the original API capability, and if the terminal has the hardware decoding capability, determining whether to normally start the hardware decoder or not or adopt a software decoder after the hardware code stream test is performed.

Optionally, detecting whether the terminal has hardware decoding capability according to the multimedia programming interface static capability and the native API capability includes: detecting the static capacity of a multimedia programming interface of the terminal based on the hardware parameter information of the terminal, wherein the static capacity of the multimedia programming interface comprises decoding-related static capacity and rendering-related static capacity; detecting a native API capability based on the hardware parameter information; and determining whether the terminal has hardware decoding capability according to the detected multimedia programming interface static capability and the detected native API capability of the terminal.

Optionally, detecting the static capability of the multimedia programming interface of the terminal based on the hardware parameter information of the terminal includes: determining the static capacity related to decoding according to the type of the hardware decoding chip and the capacity of the hardware decoding chip supported by the terminal in the hardware parameter information; and determining the rendering-related static capacity according to the rendering window type and the rendering window capacity supported by the terminal in the hardware parameter information.

Optionally, determining whether the terminal has hardware decoding capability according to the detected multimedia programming interface static capability and the native API capability of the terminal includes: if the static capacity of the multimedia programming interface of the terminal is matched with the first query condition, determining that the terminal has hardware decoding capacity; or if the static capability of the multimedia programming interface of the terminal does not match the first query condition and the native API capability matches the second query condition, determining that the terminal has hardware decoding capability.

For example, the multimedia programming interface static capability detection refers to determining the multimedia programming interface static capability of the terminal according to the detailed hardware parameter information directly obtained from the hardware interface, wherein the multimedia programming interface static capability comprises decoding-related static capability and rendering-related static capability.

And determining the static capacity related to decoding according to the type of the hardware decoding chip and the capacity of the hardware decoding chip supported by the terminal in the hardware parameter information. For example, the hardware decoding chip capability may include the highest resolution and highest frame rate supported by each chip, and for example, the highest resolution and highest frame rate supported by each chip may be determined according to the Size (Size), the compression rate (Profile), and the Level (Level) of each chip in the hardware parameter information. For example, profile is a description of the compression characteristics of video (such as arithmetic coding CABAC, number of color samples, etc.), and may represent the compression rate of an image or video, and Level is a description of the characteristics of video itself (code rate, resolution, frame rate fps), and video is generally divided into levels according to the parameters of video (code rate, resolution, frame rate fps).

Specifically, the decoding-related static capability may include the hardware decoding chip type supported by the terminal (e.g., H264/H265/VP9/AVS, etc.), as well as the highest resolution (e.g., 720p, 1080p, 2K, 4K, etc.) and highest frame rate (e.g., 30fps, 50fps, 60fps, etc.) configurations supported by each chip.

And determining rendering-related static capacity according to the rendering window type and the rendering window capacity supported by the terminal in the hardware parameter information. In particular, the rendering-related static capabilities may include rendering window types supported by the terminal (e.g., D3D9/D3D11, etc.), and rendering window capabilities (frame loss/no frame loss).

For example, DXVA is a decoder interface specification, which has the advantage that the decoding capabilities of different hardware vendors can be invoked through the same set of interfaces, but different vendors also have respective native API interfaces (e.g., intel QSV and NVIDIA CUDA, etc.). The DXVA interface depends on the system environment related to DXVA, while the QSV and CUDA interfaces are generally related to display card driving, and when the static capability of the multimedia programming interface of the terminal does not match the first query condition, and the static capability detection of the multimedia programming interface fails, the native API interface may normally enable the hardware decoder. Therefore, it is necessary to increase the detection of the native API capability of the hardware vendor when the static capability detection fails, so as to increase the success rate of the hardware decoding capability detection. When the static capability detection of the multimedia programming interface is successful, or when the static capability detection of the multimedia programming interface fails and the capability detection of the native API is successful, hardware decoding can be started; when the multimedia programming interface capability detection fails, and the capability detection of the native API fails, software decoding may be enabled.

When determining whether the terminal has hardware decoding capability according to the detected static capability of the multimedia programming interface and the original API capability of the terminal, the method can receive a first query condition input by a user, then query information matched with the first query condition based on hardware parameter information corresponding to the static capability of the multimedia programming interface, and if the information matched with the first query condition exists, determine that the terminal has the hardware decoding capability. If the information matched with the first query condition does not exist, determining that the terminal does not have the hardware decoding capability, then further receiving a second query condition input by a user, querying information matched with the second query condition based on hardware parameter information corresponding to the original API capability, and if the information matched with the second query condition exists, determining that the terminal has the hardware decoding capability. And if the information matched with the second query condition does not exist, determining that the terminal does not have the hardware decoding capability.

For example, if the terminal is determined to have hardware decoding capabilities by detecting the multimedia programming interface static capabilities of the terminal, then the native API capabilities may no longer be detected. The native API capabilities are re-probed only if the hardware decoding capabilities are not supported by the multimedia programming interface static capabilities of the probe terminal.

For example, the first query condition and the second query condition may be the same condition or different conditions. For example, when the first query condition is H265 hard-solution chip type, 1080p resolution, 50fps frame rate, D3D11 rendering window type, etc., and the multimedia programming interface static capability is detected, if the first query condition matches with the information such as the hardware decoding chip type supported by the terminal in the multimedia programming interface static capability, the highest resolution and highest frame rate supported by each chip, the rendering window type supported by the terminal, and the rendering window capability, the terminal is determined to have the hardware decoding capability. In probing native API capabilities, query conditions are also employed for matching.

In step 120, if the terminal has a hardware decoding capability, the hardware decoder is started to decode the target video.

For example, the hardware decoder may be enabled to decode the target video in case the multimedia programming interface static capabilities of the terminal match the first query condition.

For example, a hardware decoder may be initiated to decode a target video in the event that the multimedia programming interface static capabilities of the terminal do not match the first query condition and the native API capabilities match the second query condition.

Optionally, before the hardware decoder is started to decode the target video, the method further includes: performing a hardware code stream test to test whether hardware is normal or not; if the test result is that the hardware decoder is normal, the hardware decoder is started to decode the target video; or if the test result is that the hardware decoder is abnormal, starting the software decoder to decode the target video.

Optionally, performing a hardware code stream test to test whether the hardware decoder is normal, including: based on the capability of the chip of the terminal, a multimedia programming interface decoder is created through the chip, and the test code stream is input into the created multimedia programming interface decoder for decoding test; if the decoding operation is successful, determining that the hardware decoder is normal; or if the decoding operation fails, determining that the hardware decoder is abnormal.

For example, after determining that the terminal has the hardware decoding capability, a hardware code stream test is further performed to test whether the hardware is normal, the validity of the decoding capability can be verified through the hardware code stream test, and the hardware decoder is started only when the hardware decoder is tested to be normal; under the condition that the hardware decoder is tested to be abnormal, the starting of the hardware decoder is avoided, and the situation that the system crashes or is wrong or the performance is low due to the fact that the abnormal hardware decoder is started is effectively avoided. The hard decoding stream test is a query process, a multimedia programming interface object can be created through the chip based on the capability of the chip, whether the interface of the chip is matched or not is queried, namely, a multimedia programming interface decoder is created, a small test code stream is input into the created multimedia programming interface decoder, and if the decoding operation is successful, the hardware decoder is indicated to be normal; the hardware decoder may be enabled to decode the target video in case the terminal has hardware decoding capabilities and the hardware decoder is normal. If the decoding operation fails, the hardware decoder is abnormal; the terminal has hardware decoding capability and in case the hardware decoder is abnormal, the software decoder may be started to decode the target video.

And 130, if the terminal does not have the hardware decoding capability, starting a software decoder to decode the target video.

For example, after the hardware decoding capability detection, if the terminal does not have the hardware decoding capability, the software decoder is started to decode the target video.

For example, after the hardware decoding capability detection and the hardware code stream test, if the terminal has the hardware decoding capability and the hardware decoder is abnormal, the software decoder may be started to decode the target video.

Optionally, before detecting whether the terminal has the hardware decoding capability, the method further includes: detecting whether the system environment of a multimedia programming interface in the terminal meets expected conditions or not; and if the system environment of the multimedia programming interface does not meet the expected conditions, generating prompt information for indicating to repair the multimedia programming interface.

For example, the multimedia programming interface takes a DX interface as an example, and due to a display card driving version problem or a user usage environment problem (such as a dynamic library d3d9.Dll or d3d11.Dll is lost), a user cannot correctly query decoding capability through the DX interface, so that the decoder fails to be created. Where d3d9.Dll is a dynamically linked library of DirectX 9 for supporting Windows multimedia and games. d3d11.dll is a dynamic link library for DirectX 11. The version of D3D11 is higher than the version of D3D9. Therefore, whether the system environment of the multimedia programming interface in the terminal meets the expected conditions or not needs to be detected, and if the system environment of the multimedia programming interface does not meet the expected conditions, a user is prompted to repair the multimedia programming interface.

Optionally, detecting whether the system environment of the multimedia programming interface in the terminal meets the expected condition includes:

judging whether the operating system of the terminal is a target system or not;

if the operating system of the terminal is a target system, judging whether the dynamic library is successfully loaded in the system environment of the first multimedia programming interface version;

if the dynamic library is successfully loaded in the system environment of the first multimedia programming interface version, judging whether the creation of the multimedia programming interface device corresponding to the first multimedia programming interface version is successful or not;

if the dynamic library is successfully loaded in the system environment of the first multimedia programming interface version and the multimedia programming interface device corresponding to the first multimedia programming interface version is successfully created, continuing to judge whether the dynamic library is successfully loaded in the system environment of the second multimedia programming interface version, wherein the level of the second multimedia programming interface version is smaller than that of the first multimedia programming interface version;

if the dynamic library is successfully loaded in the system environment of the second multimedia programming interface version, judging whether the creation of the multimedia programming interface device corresponding to the second multimedia programming interface version is successful or not;

if the operating system of the terminal is a target system, the dynamic library is successfully loaded and the multimedia programming interface device corresponding to the first multimedia programming interface version is successfully created in the system environment of the first multimedia programming interface version, and the dynamic library is successfully loaded and the multimedia programming interface device corresponding to the second multimedia programming interface version is successfully created in the system environment of the second multimedia programming interface version, then the system environment of the multimedia programming interface is determined to accord with expected conditions.

Optionally, if the operating system of the terminal is the target system, loading the dynamic library in the system environment of the first multimedia programming interface version fails, creating the multimedia programming interface device corresponding to the first multimedia programming interface version fails, loading the dynamic library in the system environment of the second multimedia programming interface version succeeds, and creating the multimedia programming interface device corresponding to the second multimedia programming interface version succeeds, and then determining that the system environment of the multimedia programming interface meets the expected condition; or alternatively

If the operating system of the terminal is not the target system and the dynamic library is successfully loaded and the multimedia programming interface equipment corresponding to the second multimedia programming interface version is successfully created under the system environment of the second multimedia programming interface version, determining that the system environment of the multimedia programming interface meets the expected conditions.

Optionally, if the operating system of the terminal is the target system, loading the dynamic library fails and creating the multimedia programming interface device corresponding to the first multimedia programming interface version fails in the system environment of the first multimedia programming interface version, and loading the dynamic library fails and creating the multimedia programming interface device corresponding to the second multimedia programming interface version fails in the system environment of the second multimedia programming interface version, then determining that the system environment of the multimedia programming interface does not meet the expected condition; or alternatively

If the operating system of the terminal is not the target system and the loading of the dynamic library fails and/or the creation of the multimedia programming interface device corresponding to the second multimedia programming interface version fails in the system environment of the second multimedia programming interface version, determining that the system environment of the multimedia programming interface does not meet the expected conditions.

Optionally, if the system environment of the multimedia programming interface does not meet the expected condition, generating prompt information for indicating to repair the multimedia programming interface.

Optionally, whether the system environment of the multimedia programming interface meets the expected condition or not, whether the terminal has hardware decoding capability or not can be further detected according to the static capability and the native API capability of the multimedia programming interface. For example, in the case that the system environment of the multimedia programming interface meets the expected conditions and the terminal has the hardware decoding capability, the hardware decoder is started to decode the target video, so that the hardware decoder can be operated in a stable and matched system environment, and the operation stability of the hardware decoder is increased. For example, when the system environment of the multimedia programming interface does not meet the expected conditions and the static capability detection of the multimedia programming interface of the terminal fails, the decoder may be normally enabled by the native API interface, so that it is necessary to increase the detection of the native API capability of the hardware manufacturer when the system environment of the multimedia programming interface does not meet the expected conditions and/or the static capability detection of the multimedia programming interface of the terminal fails, thereby improving the success rate of the hardware decoding capability detection.

For example, taking the application to a Windows platform and the multimedia programming interface as a DX interface as an example, the target system may be an operating system above Windows 7; or the target system may be other specified versions of the operating system. The first multimedia programming interface version is D3D11, the second multimedia programming interface version is D3D9, and the version level of D3D11 is higher than the version level of D3D9.

The step 110 will be described with reference to steps 202 to 209 in fig. 3: firstly, judging whether an operating system of a terminal is more than Windows 7; if the operating system is above Windows7, detecting D3D11 and then D3D9; if the operating system is not Windows7 or more, D3D9 is detected directly.

The method mainly comprises the steps of loading a dynamic library, creating a D3D11 device or a D3D9 device to detect a system dependent environment of a multimedia programming interface version, verifying whether the D3D11 system environment and the D3D9 system environment decoded by hardware meet expected conditions, generating prompt information for indicating to repair the multimedia programming interface, informing a user or a terminal held by the user to repair the multimedia programming interface (such as updating a driving version and the like), and counting and reporting detection results.

For example, the expected conditions include: when the operating system is above Windows7, the dynamic library is successfully loaded and the D3D11 device is successfully created in the D3D11 environment, and the dynamic library is successfully loaded and the D3D9 device is successfully created in the D3D9 environment; when the operating system is above Windows7, the dynamic library loading and the D3D11 device creating under the D3D11 environment fail, and the dynamic library loading and the D3D9 device creating under the D3D9 environment succeed; or when the operating system is not above Windows7 and loading the dynamic library in the D3D9 environment is successful and creating the D3D9 device is successful.

Specifically, whether an operating system installed in a terminal is more than Windows7 is judged; if the operating system installed in the terminal is above Windows7, D3D11 environment detection is performed based on the rendering window type supported by the terminal; judging whether the dynamic library is successfully loaded in the D3D11 environment; if the dynamic library is successfully loaded in the D3D11 environment, whether the D3D11 device is successfully created is further judged; if the loading of the dynamic library in the D3D11 environment fails and the creation of the D3D11 device fails, notifying the terminal to repair the multimedia programming interface; if the D3D11 device is successfully created, D3D9 environment detection is further performed based on the rendering window type supported by the terminal; judging whether the dynamic library is successfully loaded in the D3D9 environment; if the dynamic library is successfully loaded in the D3D9 environment, whether the D3D9 device is successfully created is further judged; if the loading of the dynamic library in the D3D9 environment fails and the creation of the D3D9 equipment fails, notifying the terminal to repair the multimedia programming interface; if the D3D9 device is successfully created, the detection is ended, and the detection result is reported.

After the repair of the multimedia programming interface is performed, the repair result of the multimedia programming interface can be reported. The terminal can trigger the system environment detection of the new round of the multimedia programming interface based on the repair result of the multimedia programming interface and report the detection result of the new round.

If the repair of the multimedia programming interface is performed, the probability that the system environment of the multimedia programming interface meets the expected condition is higher in the new round of hardware decoding capability detection, namely, the supporting probability of the hardware detection capability is higher.

If the repair of the multimedia programming interface is not performed, the probability that the system environment of the multimedia programming interface meets the expected condition is low in the new round of hardware decoding capability detection, namely the supporting probability of the hardware detection capability is low.

Optionally, if it is detected that the terminal does not perform the multimedia programming interface repair, determining whether the terminal has hardware decoding capability by detecting a native API capability.

Optionally, before detecting whether the system environment of the multimedia programming interface in the terminal meets the expected condition, the method further comprises: matching cloud server configuration according to the model of the terminal, and inquiring whether the terminal is in a detected list; if the terminal is in the detected list, starting a hardware decoder to decode the target video; or if the terminal is not in the detected list, detecting whether the system environment of the multimedia programming interface in the terminal meets the expected condition.

For example, firstly, according to the model matching cloud server configuration of the terminal, inquiring whether the current terminal is in a detected list, if so, directly skipping the detection step, starting hardware decoding based on the application configuration, and reporting the decoding result; if the current terminal is not in the detected list, further detecting whether the system environment of the multimedia programming interface in the terminal meets the expected condition.

Optionally, before decoding the target video, the method further comprises:

acquiring the encoding and decoding capability information of the terminal based on a capability information acquisition request initiated by the cloud server to the terminal through the cloud game client;

transmitting the encoding and decoding capability information to the cloud server through the cloud game client so that the cloud server determines encoding and decoding configuration and encoding and decoding strategies according to the encoding and decoding capability information;

acquiring a target video which is output by the cloud server after being encoded based on encoding and decoding configuration and encoding and decoding strategies;

if the terminal has the hardware decoding capability, starting a hardware decoder to decode the target video based on the encoding and decoding configuration; or alternatively

And if the terminal does not have the hardware decoding capability, starting a software decoder to decode the target video based on the coding and decoding configuration.

For example, referring to the flowchart shown in fig. 1, the cloud server initiates a capability information obtaining request to a terminal (intelligent terminal) through a cloud game client (such as a START client) to obtain the codec capability information of the terminal. The request protocol field may include information such as a protocol version number, a specific codec protocol query, and the like. And when the terminal receives the capability information acquisition request, the terminal returns the encoding and decoding capability information of the terminal to the cloud server through the cloud game client, wherein the encoding and decoding capability information can comprise a state mark, a supported protocol version number and terminal equipment capability information, and the terminal equipment capability information can comprise terminal encoding capability information and terminal decoding capability information. Wherein the capability information acquisition request may comprise a decoding capability request and/or an encoding capability request. After receiving the encoding and decoding capability information, the cloud server can determine encoding and decoding configuration and encoding and decoding strategies according to the encoding and decoding capability information, the game type and the user network condition, the encoding and decoding configuration can comprise an optimal decoding protocol, decoding resolution, video frame rate and the like of the current terminal equipment, and the encoding and decoding strategies can comprise the number of video encoding reference frames, SVC enablement and the like. The cloud server can determine a coding function set to be started according to the game type and the network condition, and then determine the optimal coding configuration of the current terminal through the equipment type and the coding capability information reported by the terminal; the cloud server can determine the optimal encoding and decoding configuration such as the decoding protocol, the decoding resolution, the video frame rate and the like of the current terminal, and encoding and decoding strategies such as the number of video encoding reference frames, SVC starting and the like according to the decoding capability information of the terminal and by combining the game type and the network condition.

Then, the cloud server encodes the target video based on the encoding and decoding configuration and the encoding and decoding strategy, and then outputs the code stream of the target video. The cloud server also transmits the optimal decoding configuration in the encoding and decoding configuration to the terminal, and the terminal decodes the code stream of the target video based on the optimal decoding configuration in the optimal decoding configuration after receiving the optimal decoding configuration in the encoding and decoding configuration. If the terminal has the hardware decoding capability, starting a hardware decoder to decode the target video based on the optimal decoding configuration in the encoding and decoding configuration; or if the terminal does not have the hardware decoding capability, starting a hardware decoder to decode the target video based on the coding and decoding configuration.

In order to better explain the method for detecting the hardware decoding capability provided in the embodiment of the present application, please refer to fig. 4, the flow of the method for detecting the hardware decoding capability provided in the embodiment of the present application can be summarized as the following steps:

step 201, matching cloud server configuration according to the model of the terminal, and inquiring whether the terminal is in a detected list; if yes, go to step 217; if not, step 202 is performed.

Step 202, judging whether an operating system of a terminal is a target system; if yes, go to step 203; if not, step 205 is performed.

Step 203, judging whether the dynamic library is successfully loaded in the system environment of the first multimedia programming interface version; if yes, go to step 204; if not, go to step 205 and step 209.

Step 204, judging whether the creation of the multimedia programming interface device corresponding to the first multimedia programming interface version is successful; if yes, go to step 205 and step 207; if not, go to step 205 and step 209.

Step 205, judging whether the loading of the dynamic library is successful in the system environment of the second multimedia programming interface version; if yes, go to step 206; if not, go to step 208.

Step 206, judging whether the creation of the multimedia programming interface device corresponding to the second multimedia programming interface version is successful; if yes, go to step 207; if not, go to step 208.

Step 207, determining that the system environment of the multimedia programming interface meets the expected conditions. And then step 210 is performed.

Step 208, determining that the system environment of the multimedia programming interface is not in compliance with the expected conditions. Step 209 and/or step 210 may be performed after step 208.

Whether the system environment of the multimedia programming interface meets the expected conditions or not, whether the terminal has hardware decoding capability or not can be further detected according to the static capability and the native API capability of the multimedia programming interface. Specifically, after step 207 and/or step 208, step 210 may be further performed.

Step 209, generating prompt information for indicating to perform multimedia programming interface repair.

After generating the prompt message for indicating to repair the multimedia programming interface, if the user inputs a repair instruction according to the prompt message displayed by the terminal, the terminal repairs the multimedia programming interface according to the repair instruction, such as updating the driving version, etc. If it is detected that the terminal has performed the repair of the multimedia programming interface when the hardware decoding capability detection process is not finished, the method may return to the step of executing the detection of whether the system environment of the multimedia programming interface in the terminal meets the expected condition (e.g. steps 202 to 208 in fig. 5).

When the hardware decoding capability detection flow of the round is finished, if the terminal is detected to repair the multimedia programming interface, the repair result of the multimedia programming interface can be reported. The terminal can trigger the system environment detection of the new-round multimedia programming interface based on the repair result of the multimedia programming interface in the new-round hardware decoding capability detection flow, and report the new-round detection result.

After generating the prompt information for indicating that the multimedia programming interface repair is performed, if it is detected that the terminal does not perform the multimedia programming interface repair, step 213 may be performed to determine whether the terminal has a hardware decoding capability through the native API capability.

Step 210, detecting the multimedia programming interface static capability of the terminal based on the hardware parameter information of the terminal, wherein the multimedia programming interface static capability comprises decoding related static capability and rendering related static capability.

Step 211, detecting the native API capability based on the hardware parameter information.

Step 212, judging whether the static capacity of the multimedia programming interface of the terminal is matched with the first query condition; if yes, go to step 215; if not, step 213 is performed.

Step 213, judging whether the native API capability matches the second query condition; if yes, go to step 215; if not, go to step 214.

Step 214, determining that the terminal does not have hardware decoding capability.

Step 215, determining that the terminal has hardware decoding capability.

If it is determined that the terminal has the hardware decoding capability, step 216 may be performed before the hardware decoder is started to decode the target video in step 217.

Step 216, if the system environment of the multimedia programming interface meets the expected conditions and the terminal has hardware decoding capability, performing a hardware code stream test to test whether the hardware is normal; if yes, go to step 217; if not, go to step 218. The validity of the decoding capability can be verified through the hardware code stream test, and the hardware decoder is started only when the hardware decoder is tested to be normal; under the condition that the hardware decoder is tested to be abnormal, the starting of the hardware decoder is avoided, and the condition that the system crashes or is wrong or the performance is low due to the starting of the abnormal hardware decoder is effectively avoided

In step 217, the hardware decoder is enabled to decode the target video.

For example, the hardware decoder may be activated to decode the target video in case the system environment of the multimedia programming interface meets the expected conditions and the terminal has hardware decoding capabilities and the hardware decoder is normal. Specifically, the hardware decoder may be started to decode the target video when the system environment of the multimedia programming interface meets the expected condition and the static capability of the multimedia programming interface of the terminal matches the first query condition, and the hardware decoder is normal. Alternatively, the hardware decoder may be started to decode the target video when the system environment of the multimedia programming interface meets the expected conditions, the static capabilities of the multimedia programming interface of the terminal do not match the first query conditions, and the native API capabilities match the second query conditions, and the hardware decoder is normal.

For example, the hardware decoder may be started to decode the target video when the system environment of the multimedia programming interface is not in accordance with the expected condition and the terminal has the hardware decoding capability and the hardware decoder is normal. Wherein, in case that the system environment of the multimedia programming interface does not meet the expected conditions and the native API capability supports the hardware decoding capability, the hardware decoder can be normally enabled.

Step 218, the software decoder is enabled to decode the target video.

For example, the software decoder may be started to decode the target video in case the system environment of the multimedia programming interface meets the expected conditions and the terminal has hardware decoding capabilities and the hardware decoder is abnormal.

For example, after the system environment detection and the hardware decoding capability detection of the multimedia programming interface are performed, if the system environment of the multimedia programming interface does not meet the expected conditions and the terminal does not have the hardware decoding capability, the software decoder is started to decode the target video.

For example, after the system environment detection and the hardware decoding capability detection of the multimedia programming interface are performed, if the system environment of the multimedia programming interface does not meet the expected conditions and the terminal does not have the hardware decoding capability, the software decoder may be started to decode the target video.

The embodiment of the application can realize the self-adaptive detection of the hardware decoding capability: firstly, according to the model matching cloud server configuration of the terminal, inquiring whether the current terminal is in a detected list, if so, directly skipping a detection step, starting hardware decoding based on application configuration, and reporting a decoding result; and if the current terminal is not in the detected list, detecting the hardware decoding capability. The process of detecting the hardware capability comprises the following steps: firstly, detecting the system environment of a multimedia programming interface, and prompting a user or a prompting terminal to repair the multimedia programming interface if the system environment of the multimedia programming interface does not accord with the expected conditions; and then detecting the hardware decoding capability, mainly comprising the static capability detection and the original API capability detection of the multimedia programming interface, and if the terminal has the hardware decoding capability, determining whether to normally start the hardware decoder or not by performing hardware code stream test, and if not, adopting a software decoder. The self-adaptive detection of the hardware decoding capability can be realized, the coverage rate of the decoding capability of the detection terminal is improved, the probability of video jamming is reduced to a certain extent, and the user experience is improved.

All the above technical solutions may be combined to form an optional embodiment of the present application, which is not described here in detail.

According to the embodiment of the application, whether the terminal has hardware decoding capability is detected according to the static capability and the native API capability of the multimedia programming interface; if the terminal has the hardware decoding capability, starting a hardware decoder to decode the target video; and if the terminal does not have the hardware decoding capability, starting a software decoder to decode the target video. According to the embodiment of the application, the hardware decoding capability can be accurately detected by detecting the static capability and the native API capability of the multimedia programming interface, and the coverage rate of the decoding capability of the detection terminal is improved. The embodiment of the application can also verify the validity of the decoding capability through the hardware code stream test, and the hardware decoder is started only when the hardware decoder is tested to be normal; under the condition that the hardware decoder is tested to be abnormal, the starting of the hardware decoder is avoided, and the situation that the system crashes or is wrong or the performance is low due to the fact that the abnormal hardware decoder is started is effectively avoided.

In order to facilitate better implementation of the hardware decoding capability detection method of the embodiment of the application, the embodiment of the application also provides a hardware decoding capability detection device. Referring to fig. 5 and fig. 6, fig. 5 and fig. 6 are schematic structural diagrams of a hardware decoding capability detection device according to an embodiment of the present application. The hardware decoding capability detection device 300 may include:

A detecting unit 303, configured to detect whether the terminal has a hardware decoding capability according to the static capability of the multimedia programming interface and the API capability of the native application programming interface;

a first decoding unit 304, configured to start a hardware decoder to decode the target video if the terminal has a hardware decoding capability;

the second decoding unit 305 is configured to start the software decoder to decode the target video if the terminal does not have the hardware decoding capability.

Optionally, the detecting unit 303 may be configured to: detecting the static capacity of a multimedia programming interface of the terminal based on the hardware parameter information of the terminal, wherein the static capacity of the multimedia programming interface comprises decoding-related static capacity and rendering-related static capacity; detecting a native API capability based on the hardware parameter information; and determining whether the terminal has hardware decoding capability according to the detected multimedia programming interface static capability and the detected native API capability of the terminal.

Optionally, the detecting unit 303 may be configured to, when detecting the static capability of the multimedia programming interface of the terminal based on the hardware parameter information of the terminal: determining the static capacity related to decoding according to the type of the hardware decoding chip and the capacity of the hardware decoding chip supported by the terminal in the hardware parameter information; and determining the rendering-related static capacity according to the rendering window type and the rendering window capacity supported by the terminal in the hardware parameter information.

Optionally, the detecting unit 303 may be configured to, when determining whether the terminal has hardware decoding capability according to the detected multimedia programming interface static capability and the native API capability of the terminal: if the static capacity of the multimedia programming interface of the terminal is matched with the first query condition, determining that the terminal has hardware decoding capacity; or if the static capability of the multimedia programming interface of the terminal does not match the first query condition and the native API capability matches the second query condition, determining that the terminal has hardware decoding capability.

Optionally, the detection unit 303 may be further configured to: if the terminal has the hardware decoding capability, performing a hardware code stream test to test whether the hardware is normal or not;

the first decoding unit 304 is further configured to, if the test result is that the hardware decoder is normal, start the hardware decoder to decode the target video; or alternatively

The second decoding unit 305 is further configured to start the software decoder to decode the target video if the test result is that the hardware decoder is abnormal.

Optionally, when performing a hardware code stream test to test whether the hardware decoder is normal, the detecting unit 303 may be configured to: based on the capability of the chip of the terminal, a multimedia programming interface decoder is created through the chip, and the test code stream is input into the created multimedia programming interface decoder for decoding test; if the decoding operation is successful, determining that the hardware decoder is normal; or if the decoding operation fails, determining that the hardware decoder is abnormal.

Optionally, referring to fig. 6, the hardware decoding capability detection apparatus 300 may further include an inquiry unit 301, a detection unit 302, a first acquisition unit 306, a transmission unit 307, and a second acquisition unit 308.

Wherein, the detection unit 302 may be used for: detecting whether the system environment of a multimedia programming interface in the terminal meets expected conditions or not; and if the system environment of the multimedia programming interface does not meet the expected conditions, generating prompt information for indicating to repair the multimedia programming interface.

Optionally, the detecting unit 302 may be configured to: judging whether the operating system of the terminal is a target system or not; if the operating system of the terminal is a target system, judging whether the dynamic library is successfully loaded in the system environment of the first multimedia programming interface version; if the dynamic library is successfully loaded in the system environment of the first multimedia programming interface version, judging whether the creation of the multimedia programming interface device corresponding to the first multimedia programming interface version is successful or not; if the dynamic library is successfully loaded in the system environment of the first multimedia programming interface version and the multimedia programming interface device corresponding to the first multimedia programming interface version is successfully created, continuing to judge whether the dynamic library is successfully loaded in the system environment of the second multimedia programming interface version, wherein the level of the second multimedia programming interface version is smaller than that of the first multimedia programming interface version; if the dynamic library is successfully loaded in the system environment of the second multimedia programming interface version, judging whether the creation of the multimedia programming interface device corresponding to the second multimedia programming interface version is successful or not; if the operating system of the terminal is a target system, the dynamic library is successfully loaded and the multimedia programming interface device corresponding to the first multimedia programming interface version is successfully created in the system environment of the first multimedia programming interface version, and the dynamic library is successfully loaded and the multimedia programming interface device corresponding to the second multimedia programming interface version is successfully created in the system environment of the second multimedia programming interface version, then the system environment of the multimedia programming interface is determined to accord with expected conditions.

Optionally, the detection unit 302 may be further configured to: if the operating system of the terminal is a target system, the dynamic library is successfully loaded and the multimedia programming interface equipment corresponding to the first multimedia programming interface version is successfully created in the system environment of the first multimedia programming interface version, and the dynamic library is successfully loaded and the multimedia programming interface equipment corresponding to the second multimedia programming interface version is successfully created in the system environment of the second multimedia programming interface version, then the system environment of the multimedia programming interface is determined to accord with expected conditions; or if the operating system of the terminal is not the target system and the dynamic library is successfully loaded and the multimedia programming interface equipment corresponding to the second multimedia programming interface version is successfully created under the system environment of the second multimedia programming interface version, determining that the system environment of the multimedia programming interface meets the expected conditions.

Optionally, the detection unit 302 may be further configured to: if the operating system of the terminal is a target system, the loading of the dynamic library fails and the creation of the multimedia programming interface equipment corresponding to the first multimedia programming interface version fails in the system environment of the first multimedia programming interface version, and the loading of the dynamic library fails and the creation of the multimedia programming interface equipment corresponding to the second multimedia programming interface version fails in the system environment of the second multimedia programming interface version, then determining that the system environment of the multimedia programming interface does not meet the expected conditions; or if the operating system of the terminal is not the target system and the loading of the dynamic library fails and/or the creation of the multimedia programming interface device corresponding to the second multimedia programming interface version fails in the system environment of the second multimedia programming interface version, determining that the system environment of the multimedia programming interface does not meet the expected condition.

Optionally, the query unit 301 may be configured to: matching cloud server configuration according to the model of the terminal, and inquiring whether the terminal is in a detected list; if the terminal is in the detected list, starting a hardware decoder to decode the target video; or if the terminal is not in the detected list, detecting whether the system environment of the multimedia programming interface in the terminal meets the expected condition.

Optionally, the hardware decoding capability detection device 300 further includes:

a first obtaining unit 306, configured to obtain, based on a capability information obtaining request initiated by the cloud server through the cloud game client to the terminal, codec capability information of the terminal;

a sending unit 307, configured to send, through the cloud game client, the codec capability information to the cloud server, so that the cloud server determines codec configuration and codec policy according to the codec capability information;

a second obtaining unit 308, configured to obtain a target video that is output by the cloud server after encoding based on the codec configuration and the codec policy;

the first decoding unit 304 is further configured to, if the terminal has a hardware decoding capability, start the hardware decoder to decode the target video based on the codec configuration; or alternatively

The second decoding unit 305 is further configured to start the software decoder to decode the target video based on the codec configuration if the terminal does not have the hardware decoding capability.

The respective units in the above-described hardware decoding capability detection device may be implemented in whole or in part by software, hardware, and a combination thereof. The above units may be embedded in hardware or may be independent of a processor in the computer device, or may be stored in software in a memory in the computer device, so that the processor invokes and executes operations corresponding to the above units.

The hardware decoding capability detection device 300 may be integrated in a terminal or a server having a memory and a processor mounted therein and having an arithmetic capability, or the hardware decoding capability detection device 300 may be the terminal or the server.

Fig. 7 is a schematic structural diagram of a computer device provided in an embodiment of the present application, where the computer device may be the terminal shown in fig. 2. As shown in fig. 7, the computer device 400 may include: a communication interface 401, a memory 402, a processor 403 and a communication bus 404. Communication interface 401, memory 402, and processor 403 communicate with each other via communication bus 404. The communication interface 401 is used for data communication of the apparatus 700 with external devices. The memory 402 may be used to store software programs and modules, and the processor 403 may execute the software programs and modules stored in the memory 402, such as the software programs for corresponding operations in the foregoing method embodiments.

Alternatively, the processor 403 may call a software program and module stored in the memory 402 to perform the following operations: detecting whether the terminal has hardware decoding capability according to the static capability of the multimedia programming interface and the API capability of the original application programming interface; if the terminal has the hardware decoding capability, starting a hardware decoder to decode the target video; and if the terminal does not have the hardware decoding capability, starting a software decoder to decode the target video.

The present application also provides a computer-readable storage medium for storing a computer program. The computer readable storage medium may be applied to a computer device, and the computer program causes the computer device to execute a corresponding flow in the hardware decoding capability detection method in the embodiments of the present application, which is not described herein for brevity.

The present application also provides a computer program product comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the computer device executes a corresponding flow in the hardware decoding capability detection method in the embodiment of the present application, which is not described herein for brevity.

The present application also provides a computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the computer device executes a corresponding flow in the hardware decoding capability detection method in the embodiment of the present application, which is not described herein for brevity.

It should be appreciated that the processor of an embodiment of the present application may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method embodiments may be implemented by integrated logic circuits of hardware in a processor or instructions in software form. The processor may be a general purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), an off-the-shelf programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components. The disclosed methods, steps, and logic blocks in the embodiments of the present application may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of a method disclosed in connection with the embodiments of the present application may be embodied directly in hardware, in a decoded processor, or in a combination of hardware and software modules in a decoded processor. The software modules may be located in a random access memory, flash memory, read only memory, programmable read only memory, or electrically erasable programmable memory, registers, etc. as well known in the art. The storage medium is located in a memory, and the processor reads the information in the memory and, in combination with its hardware, performs the steps of the above method.

It will be appreciated that the memory in embodiments of the present application may be either volatile memory or nonvolatile memory, or may include both volatile and nonvolatile memory. The nonvolatile Memory may be a Read-Only Memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically Erasable EPROM (EEPROM), or a flash Memory. The volatile memory may be random access memory (Random Access Memory, RAM) which acts as an external cache. By way of example, and not limitation, many forms of RAM are available, such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (Double Data Rate SDRAM), enhanced SDRAM (ESDRAM), synchronous DRAM (SLDRAM), and Direct RAM (DR RAM). It should be noted that the memory of the systems and methods described herein is intended to comprise, without being limited to, these and any other suitable types of memory.

Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

It will be clear to those skilled in the art that, for convenience and brevity of description, specific working procedures of the above-described systems, apparatuses and units may refer to corresponding procedures in the foregoing method embodiments, and are not repeated herein.

In the several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods may be implemented in other manners. For example, the apparatus embodiments described above are merely illustrative, e.g., the division of the units is merely a logical function division, and there may be additional divisions when actually implemented, e.g., multiple units or components may be combined or integrated into another system, or some features may be omitted or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical or other form.

The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in the embodiments of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit.

The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer-readable storage medium. Based on such understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the present technology or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server) to perform all or part of the steps of the method described in the embodiments of the present application. And the aforementioned storage medium includes: a usb disk, a removable hard disk, a ROM, a RAM, a magnetic disk, or an optical disk, etc.

The foregoing is merely specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily think about changes or substitutions within the technical scope of the present application, and the changes and substitutions are intended to be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A method for detecting decoding capability of hardware, the method comprising:

detecting whether the terminal has hardware decoding capability according to the static capability of the multimedia programming interface and the API capability of the original application programming interface;

if the terminal has the hardware decoding capability, starting a hardware decoder to decode the target video;

and if the terminal does not have the hardware decoding capability, starting a software decoder to decode the target video.

2. The hardware decoding capability detection method as claimed in claim 1, wherein the detecting whether the terminal has hardware decoding capability according to multimedia programming interface static capability and native application programming interface API capability comprises:

detecting the static capacity of a multimedia programming interface of the terminal based on the hardware parameter information of the terminal, wherein the static capacity of the multimedia programming interface comprises decoding-related static capacity and rendering-related static capacity;

Detecting a native API capability based on the hardware parameter information;

and determining whether the terminal has hardware decoding capability according to the detected multimedia programming interface static capability and the detected native API capability of the terminal.

3. The hardware decoding capability detection method of claim 2, wherein detecting the multimedia programming interface static capability of the terminal based on the hardware parameter information of the terminal comprises:

determining the static capacity related to decoding according to the type of the hardware decoding chip and the capacity of the hardware decoding chip supported by the terminal in the hardware parameter information;

and determining the rendering-related static capacity according to the rendering window type and the rendering window capacity supported by the terminal in the hardware parameter information.

4. The hardware decoding capability detection method of claim 2, wherein the determining whether the terminal has hardware decoding capability according to the detected multimedia programming interface static capability and the native API capability of the terminal comprises:

if the static capacity of the multimedia programming interface of the terminal is matched with a first query condition, determining that the terminal has hardware decoding capacity; or alternatively

And if the static capacity of the multimedia programming interface of the terminal does not match the first query condition and the native API capacity matches the second query condition, determining that the terminal has hardware decoding capacity.

5. The hardware decoding capability detection method of claim 1, wherein before enabling the hardware decoder to decode the target video, the method further comprises:

performing a hardware code stream test to test whether hardware is normal or not;

if the test result is that the hardware decoder is normal, the hardware decoder is started to decode the target video; or alternatively

And if the test result is that the hardware decoder is abnormal, starting the software decoder to decode the target video.

6. The hardware decoding capability detection method of claim 5, wherein performing the hardware bitstream test to test whether the hardware decoder is normal comprises:

based on the capability of the chip of the terminal, a multimedia programming interface decoder is created through the chip, and a test code stream is input into the created multimedia programming interface decoder for decoding test;

if the decoding operation is successful, determining that the hardware decoder is normal; or alternatively

If the decoding operation fails, determining that the hardware decoder is abnormal.

7. The hardware decoding capability detection method of claim 1, wherein before detecting whether the terminal has hardware decoding capability, the method further comprises:

detecting whether the system environment of a multimedia programming interface in the terminal meets expected conditions or not;

and if the system environment of the multimedia programming interface does not meet the expected conditions, generating prompt information for indicating to repair the multimedia programming interface.

8. The hardware decoding capability detection method of claim 1, wherein detecting whether a system environment of a multimedia programming interface in a terminal meets an expected condition comprises:

judging whether the operating system of the terminal is a target system or not;

if the dynamic library is successfully loaded in the system environment of the first multimedia programming interface version and the multimedia programming interface equipment corresponding to the first multimedia programming interface version is successfully created, continuously judging whether the dynamic library is successfully loaded in the system environment of a second multimedia programming interface version, wherein the level of the second multimedia programming interface version is smaller than that of the first multimedia programming interface version;

9. The hardware decoding capability detection method of claim 8, wherein the method further comprises:

if the operating system of the terminal is a target system, the dynamic library loading and the multimedia programming interface equipment corresponding to the first multimedia programming interface version loading fail under the system environment of the first multimedia programming interface version, and the dynamic library loading and the multimedia programming interface equipment corresponding to the second multimedia programming interface version loading succeed under the system environment of the second multimedia programming interface version, determining that the system environment of the multimedia programming interface meets the expected conditions; or alternatively

If the operating system of the terminal is not the target system and the dynamic library is successfully loaded and the multimedia programming interface equipment corresponding to the second multimedia programming interface version is successfully created under the system environment of the second multimedia programming interface version, determining that the system environment of the multimedia programming interface meets the expected condition.

10. The hardware decoding capability detection method of claim 8, wherein the method further comprises:

if the operating system of the terminal is a target system, the loading of the dynamic library fails and the creation of the multimedia programming interface equipment corresponding to the first multimedia programming interface version fails in the system environment of the first multimedia programming interface version, and the loading of the dynamic library fails and the creation of the multimedia programming interface equipment corresponding to the second multimedia programming interface version fails in the system environment of the second multimedia programming interface version, then determining that the system environment of the multimedia programming interface does not meet the expected condition; or alternatively

If the operating system of the terminal is not the target system and the loading of the dynamic library fails and/or the creation of the multimedia programming interface device corresponding to the second multimedia programming interface version fails in the system environment of the second multimedia programming interface version, determining that the system environment of the multimedia programming interface does not meet the expected condition.

11. The hardware decoding capability detection method of claim 7, further comprising, before detecting whether a system environment of a multimedia programming interface in the terminal meets an expected condition:

matching cloud server configuration according to the model of the terminal, and inquiring whether the terminal is in a detected list or not;

if the terminal is in the detected list, starting a hardware decoder to decode the target video; or alternatively

If the terminal is not in the detected list, detecting whether the system environment of the multimedia programming interface in the terminal meets the expected condition.

12. The hardware decoding capability detection method of claim 1, wherein prior to decoding the target video, the method further comprises:

acquiring the coding and decoding capability information of the terminal based on a capability information acquisition request initiated by a cloud server to the terminal through a cloud game client;

acquiring the target video which is output by the cloud server after being coded based on the coding and decoding configuration and the coding and decoding strategy;

13. A hardware decoding capability detection apparatus, the apparatus comprising:

the detection unit is used for detecting whether the terminal has hardware decoding capability according to the static capability of the multimedia programming interface and the API capability of the original application programming interface;

the first decoding unit is used for starting a hardware decoder to decode the target video if the terminal has hardware decoding capability;

and the second decoding unit is used for starting the software decoder to decode the target video if the terminal does not have the hardware decoding capability.

14. A computer readable storage medium, characterized in that the computer readable storage medium stores a computer program adapted to be loaded by a processor for performing the steps of the hardware decoding capability detection method according to any of claims 1-12.

15. A computer device, characterized in that it comprises a processor and a memory, in which a computer program is stored, the processor being arranged to perform the steps of the hardware decoding capability detection method according to any of claims 1-12 by invoking the computer program stored in the memory.