CN117692642A

CN117692642A - Method, apparatus, device and storage medium for setting coding and decoding standard

Info

Publication number: CN117692642A
Application number: CN202211080769.XA
Authority: CN
Inventors: 杨小祥; 曹洪彬; 陈思佳; 曹健; 黄永铖; 宋美佳; 张佳
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2022-09-05
Filing date: 2022-09-05
Publication date: 2024-03-12

Abstract

The application discloses a method, a device, equipment and a storage medium for setting coding and decoding standards, and belongs to the technical field of videos. The method comprises the following steps: acquiring decoding performance information of a terminal; responding to the decoding performance information to support a first coding and decoding standard, and acquiring a test video stream coded by adopting the first coding and decoding standard; decoding the test video stream by adopting a hardware decoding mode to obtain a decoded video stream; determining a difference between a number of video frames in the test video stream and a number of video frames in the decoded video stream; and sending a test passing notification to a server in response to the difference meeting a test passing condition, wherein the test passing notification is used for instructing the server to set the first coding and decoding standard to be the coding and decoding standard adopted by sending the business video stream to the terminal. By adopting the method and the device, the decoding stability can be ensured, and the output video quality can be improved.

Description

Method, apparatus, device and storage medium for setting coding and decoding standard

Technical Field

The present invention relates to the field of video technologies, and in particular, to a method, an apparatus, a device, and a storage medium for setting a codec standard.

Background

Hardware decoding is decoding of video by the graphics processor of the terminal, which has the advantage that the occupation of resources of the central processing unit (central processing unit, CPU) can be minimized. In the fields of cloud games, live broadcasting and the like, hardware decoding is widely applied.

In the related art, an application program queries the decoding capability of a hardware decoder of a terminal through a query interface and notifies the decoding capability to a server. And the server selects a coding and decoding standard supported by the decoding capability to perform video coding based on the queried decoding capability of the hardware decoder, and sends the coded video to the terminal to perform decoding and playing.

The method for inquiring the decoding capability of the hardware decoder through the interface can only determine that the hardware decoder has the decoding capability, and cannot ensure the decoding quality of the hardware decoder. The coding and decoding standard determined by the method can cause the problems of blocking, black screen, breakdown and the like during decoding, so that the decoding is unstable and the playing quality of the video is poor.

Disclosure of Invention

The embodiment of the application provides a method, a device, equipment and a storage medium for setting coding and decoding standards, which can solve the problems in the related art, and the technical scheme is as follows:

In a first aspect, there is provided a method of setting a codec standard, the method comprising:

acquiring decoding performance information of a terminal;

responding to the decoding performance information to support a first coding and decoding standard, and acquiring a test video stream coded by adopting the first coding and decoding standard;

decoding the test video stream by adopting a hardware decoding mode to obtain a decoded video stream;

determining a difference between a number of video frames in the test video stream and a number of video frames in the decoded video stream;

and sending a test passing notification to a server in response to the difference meeting a test passing condition, wherein the test passing notification is used for instructing the server to set the first coding and decoding standard to be the coding and decoding standard adopted by sending the business video stream to the terminal.

In one possible implementation, the method further includes:

transmitting a non-support notification to the server in response to the decoding performance information not supporting the first codec standard, wherein the non-support notification is used for instructing the server to set the second codec standard to a codec standard adopted by transmitting the service video stream to the terminal;

and sending a test failure notification to the server in response to the difference value not meeting the test passing condition, wherein the test failure notification is used for instructing the server to set the second coding and decoding standard as the coding and decoding standard adopted by sending the business video stream to the terminal.

In one possible implementation manner, the obtaining the test video stream encoded using the first codec standard includes:

sending a decoding performance test request to the server, wherein the decoding performance test request carries equipment information of the terminal, and the equipment information comprises a display card model and a display card driver version;

and receiving a test video stream which is transmitted by the server and is encoded by adopting the first coding and decoding standard, wherein the test video stream is transmitted by the server after determining that the blacklist does not comprise the equipment information of the terminal.

In one possible implementation, the method further includes:

and in the process of decoding and displaying the test video stream and the subsequent service video stream, an error notification is sent to the server in response to the detection of the blocking of a black screen or an image, wherein the error notification is used for instructing the server to add the equipment information of the terminal into the blacklist.

In one possible implementation manner, the obtaining, in response to the decoding performance information supporting a first codec standard, a test video stream encoded using the first codec standard includes:

Responding to the decoding performance information to support the use of a first coding and decoding standard under a target display parameter, and acquiring a test video stream which is coded by adopting the first coding and decoding standard and accords with the target display parameter, wherein the target display parameter comprises a target resolution and a target frame rate;

the setting the first codec standard as a codec standard adopted for sending a service video stream to the terminal includes:

and setting the first coding and decoding standard as a coding and decoding standard adopted by the business video stream for sending the target display parameters to the terminal.

In one possible implementation manner, the acquiring decoding performance information of the terminal includes:

inquiring basic decoding performance information of a terminal through a first inquiry interface of a system program, wherein the basic decoding performance information comprises coding and decoding standards supported by the terminal;

determining that the base decoding performance information supports the first codec standard;

and inquiring decoding performance information of the terminal through a second inquiry interface of the display card driver, wherein the decoding performance information comprises display parameters which can be supported by the terminal under various supported coding and decoding standards.

In a second aspect, there is provided a method of setting a codec standard, the method comprising:

receiving a decoding performance test request sent by a terminal;

transmitting a test video stream coded by a first coding and decoding standard to the terminal;

and in response to receiving a test passing notification sent by the terminal, setting the first coding and decoding standard as a coding and decoding standard adopted by sending the service video stream to the terminal.

In one possible implementation, the method further includes:

in response to receiving the unsupported notification sent by the terminal, the server sets a second codec standard to a codec standard adopted by sending the service video stream to the terminal;

and in response to receiving the test failure notification sent by the terminal, the server sets the second coding and decoding standard as the coding and decoding standard adopted by sending the service video stream to the terminal.

In a possible implementation manner, the decoding performance test request carries device information of the terminal, wherein the device information comprises a display card model and a display card driver version;

before the test video stream coded by the first coding and decoding standard is sent to the terminal, the method further comprises:

And determining that the blacklist does not comprise the equipment information of the terminal.

In one possible implementation, the method further includes:

receiving the error notification sent by the terminal to the server in response to the detection of the black screen or the image blocking in the process of decoding and displaying the test video stream and the service video stream;

and adding the equipment information of the terminal into the blacklist.

In a third aspect, there is provided an apparatus for setting a codec standard, the apparatus comprising:

the acquisition module is used for acquiring decoding performance information of the terminal;

the video stream acquisition module is used for responding to the decoding performance information to support a first coding and decoding standard and acquiring a test video stream coded by the first coding and decoding standard;

the decoding module is used for decoding the test video stream by adopting a hardware decoding mode to obtain a decoded video stream;

a determining module configured to determine a difference between a number of video frames in the test video stream and a number of video frames in the decoded video stream;

and the sending module is used for responding to the difference value to meet the test passing condition and sending a test passing notification to a server, wherein the test passing notification is used for indicating the server to set the first coding and decoding standard as the coding and decoding standard adopted by sending the business video stream to the terminal.

In one possible implementation, the sending module is further configured to:

In one possible implementation manner, the video stream acquisition module is configured to:

In one possible implementation, the apparatus further includes:

and the detection module is used for responding to detection of black screen or image blocking in the process of decoding and displaying the test video stream and the subsequent service video stream and sending an error notification to the server, wherein the error notification is used for indicating the server to add the equipment information of the terminal into the blacklist.

In one possible implementation manner, the acquiring module is configured to:

In a fourth aspect, there is provided an apparatus for setting a codec standard, the apparatus comprising:

the receiving module is used for receiving a decoding performance test request sent by the terminal;

the sending module is used for sending the test video stream coded by the first coding and decoding standard to the terminal;

and the setting module is used for setting the first coding and decoding standard to be the coding and decoding standard adopted by the service video stream sent to the terminal in response to receiving the test passing notification sent by the terminal.

In one possible implementation, the setting module is further configured to:

in response to receiving the unsupported notification sent by the terminal, setting a second coding and decoding standard as a coding and decoding standard adopted by sending the service video stream to the terminal;

and in response to receiving the test failure notification sent by the terminal, setting the second coding and decoding standard as the coding and decoding standard adopted by the service video stream sent to the terminal.

the apparatus further comprises:

and the determining module is used for determining equipment information which does not comprise the terminal in the blacklist.

In one possible implementation, the receiving module is further configured to:

the apparatus further comprises:

and the adding module is used for adding the equipment information of the terminal into the blacklist.

In a fifth aspect, a system for setting a codec standard is provided, the system including a server and a terminal; the terminal is configured to perform the method of the first aspect and possible implementations thereof; the server is adapted to perform the method of the second aspect and possible implementations thereof.

In a sixth aspect, a computer device is provided, the computer device comprising a processor and a memory, the memory having stored therein at least one instruction that is loaded and executed by the processor to implement the method of the first aspect, the second aspect and possible implementations thereof as described above.

In a seventh aspect, there is provided a computer readable storage medium having stored therein at least one instruction that is loaded and executed by the processor to implement a method as described above for the first aspect, the second aspect and possible implementations thereof.

In an eighth aspect, a computer program product is provided, the computer program product comprising computer program code which, when executed by a computer device, performs the method of the first aspect, the second aspect and possible implementations thereof.

The beneficial effects that technical scheme that this application embodiment provided include at least:

in the embodiment of the application, the terminal decodes the test video stream coded by the first coding and decoding standard to obtain a decoded video stream, determines a difference value between the number of video frames in the test video stream and the number of video frames in the decoded video stream, and further sets the coding and decoding standard adopted for sending the service video stream to the terminal based on the difference value. The difference value can reflect the condition that the terminal carries out hardware decoding on the test video stream, if the difference value is very low, the terminal can carry out high-quality decoding under the first coding and decoding standard, otherwise, if the difference value is relatively high, the terminal cannot carry out high-quality decoding under the first coding and decoding standard, the coding and decoding standard adopted for sending the service video stream to the terminal is set based on the difference value, the decoding stability can be better improved, and the playing quality of the video is improved.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

Fig. 1 is a schematic structural diagram of a codec standard setting system according to an embodiment of the present application;

fig. 2 is a schematic structural diagram of a terminal in a codec standard setting system according to an embodiment of the present application;

fig. 3 is a schematic structural diagram of a server in a codec standard setting system according to an embodiment of the present application;

FIG. 4 is a flowchart of a method for setting codec standards according to an embodiment of the present application;

fig. 5 is a schematic diagram of a frame loss according to an embodiment of the present application;

FIG. 6 is a flow chart of a method for decoding a performance query provided by an embodiment of the present application;

FIG. 7 is a flowchart of a method for setting codec standards according to an embodiment of the present application;

fig. 8 is a schematic diagram of an apparatus for setting a codec standard according to an embodiment of the present application;

Fig. 9 is a schematic diagram of an apparatus for setting a codec standard according to an embodiment of the present application;

fig. 10 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

Detailed Description

For the purpose of making the objects, technical solutions and advantages of the present application more apparent, the embodiments of the present application will be described in further detail below with reference to the accompanying drawings.

The embodiment of the application provides a method for setting a coding and decoding standard, which can be realized by a coding and decoding standard setting system. The codec standard setting system may include a terminal and a server, and the system architecture may be as shown in fig. 1. The coding and decoding standard setting system is used for detecting the decoding performance of the terminal and setting the coding and decoding standard adopted by the terminal in subsequent use.

The server may be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, basic cloud computing services such as big data and artificial intelligence platforms, and the like.

The terminal may be, but is not limited to, a smart phone, a tablet computer, a notebook computer, a desktop computer, a smart speaker, a smart watch, etc. The terminal is used for decoding the video stream and may also be used for communication with other devices, e.g. with a server, etc. The terminal is provided with an application program, which can be a video call type, a video player type, a screen sharing type or a cloud game type application program with the function of decoding and playing the network video stream.

The terminal and the server may be directly or indirectly connected through wired or wireless communication, which is not limited herein.

Cloud gaming (Cloud gaming), which may also be referred to as game on demand, is an online gaming technology based on Cloud computing technology. Cloud gaming technology enables lightweight devices (thin clients) with relatively limited graphics processing and data computing capabilities to run high quality games. In a cloud game scene, the game is not run in a player game terminal, but is run in a cloud server, the cloud server renders the game scene into a video and audio stream, and the video and audio stream is transmitted to the player game terminal through a network. The player game terminal does not need to have strong graphic operation and data processing capability, and only needs to have basic streaming media playing capability and the capability of acquiring player input instructions and sending the player input instructions to the cloud server.

From a hardware composition point of view, the structure of the terminal may include a processor 210, a memory 220, a graphic card 230, a display part 240, and a communication part 250 as shown in fig. 2.

The processor 210 may be a CPU or a system on chip (SoC) or the like, and the processor 210 may be configured to execute various instructions or the like involved in setting a codec standard.

The memory 220 may include various volatile memories or nonvolatile memories, such as Solid State Disk (SSD), dynamic random access memory (dynamic random access memory, DRAM) memory, and the like. The memory 220 may be used to store pre-stored data, intermediate data, and result data used in setting the codec standard, for example, to store the set codec standard.

The graphic card 230 may include a graphic processor (graphic processing unit, GPU), a memory, etc., and the graphic card 230 may be used for video encoding and decoding, etc.

The display part 230 may be a separate screen, or a screen, a projector, etc., which is integrated with the terminal body, and the screen may be a touch screen, or may be a non-touch screen, and the display part 230 is used to display video, an application program interface, etc. For example, the display part 230 may display an interface in an application program, decoded video, and the like.

The communication component 240 may be a wired network connector, a wireless fidelity (wireless fidelity, wiFi) module, a bluetooth module, a cellular network communication module, or the like. The communication means may be used for data transmission with other devices, which may be servers, other terminals, etc. For example, the terminal transmits a test passing notification to the server, and the terminal receives a test video stream transmitted from the server, and the like.

From a hardware composition perspective, the server may be structured as shown in fig. 3, including a processor 310, a memory 320, and a communication unit 330.

The processor 310 may be a CPU or a system on chip (SoC) or the like, and the processor 310 may be configured to execute various instructions or the like involved in the method.

The memory 320 may include various volatile memories or nonvolatile memories, such as Solid State Disk (SSD), dynamic random access memory (dynamic random access memory, DRAM) memory, and the like. The memory 120 may be used to store pre-stored data, intermediate data, and result data for various types of messages transmitted and received by the server. Such as testing the video stream, decoding device information and blacklists of the terminals in the performance test request, etc.

The communication component 330 may be a wired network connector, a wireless fidelity (wireless fidelity, wiFi) module, a bluetooth module, a cellular network communication module, or the like. The communication means may be used for data transmission with other devices, which may be servers, other terminals, etc. For example, the server receives a decoding performance test request sent by the terminal, and the server sends a test video stream or the like to the terminal.

In this embodiment, a detailed description of a scheme is performed by taking a desktop computer running a cloud game as an example, and other cases are similar to the detailed description of the scheme.

In the process of running the cloud game by the terminal, the server can render to obtain game images and send the game images to the terminal in a video stream mode. The terminal displays the received video stream, and the user can see the game picture on the terminal. Although the game screen is displayed in the form of video, the content is the same as the game screen locally rendered and displayed by the terminal, and various operation controls, windows and the like are included. In the course of displaying a game screen at the terminal, the user can perform various operations in the game screen. The application program can determine a corresponding operation instruction based on the position operated by the user and the position of the control, send the operation instruction to the server, execute the operation instruction by the server, and render the subsequent game screen based on the execution result.

In the process that the server sends the video stream to the terminal, the encoding and decoding standards of the video stream sent by the terminal need to be set in combination with related information. After the coding and decoding standard is set, the game picture of the terminal is coded based on the coding and decoding standard to obtain a video stream, the video stream is sent to the terminal, and the terminal can decode the video stream based on the decoding mode corresponding to the coding and decoding standard.

For the above application scenario, the embodiment of the present application provides a method for setting a codec standard, which is used for setting the codec standard of a terminal, and the corresponding process flow may be as shown in fig. 4, and includes the following steps:

the terminal acquires decoding performance information 401.

The decoding performance information may include at least one codec standard supported by the terminal and a highest display parameter that the terminal can support under various supported codec standards, where the highest display parameter includes a highest resolution and a highest frame rate.

When the user wants to play the game, the application program for starting the cloud game can be operated at the terminal. After the application program is started, the setting process of the terminal coding and decoding standard can be triggered. In an application, there are many methods for triggering the setting of the codec standard of the terminal, and several possible ways are given below:

In the first mode, each time the application is started, the process of setting the terminal codec standard is automatically triggered, and the execution of step 401 is started.

And in the second mode, a certain period (such as one week or one month) does not elapse, when an application program is started, the process of setting the coding and decoding standard of the terminal is automatically triggered, and after the coding and decoding standard is set, the setting process is not triggered in the current period.

In a third mode, an operation control is set in the game setting page, and the operation control is used for triggering the setting of the terminal coding and decoding standard. And responding to the operation of the user to enter the game setting page by the application program and clicking the operation control, and triggering the process of setting the terminal coding and decoding standard.

After triggering to set the encoding and decoding standards of the terminal, the application program can acquire the decoding performance information of the terminal. There are various implementation methods for the acquisition mode of the decoding performance information. For example, a device model may be selected at the terminal, and decoding performance information corresponding to the device model may be determined in a decoding performance information list stored in advance in a local or server based on the selected device model; the terminal may also be used to query via a query interface to determine decoding performance information, and the method for obtaining decoding performance information via the query interface will be described in detail later.

And 402, the terminal responds to the decoding performance information to support the first coding and decoding standard and sends a decoding performance test request to the server.

In the first case, when the server sets the codec standard for the terminal, only one codec standard (e.g., h.264) needs to be selected from the two codec standards, and one codec standard (e.g., h.265) can be smoothly used for the general terminal, so that only the other codec standard (e.g., h.265) needs to be tested.

For this case, after the terminal obtains the decoding performance information, it may determine whether the decoding performance information supports the codec standard (i.e. the first codec standard) to be tested, if so, it may further generate a decoding performance test request and send the test request to the server to trigger a test procedure for the first codec standard, and if not, it may send a non-support notification to the server, where the non-support notification message is used to instruct the server that the terminal does not support the first codec standard, so as to trigger the server to set the second codec standard as the codec standard adopted by sending the service video stream to the terminal, and the second codec standard is the codec standard that half of the terminals can be used smoothly.

In the second case, the server has a plurality of alternative codec standards when setting the codec standard for the terminal, and there are a plurality of codec standards to be tested.

For this case, after acquiring the decoding performance information, the terminal may separately determine whether the decoding performance information supports the codec standard for each of the alternative codec standards. A codec standard (which may include one or more codec standards) supported by the decoding performance information is determined as a first codec standard that needs to be tested. The decoding performance test request carrying all the first codec standards may be generated, or the decoding performance test request carrying the first codec standards may be generated separately for each first codec standard. A decoding performance test request is then sent to the server to trigger a test procedure for each of the first codec standards.

In the above two cases, the decoding performance test request may also carry a device identifier of the terminal, where the device identifier may be a physical address, etc., and may also carry an account number of the locally logged-in application program, and may also carry a display parameter to be tested. The display parameters may include a target resolution and a target frame rate. The display parameter acquisition mode is as follows: one way, the display parameter may be a display parameter currently used by the application; in another mode, the operation control can be correspondingly provided with a display parameter selection field, a user can select the display parameter to be tested in the display parameter selection field, and then click the operation control, so that the terminal can acquire the display parameter selected by the user in the display parameter selection field as the display parameter.

After generating the decoding performance test request, the terminal sends the decoding performance test request to the server.

403, the server sends the test video stream encoded using the first codec standard to the terminal.

The codec standard is a standard of a codec operation mode to be followed when encoding and decoding video. Common video codec standards are h.261, h.263, h.264 and h.265, with h.264 and h.265 being the most common codec standards in the current cloud gaming field. In the embodiment of the present application, the first codec standard is h.265 and the subsequent second codec standard is h.264.

The server can encode a section of video by using different coding and decoding standards in advance, so as to obtain test video streams corresponding to various coding and decoding standards, and the test video streams are stored in a database. After receiving the decoding performance test request sent by any terminal, the server can determine a first coding and decoding standard for testing the terminal, select a test video stream corresponding to the first coding and decoding standard, and send the test video stream to the terminal.

Regarding the manner in which the first codec standard is determined: for the first case, the decoding performance test request may not carry the codec standard, and the server pre-stores the first codec standard as the codec standard to be tested; for the second case, the decoding performance test request carries the first codec standard, and the server may parse the decoding performance test request to determine the first codec standard.

Optionally, display parameters (resolution, frame rate, etc.) may be considered in the test process, and accordingly, the server may first obtain a section of video corresponding to each display parameter, and then encode each section of video using different encoding and decoding standards, so as to obtain a test video stream corresponding to multiple encoding and decoding standards under multiple display parameters. For example, 4 display parameters, 3 codec standards may yield 12 test video streams. After receiving the decoding performance test request sent by any terminal, the server can determine the target display parameters besides determining the first coding and decoding standard for testing the terminal. And selecting a test video stream corresponding to the first coding and decoding standard and the target display parameter, and sending the test video stream to the terminal. For the way in which the first codec standard is determined, see above.

Regarding the manner in which the target display parameters are determined: for the situation that the decoding performance test request carries the display parameters, the server acquires the display parameters carried in the decoding performance test request as target display parameters to be tested; for the case that the decoding performance test request does not carry the display parameters, the server may determine all possible display parameters as target display parameters to be tested, or determine the highest display parameters (highest resolution and highest frame rate) as target display parameters to be tested.

In the above-described process, the server may transmit the test video stream frame by frame. If the server determines a plurality of test video streams, the test video streams may be sequentially transmitted to the terminal, and a certain period of time may be appropriately set between two adjacent test video streams, or the test video streams may be transmitted together to the terminal (in this case, not transmitted frame by frame).

And 404, the terminal decodes the test video stream by adopting a hardware decoding mode to obtain a decoded video stream.

And the terminal receives the test video stream sent by the server frame by frame, and decodes and displays the video frame when receiving one video frame. If the server is not the test video sent frame by frame, the terminal can firstly receive the test video stream and then decode and display the test video stream. The processing of the test video stream by the terminal may be the same as the processing of the normal service video stream.

The terminal determines 405 a difference between the number of video frames in the test video stream and the number of video frames in the decoded video stream.

The number of video frames in the test video stream is acquired in a plurality of modes, namely, the first mode is transmitted together when the server transmits the test video stream; and in a second mode, when the terminal receives the test video stream, determining the number of video frames contained in the test video stream. The number of video frames in the decoded video stream may be read when the decoded video stream is output.

The reason why the frames of the decoded video stream and the test video stream may not be identical is that: when a terminal decodes a video stream, the time required for decoding is different because the information contained in each video frame is different. If a certain video frame is decoded, the decoding time exceeds the time that the previous video frame is displayed on the display unit, which causes a delay in video pictures. When a certain video frame is decoded longer, if all subsequent video frames are still decoded, the delay of the output video frame is possibly increased, so when the time for decoding the video frame exceeds a certain threshold value, the delay of the output video frame needs to be reduced by frame loss, namely one or more video frames behind the video frame are skipped when decoding, the delay generated by decoding is reduced, the frame loss operation can lead to that the output video frame is smaller than the input video frame, and a schematic diagram of the frame loss operation is shown in fig. 5.

And 406, the terminal sends a test passing notification to the server in response to the difference value meeting the test passing condition, and sends a test failure notification to the server in response to the difference value not meeting the test passing condition.

Wherein the test passing condition is that the difference obtained in step 405 is less than a threshold, which may be manually set, which may generally be referred to the number of video frames in the test video stream.

In response to the difference value obtained by step 405 being less than the threshold, the terminal sends a test passing notification to the server. In response to the difference value obtained by step 405 being greater than the threshold, the terminal sends a test failure notification to the server.

Alternatively, the test passing condition may be that the ratio of the difference value to the number of video frames in the test video stream is less than a certain threshold, which may be set manually.

And 407, the server responds to the notification sent by the terminal after the terminal completes the test, and sets the coding and decoding standard adopted by the service video stream sent to the terminal.

The server responds to the test passing notification sent by the terminal, and the server sets the first coding and decoding standard as the coding and decoding standard adopted by the service video stream sent to the terminal. For example, after receiving a notification that the test sent by the terminal passes, the server uses h.265 as a codec standard adopted by a service video stream that is subsequently sent to the terminal.

And the server responds to the test failure notification sent by the terminal, and sets the second coding and decoding standard as the coding and decoding standard adopted by the service video stream sent to the terminal. For example, after receiving a test failure notification sent by the terminal, the server uses the h.264 as a coding and decoding standard adopted by a service video stream subsequently sent to the terminal.

The processing flow of the method for decoding a performance query provided in the embodiment of the present application may be as shown in fig. 6, and includes the following steps:

and 601, inquiring basic decoding performance information of the terminal through a first inquiry interface of the system program.

The basic decoding performance information comprises a coding and decoding standard supported by the terminal.

The terminal can obtain basic decoding performance information through the first query interface. The first query interface is an interface in a system program, such as a video hardware acceleration (directX video acceleration, DXVA) interface.

It is determined 602 that the base decoding performance information supports a first codec standard.

Based on the base decoding performance information and the first codec standard, it is determined whether the codec standard supported in the base decoding performance information contains the first codec standard. If so, the process of step 603 may continue, and if not, a non-support notification may be sent to the server, where the non-support notification message is used to indicate to the server that the terminal does not support the first codec standard, to trigger the server to set the second codec standard to the codec standard employed to send the service video stream to the terminal.

603, querying decoding performance information of the terminal through a second query interface of the display card driver.

The decoding performance information comprises display parameters which can be supported by the terminal under various supported coding and decoding standards.

The terminal can obtain the display parameters through the second query interface. The second query interface is an interface in the graphics card driver, such as a quick video synchronization (QSV) interface and a compute unified device architecture (compute unified device architecture, CUDA) interface.

For the above application scenario, the embodiment of the present application further provides a method for setting a codec standard based on a blacklist, and the corresponding process flow may be as shown in fig. 7, including the following steps:

701, the terminal acquires decoding performance information.

The processing method of step 701 is the same as that of step 401, and reference may be made to the related description of step 401, which is not repeated here.

And 702, the terminal responds to the decoding performance information to support the first coding and decoding standard and sends a decoding performance test request to the server.

The decoding performance test request carries device information of the terminal, wherein the device information comprises a display card model and a display card driver version, and the device information can also comprise a central processing unit model, power supply power and the like of the device.

The processing method of step 702 is the same as that of step 402, and reference may be made to the description of step 402, which is not repeated here.

703, based on the decoding performance test request and the blacklist, the server determines whether the device information of the terminal is in the blacklist.

The display card model and the display card driver version basically determine the decoding capability of the terminal for hardware decoding, and the decoding capability of the terminal with the same display card model and display card driver version can be considered to be the same. The server may pre-establish a blacklist for recording which terminals with which device information may have serious errors when using certain codec standards. The blacklist may be initially empty, and device information may be gradually added to the blacklist based on the processing of a large number of terminals during the decoding and playing of the test video stream and the service video stream. Alternatively, some device information may be added by the technician in the blacklist. The blacklist may be established for one codec standard (this way mainly for the first case mentioned at step 402) or may be established for a plurality of codec standards, respectively (this way mainly for the second case mentioned at step 402).

The server acquires the device information of the terminal carried in the decoding performance test request, compares the device information of the terminal with the device information in the blacklist, and determines whether the blacklist contains the device information of the terminal. For example, the device information of the terminal includes a graphic card type a and a graphic card driver version 1.1. And the blacklist also contains the equipment information of which the display card model is display card A and the display card driver version is 1.1, and the blacklist is determined to contain the equipment information of the terminal.

And 704, if the device information of the terminal is in the blacklist, the server sets the second coding and decoding standard as the coding and decoding standard adopted by the service video stream sent to the terminal.

And 705, if the device information of the terminal is not in the blacklist, the server transmits the test video stream coded by the first coding and decoding standard to the terminal.

The processing method of step 705 is the same as that of step 403, and reference may be made to the related description of step 403, which is not repeated here.

And 706, if the black screen or the image is detected to be stuck in the process of decoding and displaying the test video stream, the terminal sends an error notification to the server, and the server adds the device information of the terminal to the blacklist.

Wherein the error notification is used to instruct the server to add the device information of the terminal to the blacklist. The device information of the terminal is extracted by the server in the error notification.

The terminal can determine whether a black screen appears by reading the state parameters of the display part, and can determine that the image is jammed when a new video frame is not output for a certain time while decoding the video stream. Optionally, other abnormal states such as flash back and the like can be detected simultaneously when the software runs. If a black screen, image sticking or other abnormal state occurs, an abnormal notification can be sent to the server, and the abnormal notification message is used for indicating that the terminal can have serious faults when decoding under the first coding and decoding standard. After the server receives the anomaly notification message, the device information of the terminal can be added into the blacklist, and the server can set the second coding and decoding standard as the coding and decoding standard adopted by sending the service video stream to the terminal.

707, in the process of decoding and displaying the test video stream, if the black screen or the image is not detected to be stuck, determining a difference value between the number of video frames in the test video stream and the number of video frames in the decoded video stream.

Step 707 is similar to the processing method of step 405, and reference may be made to the relevant description of step 405, which is not repeated here.

And 708, the terminal sends a test passing notification to the server in response to the difference meeting the test passing condition. And when the difference value does not meet the test passing condition, the terminal sends a test failure notification to the server.

The processing method of step 708 is the same as that of step 406, and reference may be made to the related description of step 406, which is not repeated here.

709, the server responds to the notification sent by the terminal after the terminal completes the test, and sets the coding and decoding standard adopted by the service video stream sent to the terminal.

The processing method of step 609 is the same as that of step 407, and reference may be made to the related description of step 407, which is not repeated here.

The server transmits 710 the service video stream encoded using the set codec standard to the terminal.

If the set codec standard is the first codec standard, the server sends a service video stream with a target display parameter under the first codec standard to the terminal, for example, the set codec standard is h.265, and the service video stream sent by the server is a game image with a target resolution of 1920×1080 and a target frame rate of 90 hz under the h.265 codec standard.

And 711, if the terminal detects that the black screen or the image is stuck in the process of decoding and displaying the service video stream, sending an error notification to the server, and adding the device information of the terminal into the blacklist by the server.

The manner in which the black screen and image sticking are detected can be seen in the relevant description of step 706. After detecting that the black screen or the image is stuck, the terminal can send an error notification carrying the equipment information to the server.

After receiving the error notification, the server establishes only a blacklist for the first codec standard for the first case, and may add the device information of the terminal to the blacklist. In addition, the server may set the second codec standard as a codec standard adopted by the service video stream transmitted to the terminal.

For the second case, the server may establish a blacklist corresponding to a plurality of coding and decoding standards, and at this time, the current coding and decoding standard of the terminal may be determined, and the device information of the terminal may be added to the blacklist corresponding to the coding and decoding standard. In addition, the server may also re-set the codec standard for the terminal.

In the above embodiment, the server stores the test video stream, and transmits the test video stream to the terminal when the terminal requests the test video stream. In the embodiment of the application, the terminal can also store the test video stream, and when the test is needed, the terminal can acquire the locally stored test video stream to perform the decoding test. The specific processing procedure is similar to the solution of storing the test video stream in the server, and the embodiments of the present application are not repeated.

Any combination of the above-mentioned optional solutions may be adopted to form an optional embodiment of the present disclosure, which is not described herein in detail.

Based on the same technical concept, the embodiment of the present application further provides an apparatus for setting a codec standard, as shown in fig. 8, where the apparatus includes:

an acquiring module 810, configured to acquire decoding performance information of a terminal;

A video stream acquisition module 820 for acquiring a test video stream encoded using a first codec standard in response to the decoding performance information supporting the first codec standard;

the decoding module 830 is configured to decode the test video stream by using a hardware decoding mode to obtain a decoded video stream;

a determining module 840 for determining a difference between the number of video frames in the test video stream and the number of video frames in the decoded video stream;

and a sending module 850, configured to send a test passing notification to the server in response to the difference value meeting the test passing condition, where the test passing notification is used to instruct the server to set the first codec standard to a codec standard adopted by sending the service video stream to the terminal.

In one possible implementation, the sending module 850 is further configured to:

transmitting a non-support notification to the server in response to the decoding performance information not supporting the first codec standard, wherein the non-support notification is used for instructing the server to set the second codec standard to the codec standard employed for transmitting the service video stream to the terminal;

and sending a test failure notification to the server in response to the difference value not meeting the test passing condition, wherein the test failure notification is used for indicating the server to set the second coding and decoding standard as the coding and decoding standard adopted by sending the business video stream to the terminal.

In one possible implementation, the video stream acquisition module 820 is configured to:

transmitting a decoding performance test request to a server, wherein the decoding performance test request carries equipment information of a terminal, and the equipment information comprises a display card model and a display card driver version;

and receiving a test video stream which is transmitted by the server and is encoded by adopting the first coding and decoding standard, wherein the test video stream is transmitted by the server after the equipment information of the terminal is not included in the blacklist is determined.

In one possible implementation, the apparatus further includes:

and the detection module 860 is configured to send an error notification to the server in response to detecting that the black screen or the image is stuck in decoding and displaying the test video stream and the subsequent service video stream, where the error notification is used to instruct the server to add the device information of the terminal to the blacklist.

Setting the first coding and decoding standard as the coding and decoding standard adopted by sending the service video stream to the terminal, wherein the coding and decoding standard comprises the following steps:

the first coding and decoding standard is set as the coding and decoding standard adopted by the business video stream for sending the target display parameters to the terminal.

In one possible implementation, the obtaining module 810 is configured to:

inquiring basic decoding performance information of the terminal through a first inquiry interface of the system program, wherein the basic decoding performance information comprises coding and decoding standards supported by the terminal;

determining that the base decoding performance information supports a first codec standard;

Based on the same technical concept, the embodiment of the present application further provides an apparatus for setting a codec standard, as shown in fig. 9, where the apparatus includes:

a receiving module 910, configured to receive a decoding performance test request sent by a terminal;

a sending module 920, configured to send a test video stream encoded using a first codec standard to a terminal;

a setting module 930, configured to set the first codec standard to a codec standard adopted for sending the service video stream to the terminal in response to receiving the test passing notification sent by the terminal.

In one possible implementation, the setting module 930 is further configured to:

in response to receiving the unsupported notification sent by the terminal, the server sets the second codec standard to a codec standard adopted by sending the service video stream to the terminal;

and in response to receiving the test failure notification sent by the terminal, the server sets the second coding and decoding standard as the coding and decoding standard adopted by the service video stream sent to the terminal.

In one possible implementation manner, the decoding performance test request carries device information of the terminal, wherein the device information comprises a display card model and a display card driver version;

the apparatus further comprises:

a determining module 940, configured to determine device information that does not include the terminal in the blacklist.

In one possible implementation, the receiving module 910 is further configured to:

the receiving terminal responds to the detection of the black screen or the image blocking in the process of decoding and displaying the test video stream and the service video stream to send an error notification to the server;

the apparatus further comprises:

an adding module 950, configured to add device information of the terminal to the blacklist.

It should be noted that: the apparatus for setting the codec standard provided in the above embodiment is only exemplified by the division of the above functional modules when setting the codec standard, and in practical application, the above functional allocation may be performed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules, so as to perform all or part of the functions described above. In addition, the device for setting the codec standard provided in the above embodiment belongs to the same concept as the method embodiment for setting the codec standard, and the specific implementation process is detailed in the method embodiment, which is not repeated here.

Fig. 10 shows a block diagram of an electronic device 1000 according to an embodiment of the present application. The electronic device may be each terminal in the above-described embodiments. The electronic device 1000 may be a portable mobile terminal such as: a smart phone, a tablet computer, an MP3 player (Moving Picture Experts Group Audio Layer III, motion picture expert compression standard audio plane 3), an MP4 (Moving Picture Experts Group Audio Layer IV, motion picture expert compression standard audio plane 4) player, a notebook computer, or a desktop computer. Electronic device 1000 may also be referred to by other names of user devices, portable terminals, laptop terminals, desktop terminals, and the like.

Generally, the electronic device 1000 includes: a processor 1001 and a memory 1002.

The processor 1001 may include one or more processing cores, such as a 4-core processor, an 8-core processor, and so on. The processor 1001 may be implemented in at least one hardware form of DSP (Digital Signal Processing ), FPGA (Field-Programmable Gate Array, field programmable gate array), PLA (Programmable Logic Array ). The processor 1001 may also include a main processor, which is a processor for processing data in an awake state, also referred to as a CPU (Central Processing Unit ), and a coprocessor; a coprocessor is a low-power processor for processing data in a standby state. In some embodiments, the processor 1001 may be integrated with a GPU (Graphics Processing Unit, image processor) for taking care of rendering and drawing of content that the display screen needs to display. In some embodiments, the processor 1001 may also include an AI (Artificial Intelligence ) processor for processing computing operations related to machine learning.

Memory 1002 may include one or more computer-readable storage media, which may be non-transitory. Memory 1002 may also include high-speed random access memory, as well as non-volatile memory, such as one or more magnetic disk storage devices, flash memory storage devices. In some embodiments, a non-transitory computer readable storage medium in memory 1002 is used to store at least one instruction for execution by processor 1001 to implement the methods provided by embodiments of the present application.

In some embodiments, the electronic device 1000 may further optionally include: a peripheral interface 1003, and at least one peripheral. The processor 1001, the memory 1002, and the peripheral interface 1003 may be connected by a bus or signal line. The various peripheral devices may be connected to the peripheral device interface 1003 via a bus, signal wire, or circuit board. Specifically, the peripheral device includes: at least one of radio frequency circuitry 1004, a display 1005, a camera assembly 1006, audio circuitry 1007, a positioning assembly 1008, and a power supply 1009.

Peripheral interface 1003 may be used to connect I/O (Input/Output) related at least one peripheral to processor 1001 and memory 1002. In some embodiments, processor 1001, memory 1002, and peripheral interface 1003 are integrated on the same chip or circuit board; in some other embodiments, either or both of the processor 1001, memory 1002, and peripheral interface 1003 may be implemented on a separate chip or circuit board, which is not limited in this embodiment.

Radio Frequency circuit 1004 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. Radio frequency circuitry 1004 communicates with a communication network and other communication devices via electromagnetic signals. The radio frequency circuit 1004 converts an electrical signal into an electromagnetic signal for transmission, or converts a received electromagnetic signal into an electrical signal. Optionally, the radio frequency circuit 1004 includes: antenna systems, RF transceivers, one or more amplifiers, tuners, oscillators, digital signal processors, codec chipsets, subscriber identity module cards, and so forth. Radio frequency circuitry 1004 may communicate with other terminals via at least one wireless communication protocol. The wireless communication protocol includes, but is not limited to: the world wide web, metropolitan area networks, intranets, generation mobile communication networks (2G, 3G, 4G, and 5G), wireless local area networks, and/or WiFi (Wireless Fidelity ) networks. In some embodiments, the radio frequency circuitry 1004 may also include NFC (Near Field Communication ) related circuitry, which is not limited in this application.

The display screen 1005 is used to display a UI (User Interface). The UI may include graphics, text, icons, video, and any combination thereof. When the display 1005 is a touch screen, the display 1005 also has the ability to capture touch signals at or above the surface of the display 1005. The touch signal may be input to the processor 1001 as a control signal for processing. At this time, the display 1005 may also be used to provide virtual buttons and/or virtual keyboards, also referred to as soft buttons and/or soft keyboards. In some embodiments, the display 1005 may be one, disposed on the front panel of the electronic device 1000; in other embodiments, the display 1005 may be at least two, respectively disposed on different surfaces of the electronic device 1000 or in a folded design; in other embodiments, the display 1005 may be a flexible display disposed on a curved surface or a folded surface of the electronic device 1000. Even more, the display 1005 may be arranged in a non-rectangular irregular pattern, i.e., a shaped screen. The display 1005 may be made of LCD (Liquid Crystal Display ), OLED (Organic Light-Emitting Diode) or other materials.

The camera assembly 1006 is used to capture images or video. Optionally, camera assembly 1006 includes a front camera and a rear camera. Typically, the front camera is disposed on the front panel of the terminal and the rear camera is disposed on the rear surface of the terminal. In some embodiments, the at least two rear cameras are any one of a main camera, a depth camera, a wide-angle camera and a tele camera, so as to realize that the main camera and the depth camera are fused to realize a background blurring function, and the main camera and the wide-angle camera are fused to realize a panoramic shooting and Virtual Reality (VR) shooting function or other fusion shooting functions. In some embodiments, camera assembly 1006 may also include a flash. The flash lamp can be a single-color temperature flash lamp or a double-color temperature flash lamp. The dual-color temperature flash lamp refers to a combination of a warm light flash lamp and a cold light flash lamp, and can be used for light compensation under different color temperatures.

The audio circuit 1007 may include a microphone and a speaker. The microphone is used for collecting sound waves of users and environments, converting the sound waves into electric signals, and inputting the electric signals to the processor 1001 for processing, or inputting the electric signals to the radio frequency circuit 1004 for voice communication. For purposes of stereo acquisition or noise reduction, the microphone may be multiple and separately disposed at different locations of the electronic device 1000. The microphone may also be an array microphone or an omni-directional pickup microphone. The speaker is used to convert electrical signals from the processor 1001 or the radio frequency circuit 1004 into sound waves. The speaker may be a conventional thin film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, not only the electric signal can be converted into a sound wave audible to humans, but also the electric signal can be converted into a sound wave inaudible to humans for ranging and other purposes. In some embodiments, audio circuit 1007 may also include a headphone jack.

The location component 1008 is used to locate a current geographic location of the electronic device 1000 to enable navigation or LBS (Location Based Service, location-based services). The positioning component 1008 may be a GPS (Global Positioning System ) or beidou system based positioning component.

The power supply 1009 is used to power the various components in the electronic device 1000. The power source 1009 may be alternating current, direct current, disposable battery or rechargeable battery. When the power source 1009 includes a rechargeable battery, the rechargeable battery may be a wired rechargeable battery or a wireless rechargeable battery. The wired rechargeable battery is a battery charged through a wired line, and the wireless rechargeable battery is a battery charged through a wireless coil. The rechargeable battery may also be used to support fast charge technology.

Those skilled in the art will appreciate that the structure shown in fig. 10 is not limiting of the electronic device 1000 and may include more or fewer components than shown, or may combine certain components, or may employ a different arrangement of components.

It should be noted that, the information (including but not limited to user equipment information, user personal information, etc.), data (including but not limited to data for analysis, stored data, presented data, etc.), and signals (including but not limited to signals transmitted between the user terminal and other devices, etc.) referred to in this application are all authorized by the user or are fully authorized by the parties, and the collection, use, and processing of relevant data is required to comply with relevant laws and regulations and standards of relevant countries and regions.

It will be appreciated by those of ordinary skill in the art that all or part of the steps of implementing the above embodiments may be implemented by hardware, or may be implemented by a program to instruct related hardware, and the program may be stored in a computer readable storage medium, where the storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.

The foregoing is only a partial example of the possible embodiments of the present application and is not intended to limit the present application, and any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application are intended to be included in the scope of the present application.

Claims

1. A method of setting a codec standard, the method comprising:

acquiring decoding performance information of a terminal;

2. The method according to claim 1, wherein the method further comprises:

3. The method of claim 1, wherein the obtaining the test video stream encoded using the first codec standard comprises:

4. A method according to claim 3, characterized in that the method further comprises:

5. The method of claim 1, wherein the obtaining the test video stream encoded using the first codec standard in response to the decoding performance information supporting the first codec standard comprises:

6. The method according to claim 5, wherein the acquiring decoding performance information of the terminal includes:

7. A method of setting a codec standard, the method comprising:

receiving a decoding performance test request sent by a terminal;

8. The method of claim 7, wherein the method further comprises:

9. The method according to claim 7, wherein the decoding performance test request carries device information of the terminal, and the device information includes a graphics card model and a graphics card driver version;

10. The method according to claim 9, wherein the method further comprises:

and adding the equipment information of the terminal into the blacklist.

11. An apparatus for setting a codec standard, the apparatus comprising:

12. An apparatus for setting a codec standard, the apparatus comprising:

13. A computer device comprising a memory and a processor, the memory for storing computer instructions; the processor executes the computer instructions stored in the memory to cause the computer device to perform the method of any one of the preceding claims 1 to 10.

14. A computer readable storage medium, characterized in that the computer readable storage medium stores computer program code which, in response to being executed by a computer device, performs the method of any of the preceding claims 1 to 10.