CN110740313A

CN110740313A - Hardware coding capability detection method and device

Info

Publication number: CN110740313A
Application number: CN201911025139.0A
Authority: CN
Inventors: 郭靖; 刘杉; 王诗涛; 刘海军; 金飞剑; 张清
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2019-10-25
Filing date: 2019-10-25
Publication date: 2020-01-31
Anticipated expiration: 2039-10-25
Also published as: CN110740313B

Abstract

The invention provides hardware coding capacity detection methods, devices, electronic equipment and storage media, wherein the method comprises the steps of receiving a reference code stream which is issued by a server and accords with a video standard to be tested, carrying out software decoding on the received reference code stream, coding a decoded image obtained through the software decoding through a hardware encoder, reporting the coded code stream obtained through the hardware encoder to the server so that the server carries out code control detection and similarity detection on the coded code stream, and determining the hardware coding capacity of the equipment type of equipment which runs a playing client and aims at the video standard to be tested according to a corresponding detection result.

Description

Hardware coding capability detection method and device

Technical Field

The present invention relates to multimedia technology, and in particular, to a method and an apparatus for detecting types of hardware encoding capabilities, an electronic device, and a storage medium.

Background

In recent years, with the development of internet technology and mobile terminal technology, users can watch videos or make video calls on mobile terminals at any time.

Aiming at the fragmentation of systems provided by various manufacturers at present, before formally using a hardware encoder of equipment to encode collected video data, the hardware encoding compatibility of the equipment needs to be verified, whether the volatility of hard encoding time control meets requirements, whether a hard encoding code stream meets corresponding standards, whether the hard encoding code stream possibly generates screen splash, screen greenness, decoding failure and the like when the hard encoding code stream is decoded.

However, the related art lacks an effective scheme for detecting the hardware encoding capability possessed by the device.

Disclosure of Invention

The embodiment of the invention provides hardware coding capacity detection methods and devices, electronic equipment and a storage medium, which can determine the hardware coding capacity of the equipment.

The technical scheme of the embodiment of the invention is realized as follows:

the embodiment of the invention provides an hardware coding capacity detection method, which comprises the following steps:

receiving a reference code stream which is issued by a server and meets the standard of the video to be tested through a playing client;

carrying out software decoding on the received reference code stream, and encoding a decoded image obtained by the software decoding through a hardware encoder;

reporting the code stream coded by the hardware coder to a server so as to ensure that

And the server performs code control detection and similarity detection on the coded code stream, and determines the hardware coding capacity of the equipment type of the equipment operating the playing client side aiming at the video standard to be tested according to the corresponding detection result.

The embodiment of the invention provides a method for detecting coding capacity of hardware, which comprises the following steps:

sending a reference code stream meeting a to-be-tested video standard to a playing client;

receiving a coding code stream obtained by coding through a hardware coder and reported by the playing client;

and carrying out code control detection and similarity detection on the received coding code stream, and determining the hardware coding capacity of the equipment type of the equipment operating the playing client side aiming at the video standard to be tested according to the corresponding detection result.

The embodiment of the invention provides hardware coding capability detection devices, including:

the receiving module is used for receiving a reference code stream which is issued by the server and accords with the video standard to be tested through the playing client;

the decoding module is used for carrying out software decoding on the reference code stream received by the receiving module;

the coding module is used for coding a decoded image obtained by the decoding module through software decoding through a hardware coder;

a reporting module for reporting the code stream obtained by the coding module to a server so as to ensure that the code stream is encoded by the coding module

And the server performs code control detection and similarity detection on the coded code stream, and determines the hardware coding capacity of the equipment type of the equipment operating the playing client aiming at the video standard to be tested according to the corresponding detection result.

In the above scheme, the reporting module is further configured to report, to a server, an encoded code stream composed of encoding results corresponding to each frames in the reference code stream, so that the server performs an operation of determining that a type of a device to which the playing client belongs has a hardware encoding capability compatible with the video standard to be tested when it is detected that code rate volatility of the encoded code stream is smaller than a volatility threshold and similarity between a decoded image of the encoded code stream and a decoded image of the reference code stream is greater than a similarity threshold.

In the above scheme, the reporting module is further configured to report, to a server, an encoded code stream composed of encoding results corresponding to each frames in the reference code stream and an average frame number of frames of an image to be encoded that is accumulated in the hardware encoder buffer during an encoding process, so that the server performs the following operations:

and when determining that the similarity between the decoded image of the coding code stream and the decoded image of the reference code stream is greater than a similarity threshold, the code rate fluctuation of the coding code stream is less than a fluctuation threshold, and the average frame number is less than a frame number threshold, determining that the type of the equipment to which the equipment operating the playing client belongs has the hardware coding capability compatible with the video standard to be tested.

In the foregoing scheme, the reporting module is further configured to report the encoded code stream to a server, so that the server performs the following operations:

when abnormal information is generated in the process of decoding the coded code stream, recording coding error information in a report;

when the frame number of the coding code stream is determined to be smaller than the frame number of the reference code stream, recording coding error information in the report;

the coding error information is used for indicating that the type of the equipment which runs the playing client does not have the hardware coding capability compatible with the video standard to be tested.

In the foregoing solution, the reporting module is further configured to report, when the playing client generates abnormal information in a process of encoding a decoded image of the reference code stream, the abnormal information to the server, so that the server reports the abnormal information to the playing client, so that the playing client can encode the decoded image of the reference code stream

And the server determines that the type of the equipment which runs the playing client does not have the hardware coding capability compatible with the video standard to be tested.

In the above scheme, the reporting module is further configured to report, to a server, an encoded code stream generated by a playing client in multiple devices of the same type for the reference code stream and an average stocked frame number in an encoding process, so that the server performs the following operations:

and when the number of the devices with the hardware coding capability compatible with the video standard to be tested is determined to be larger than a preset threshold value, determining that the devices of the same type have the hardware coding capability compatible with the video standard to be tested.

The embodiment of the present invention provides another hardware coding capability detection apparatus, including:

the sending module is used for sending the reference code stream meeting the video standard to be tested to the playing client;

the receiving module is used for receiving a coding code stream which is obtained by coding through a hardware coder and reported by the playing client;

and the detection module is used for carrying out code control detection and similarity detection on the coded code stream received by the receiving module, and determining the hardware coding capacity of the equipment type to which the equipment operating the playing client belongs, aiming at the video standard to be tested, according to the corresponding detection result.

In the above scheme, the detection module is further configured to determine that a device type to which a device operating the playing client belongs has a hardware encoding capability compatible with the video standard to be tested when it is detected that the similarity between the decoded image of the encoded code stream and the decoded image of the reference code stream is greater than a similarity threshold and the code rate volatility of the encoded code stream is less than a volatility threshold.

In the above scheme, the receiving module is further configured to receive an average frame number of images to be encoded that are accumulated in a hardware encoder cache area in an encoding process and reported by the playing client, so that the detecting module determines that a device type to which a device operating the playing client belongs has a hardware encoding capability compatible with the video standard to be encoded when detecting that the average frame number is less than a frame number threshold, a similarity between a decoded image of the encoded code stream and a decoded image of the reference code stream is greater than a similarity threshold, and a code stream volatility of the encoded code stream is less than a volatility threshold.

In the above solution, the detecting module is further configured to, when detecting that abnormal information is generated in the process of decoding the encoded code stream, or,

and when the frame number of the coding code stream is smaller than that of the reference code stream, determining that the equipment type of the equipment running the playing client side does not have the hardware coding capability compatible with the video standard to be tested.

In the above scheme, the detection module is further configured to determine that a device type to which a device running the playing client belongs does not have a hardware encoding capability compatible with the video standard to be tested when detecting abnormal information reported by the playing client in a process of encoding a decoded image of the received reference code stream.

An embodiment of the present invention provides electronic devices, including:

a memory for storing executable instructions;

and the processor is used for implementing any hardware coding capability detection methods provided by the embodiment of the invention when executing the executable instructions stored in the memory.

The embodiment of the present invention provides storage media, which store executable instructions for causing a processor to execute the method for detecting any hardware coding capability provided in the embodiment of the present invention.

The embodiment of the invention has the following beneficial effects:

the server sends a reference code stream meeting the video standard to be tested to a playing client of a user, the playing client performs soft decoding and hard coding, and reports the coded code stream to the server for further detection in step .

Drawings

FIG. 1 is an alternative architecture diagram of a hardware coding capability detection system provided by an embodiment of the present invention;

fig. 2A is an optional structural schematic diagram of a hardware coding capability detection device provided in an embodiment of the present invention;

fig. 2B is a schematic structural diagram of optional structures of the hardware coding capability detection device according to the embodiment of the present invention;

FIG. 3 is a flow chart illustrating alternatives of a hardware coding capability detection method according to an embodiment of the present invention;

FIG. 4 is a flow chart illustrating alternatives of a hardware coding capability detection method according to an embodiment of the present invention;

FIG. 5 is a flow chart illustrating alternatives of a hardware coding capability detection method according to an embodiment of the present invention;

FIG. 6 is a flow chart illustrating alternatives of a hardware coding capability detection method according to an embodiment of the present invention;

fig. 7 is a flow chart illustrating alternatives of the hardware coding capability detection method according to the embodiment of the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention clearer, the present invention will be described in further with reference to the attached drawings, the described embodiments should not be construed as limiting the present invention, and all other embodiments obtained by those skilled in the art without creative efforts shall fall within the protection scope of the present invention.

In the following description, reference is made to " embodiments," which describe a subset of all possible embodiments, but it is understood that " embodiments" may be the same subset or a different subset of all possible embodiments, and may be combined with each other without conflict.

Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. The terminology used herein is for the purpose of describing embodiments of the invention only and is not intended to be limiting of the invention.

The inventor finds that, in the process of implementing the embodiment of the invention, when the hardware coding capability is detected in the related art, the hardware coding capability of the device is obtained by looking up the parameter description of a manufacturer, and then, whether the compatibility problem of the hardware coding capability exists in a certain model is verified in a mode of continuous gray scale. However, when determining whether the hardware coding capability of a certain model has a compatibility problem by continuous gray scale, the product application on the model with incompatible hardware coding capability is inevitably influenced. If the user model is not determined to have stable hard-coding capability, the hard-coding function is enabled for the user model, so that the application flash back of the model which does not support hard-coding in the application using process can be caused, and decoding problems such as screen splash, screen greenness and the like can also occur on the hard-coded code stream on other models.

In addition, the related technology also provides a method for detecting the hardware coding capacity of the user machine type through a cloud real machine platform. After the hard-programming capability detection is verified through the cloud real machine platform, the user machine type with compatibility problems found in the cloud real machine verification process cannot be influenced. However, the cloud-real machine platform also has limitations, the number of most machine types is small, even some machine types are not available, and in addition, whether the machine type has stable hard-programming compatibility or not cannot be judged only by performing verification on a few mobile phones.

For this, a reference code stream is issued to a playing client of a user, the hardware coding capability of equipment of different equipment types is determined by collecting a coding code stream generated by coding a decoded image decoded by the reference code stream reported by the playing client, so that the reference code stream conforming to the video standard to be tested issued by a server can be received by the playing client, each frames in the received reference code stream are read in a traversing manner, software decoding is performed on each frame read, the decoded image obtained by the software decoding is coded by a hardware coder, when the coding of the decoded image decoded by any frame in the reference code stream is completed, the coding result corresponding to any frame and the number of frames of the image to be coded currently stored in a hardware coder buffer area are stored, when the coding of each frames decoded image in the reference code stream is completed, the coding stream composed of the coding result corresponding to each frame in the reference code stream, and the number of frames stored in the hardware coder buffer area in the coding process are reported to the server, so that the average coding capability of the equipment to be tested is determined according to the hardware coding capability of the equipment to be tested, and the average number of the coding server.

In view of this, the embodiments of the present invention provide methods and apparatuses for detecting hardware encoding capability, an electronic device, and a storage medium, which are capable of determining the hardware encoding capability of a device.

The following describes an exemplary application of the hardware coding capability detection device provided in the embodiment of the present invention, and the hardware coding capability detection device provided in the embodiment of the present invention may be implemented as various types of user terminals such as a notebook computer, a tablet computer, a desktop computer, a set-top box, a mobile device (e.g., a mobile phone, a portable music player, a personal digital assistant, a dedicated messaging device, and a portable game device), and may also be implemented as a server or a server cluster, and may also be implemented in a manner that the user terminal and the server cooperate with each other.

Referring to fig. 1, fig. 1 is an optional architecture diagram of a hardware coding capability detection system 100 provided by an embodiment of the present invention, in which a user terminal 400 (illustratively, a terminal 400-1 and a terminal 400-2) is connected to a server 200 through a network 300, and the network 300 may be a domain network or a local area network, or a combination of the two.

As shown in fig. 1, the hardware coding capability detection system 100 includes a server 200, a network 300 and a user terminal 400, the server 200 sends a reference code stream meeting a video standard to be tested to a playing client 410 of the user terminal 400 through the network 300, the playing client 410 calls a central processing unit in the user terminal 400 after confirming that the reference code stream issued by the server 200 is received, and performs software decoding on the read reference code stream in a manner of running decoding software, and then sends a decoded image obtained through software decoding into a hardware encoder of the user terminal 400 for hardware coding, reports the hard coded stream obtained through coding to the server 200 for detection, and reports an average frame number of images to be coded accumulated in a buffer of the hardware encoder during coding at the same time, after receiving the hard coded stream reported by the playing client 410, the server 200 decodes the received hard coded stream, determines a similarity of the decoded image of the reference code stream and the decoded image of the hard coded stream, and evaluates a code control of the coded stream during coding, and further, the server 200 determines a coding capability of the running video equipment according to the reported standard of the hardware coding capability of the running video standard.

Referring to fig. 2A, fig. 2A is a schematic structural diagram of a server 200 according to an embodiment of the present invention, taking a hardware coding capability detection device as the server 200 as an example, the server 200 shown in fig. 2A includes at least processors 210, a memory 250, at least network interfaces 220, and a user interface 230, each component in the server 200 is coupled at through a bus system 240, it is understood that the bus system 240 is used to implement connection communication between these components, the bus system 240 includes a power bus, a control bus, and a status signal bus in addition to a data bus, but for clarity of explanation, the various buses are labeled as the bus system 240 in fig. 2A.

The Processor 210 may be an type integrated circuit chip having Signal processing capabilities such as a general purpose Processor, a Digital Signal Processor (DSP), or other programmable logic device, discrete or transistor logic devices, discrete hardware components, etc., wherein the general purpose Processor may be a microprocessor or any conventional Processor, etc.

The user interface 230 includes or more output devices 231 including or more speakers and/or or more visual display screens that enable presentation of media content the user interface 230 also includes or more input devices 232 including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

The memory 250 may be removable, non-removable, or a combination thereof exemplary hardware devices include solid state memory, hard disk drives, optical disk drives, and the like, the memory 250 optionally includes or more storage devices physically remote from the processor 210.

The memory 250 includes volatile memory or nonvolatile memory, and may include both volatile and nonvolatile memory. The nonvolatile memory may be a Read Only Memory (ROM), and the volatile memory may be a Random Access Memory (RAM). The memory 250 described in embodiments of the invention is intended to comprise any suitable type of memory.

In embodiments, memory 250 is capable of storing data to support various operations, examples of which include programs, modules, and data structures, or subsets or supersets thereof, as exemplified below.

An operating system 251 including system programs for processing various basic system services and performing hardware-related tasks, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks;

a network communication module 252 for communicating to other computing devices via one or more (wired or wireless) network interfaces 220, exemplary network interfaces 220 including bluetooth, wireless-compliant authentication (WiFi), and Universal Serial Bus (USB), among others;

a presentation module 253 to enable presentation of information (e.g., user interfaces for operating peripherals and displaying content and information) via one or more output devices 231 (e.g., display screens, speakers, etc.) associated with the user interface 230;

an input processing module 254 for detecting one or more or more user inputs or interactions from or of the one or more input devices 232 and translating the detected inputs or interactions.

, the hardware encoding capability detecting device provided by the embodiment of the invention can be implemented by software, fig. 2A shows the hardware encoding capability detecting device 255 stored in the memory 250, which can be software in the form of programs and plug-ins, etc., and includes the following software modules, i.e., a sending module 2551, a receiving module 2552 and a detecting module 2553, which are logical, so that any combination or further splitting can be performed according to the implemented functions.

In other embodiments, the hardware coding capability detection apparatus provided by the embodiments of the present invention may be implemented in hardware, and as an example, the hardware coding capability detection apparatus provided by the embodiments of the present invention may be a processor in the form of a hardware decoding processor, which is programmed to execute the hardware coding capability detection method provided by the embodiments of the present invention, for example, the processor in the form of the hardware decoding processor may employ or more Application Specific Integrated Circuits (ASICs), DSPs, Programmable Logic Devices (PLDs), Complex Programmable Logic devices (C PLDs), field Programmable arrays (FPGAs), or other electronic components.

Referring to fig. 2B, fig. 2B is a schematic structural diagram of a user terminal 400 according to an embodiment of the present invention, and taking the hardware coding capability detection device as the user terminal 400 as an example, the user terminal 400 shown in fig. 2B includes at least processors 410, a memory 450, at least network interfaces 420, and a user interface 430.

, in some embodiments, the hardware coding capability detection apparatus provided in the embodiments of the present invention may be implemented in a software manner, and fig. 2B shows the hardware coding capability detection apparatus 455 stored in the memory 450, which may be software in the form of programs and plug-ins, and includes software modules, that is, a receiving module 4551, a decoding module 4552, an encoding module 4553, and a reporting module 4554, which are logical, and therefore may be arbitrarily combined or split in step according to the implemented functions.

An exemplary application of the hardware coding capability detection device provided in the embodiment of the present invention will be described below with reference to the accompanying drawings, which illustrate the hardware coding capability detection method provided in the embodiment of the present invention.

It should be noted that whether the device has the hardware coding capability compatible with the video standard to be tested may be detected only for the encoded code stream, for example, whether the code control and the similarity of the encoded code stream are within the range defined by , or may be combined with the average number of frames of the image to be coded stored in the cache region of the hardware encoder during the encoding process to perform more comprehensive detection.

Referring to fig. 3, fig. 3 is optional flowcharts of the hardware coding capability detection method provided in the embodiment of the present invention, which can be applied to the hardware coding capability detection system 100 shown in fig. 1 to determine the hardware coding capability of a device, the following is a detailed description of the hardware coding capability detection method provided in the embodiment of the present invention from the user terminal 400 side, and as shown in fig. 3, the hardware coding capability detection method provided in the embodiment of the present invention includes the following steps:

in step S301, the server issues a reference code stream meeting the standard of the video to be tested to the playing client of the user terminal.

Here, the reference code stream includes a standard reference test code stream compliant with H264 or H265.

In embodiments, the server may issue a standard reference test code stream conforming to H264 to different types of devices, and determine hardware encoding capabilities of the different types of devices for the H264 standard by analyzing an encoded code stream generated by encoding a decoded image of the standard reference test code stream conforming to H264 by the different types of devices.

In another embodiments, the server may also issue a reference test code stream conforming to the H264 standard to a plurality of devices of the same type, and acquire an encoded code stream generated by encoding, by a plurality of devices of the same type, a decoded image decoded with respect to the reference code stream conforming to the H264 standard, and analyze the acquired encoded code stream, so as to determine the hardware encoding capability of the device type with respect to the H264 standard.

In step S302, after confirming that the reference code stream issued by the server is received, the playing client reads every frames in the received reference code stream in a traversing manner, and performs software decoding on every frames that are read.

Here, after receiving the reference code stream issued by the server, the playing client reads every frames in the received reference code stream in a traversing manner, and invokes a central processing unit in the user terminal to perform software decoding on every frames read in a manner of running decoding software.

In step S303, it is determined whether the encoding is normal, and if abnormal information is generated during the encoding process, step S304 is performed; if the encoding is completed, step S305 is performed.

Here, the playing client sends the decoded image obtained by decoding the reference code stream by the software into a hardware encoder of the user terminal for encoding, if the hardware encoder throws out abnormal information in the encoding process, indicating that the encoding is failed, executing step S304; if the hardware encoder does not return a failure indicating that the encoding is complete, step S305 is performed.

In step S304, the playing client reports the abnormal encoding information to the server.

Here, the hardware encoder of the user terminal throws an exception when encoding the decoded image of the reference code stream, indicating that the hardware encoder of the user terminal does not have a hardware encoding capability for the video standard to be tested. When receiving the coding abnormal information reported by the playing client, the server can directly log the user terminal into a list without hardware coding capability, and simultaneously inform the playing client to stop coding the residual image frames so as to reduce the consumption of system resources.

For example, assuming that the reference code stream is 50 frames, the decoded images of the first 19 frames can all be normally encoded, and the hardware encoder throws out the abnormal information when encoding the decoded image of the 20 th frame, the server, when receiving the encoded abnormal information reported by the playing client, logs the encoded abnormal information into a list without hardware encoding capability, and notifies the user terminal to stop the software decoding and subsequent encoding processes of the remaining 30 frames of reference code streams.

In step S305, the playing client saves the encoding result of the current frame decoded image in the reference code stream and the number of frames of the image to be encoded currently stocked in the hardware encoder buffer of the playing client.

Here, each frame of decoded image after decoding the reference code stream is sent to a hardware encoder of the user terminal for encoding, and when encoding is completed, the encoding result of the current frame of decoded image and the number of frames of the image to be encoded currently accumulated in the buffer of the hardware encoder are saved.

For example, assuming that the reference code stream is 50 frames, and the currently decoded frame is the 19 th frame in the reference code stream, the decoded image of the 19 th frame after decoding is sent to a hardware encoder for encoding, and when encoding is completed, the encoding result of the 19 th frame of decoded image is saved, and the number of frames of the image to be encoded, which is accumulated in a cache region of the hardware encoder when the 19 th frame is encoded, is saved.

In step S306, the playing client reports the encoded code stream and the average storage frame number to the server.

When the decoded images of each frames in the reference code stream sent by the server are all encoded, reporting the encoded code stream consisting of the encoding results corresponding to each frames in the reference code stream and the average frame number of the frames of the images to be encoded accumulated in the hardware encoder cache area in the encoding process to the server.

In an example, assuming that the reference code stream is 50 frames, when each frames of decoded images in 50 frames of decoded images obtained after decoding the reference code stream are encoded, the playing client a reports to the server an encoded code stream generated by hardware encoding of the 50 frames of decoded images, and accumulates the average number of frames of images to be encoded in the encoder cache area in the process of encoding the 50 frames of decoded images.

In step S307, the server determines, according to the encoded code stream and the average storage frame number reported by the playing client, a hardware encoding capability of a device type to which the device operating the playing client belongs, for the video standard to be tested.

The server extracts the coded code stream contained in the coded data after receiving the coded data reported by the playing client, performs software decoding on the coded code stream, compares a decoded image of the coded code stream with a decoded image of a reference code stream, determines the similarity between the decoded image of the coded code stream and the decoded image of the reference code stream, evaluates the code control fluctuation of the coded code stream in the coding process, and determines that the equipment for operating the playing client has the hardware coding capacity compatible with the video standard only when the similarity between the decoded image of the coded code stream and the decoded image of the reference code stream and the code rate fluctuation of the coded code stream are within a fixed range of and the average accumulated frame number in the coding process is smaller than the accumulated frame number threshold.

In , when it is determined that the device running the playback client has hardware encoding capabilities compatible with the video standard to be tested, the default other devices of the same type each have hardware encoding capabilities compatible with the video standard to be tested.

It should be noted that, when detecting the hardware coding capability of the device, the device may only detect video standards (for example, only detect whether the device has the hardware coding capability for the reference test code stream conforming to the H264 standard), and when the detection passes, the device is also determined to have the hardware coding capability for other video standards by default.

In other embodiments, the server may also determine, according to the detection results of the multiple playback clients for the same reference codestream, the hardware encoding capability of the device type to which the device running the playback client belongs for the video standard to be tested.

For example, referring to fig. 4, fig. 4 is a schematic flow chart of a method for detecting hardware coding capability according to an embodiment of the present invention, as shown in fig. 4, a server issues a reference code stream to N (where N is a positive integer, for example, N may be 200 or 500, etc.) playing clients, and receives detection results of the N playing clients for the reference code stream, the server performs statistical analysis on the detection results reported by the N playing clients, determines whether a number of coding successes reported by devices of the same type exceeds a defined threshold (for example, exceeds 85% of device coding successes or all devices are coded successfully), and when determining that the number of coding successes reported by devices of the same type is greater than the defined threshold, further determines whether an average stocked frame number of the devices in a coding process is less than a stocked threshold, and only when the two conditions are simultaneously satisfied, determines that the device type of the device has hardware coding capability compatible with a video standard to be tested, and adds the device type of the device into a whitelist configured correspondingly, and when the device of the playing clients perform call on-video-encoding data.

Referring to fig. 5, fig. 5 is optional flowcharts of the hardware coding capability detection method according to the embodiment of the present invention, which can be applied to the hardware coding capability detection system 100 shown in fig. 1 to determine the hardware coding capabilities of devices of different device types, the following is a detailed description of the hardware coding capability detection method according to the embodiment of the present invention from the server 200 side, and as shown in fig. 5, the hardware coding capability detection method according to the embodiment of the present invention includes the following steps:

in step S501, the server issues a reference code stream meeting the standard of the video to be tested to the playing client of the user terminal.

In step S502, the server receives a coded code stream composed of coding results corresponding to each frames in the reference code stream reported by the playing client, and an average number of frames of images to be coded accumulated in a hardware encoder buffer during the coding process.

Here, the specific process of the playing client encoding the decoded image of the reference code stream issued by the server to generate the encoded code stream may refer to the above step S302 to step S306, and the embodiment of the present invention is not described herein again.

In step S503, after confirming that the encoded code stream reported by the playing client is received, the server alternately reads frames of the reference code stream and the encoded code stream in a traversal manner, and performs software decoding on frames of the read reference code stream and the read encoded code stream.

The server firstly reads frames in the reference code stream, performs soft decoding on frames in the read reference code stream, then reads corresponding frames in the coding code stream, and performs soft decoding on the corresponding frames in the read coding code stream.

For example, assuming that the reference code stream is 50 frames and the encoded code stream is also 50 frames, the server first reads the 1 st frame of the reference code stream and performs soft decoding, and then reads the 1 st frame of the encoded code stream and also performs soft decoding.

In , when reading the reference code stream and each frames of the encoded code stream alternately, before the reading of the reference code stream is finished, the encoded code stream is already read, which indicates that the number of frames of the encoded code stream is less than the number of frames of the reference code stream, i.e. the encoded code stream is in an encoding error, and the server reports the encoding error information to the report of the server.

For example, assuming that the reference code stream is 50 frames, when the 48 th frame of the reference code stream is read, the encoded code stream is already read, that is, the encoded code stream has only 47 frames, which is less than the number of frames of the reference code stream, indicating that the encoded code stream is encoded incorrectly.

In step S504, it is determined whether the current frame in the read encoded code stream is decoded successfully, and if the decoding fails, step S505 is executed; if the decoding is successful, step S506 is performed.

In step S505, the server sends the encoded code stream error information to a report of the server.

Here, the decoding of the software decoder fails when the encoded code stream reported by the playing client is decoded, which indicates that the encoded code stream reported by the playing client does not meet the corresponding standard, i.e., the encoded code stream is in an encoded error, and the server records the encoded error information in the report.

In step S506, the server determines the similarity between the decoded image of the current frame in the encoded code stream and the decoded image of the corresponding frame in the reference code stream, and the current code stream of the encoded code stream.

For example, assuming that the currently decoded frame 17 in the encoded code stream, the server determines the similarity between the decoded image decoded by the frame 17 in the encoded code stream and the decoded image decoded by the frame 17 in the reference code stream, and the code rate of the encoded code stream when the frame 17 is decoded.

In step S507, the server evaluates the total similarity between the decoded image after decoding the encoded code stream and the decoded image after decoding the reference code stream, and the code rate fluctuation of the encoded code stream in the encoding process, and if the total similarity and the code rate fluctuation of the encoded code stream are not within a preset range, then step S508 is executed; if the total similarity and the code rate fluctuation of the encoded code stream are within the preset range, step S509 is executed.

Here, the server compares every decoded pictures of the encoded code stream with every decoded pictures of the reference code stream to determine the total similarity between the decoded pictures of the encoded code stream and the decoded pictures of the reference code stream.

For example, assuming that the reference code stream is 50 frames and the encoded code stream is also 50 frames, the decoded image of the 1 st frame of the reference code stream is compared with the decoded image of the 1 st frame of the encoded code stream, and the decoded image of the 2 nd frame of the reference code stream is compared with the decoded image of the 2 nd frame of the encoded code stream in sequence until the decoded image of the 50 th frame of the reference code stream is compared with the decoded image of the 50 th frame of the encoded code stream, so as to determine the total similarity between the decoded image of the reference code stream and the decoded image of the encoded code stream.

In step S508, the server determines that the device type of the device running the playing client does not have the hardware encoding capability compatible with the video standard to be tested.

When the total similarity between the decoded image of the reference code stream and the decoded image of the encoded code stream is smaller than a preset similarity threshold (for example, the total similarity is smaller than 85%), or the code rate fluctuation of the encoded code stream is larger than a fluctuation threshold (for example, larger than 30%), it is indicated that the quality of the encoded code stream is poor and the encoded code stream cannot be used practically, it is indicated that the device operating the playing client does not have the hardware encoding capability compatible with the video standard to be tested, and the server adds the device type to which the device belongs to a correspondingly configured blacklist.

In , the server may issue a reference code stream to multiple devices of the same type, and determine that the device type does not have hardware encoding capability only when more than a predetermined number of devices do not have hardware encoding capability compatible with the video standard to be tested.

In step S509, the server determines that the device type to which the device running the playing client belongs has a hardware encoding capability compatible with the video standard to be tested.

When the total similarity between the decoded image of the reference code stream and the decoded image of the encoded code stream is greater than a preset similarity threshold (for example, the total similarity is greater than 85%), and the code rate fluctuation of the encoded code stream is less than a fluctuation threshold (for example, less than 30%), which indicates that the quality of the encoded code stream is better and meets the requirement of practical application, the server determines that the type of the device to which the device operating the playing client belongs has the hardware encoding capability compatible with the video standard to be tested.

The following describes an exemplary structure of the hardware encoding capability detecting apparatus 255 provided by the embodiment of the present invention, which is implemented as a software module and can run in the server 200 shown in fig. 1, in embodiments, as shown in fig. 2A, the software module stored in the hardware encoding capability detecting apparatus 255 of the memory 250 may include a sending module 2551, a receiving module 2552 and a detecting module 2553.

The sending module 2551 is configured to send a reference code stream meeting a video standard to be tested to the playing client;

the receiving module 2552 is configured to receive an encoded code stream obtained by encoding with a hardware encoder and reported by the playing client;

the detection module 2553 is configured to perform code control detection and similarity detection on the coded code stream received by the receiving module 2552, and determine, according to a corresponding detection result, a hardware coding capability of a device type to which the device operating the playing client belongs, for the video standard to be tested.

In , in some embodiments, the detecting module 2553 is further configured to determine that the device type to which the device that operates the playing client belongs has a hardware encoding capability compatible with the video standard to be tested when it is detected that the similarity between the decoded image of the encoded code stream and the decoded image of the reference code stream is greater than a similarity threshold and the code rate volatility of the encoded code stream is less than a volatility threshold.

In embodiments, the receiving module 2552 is further configured to receive an average frame number of images to be encoded that are accumulated in a hardware encoder cache area in an encoding process and reported by the playing client, so that the detecting module 2553 determines that a device type to which a device that operates the playing client belongs has a hardware encoding capability compatible with the video standard to be encoded when detecting that the average frame number is less than a frame number threshold, that a similarity between a decoded image of the encoded code stream and a decoded image of the reference code stream is greater than a similarity threshold, and that a code stream volatility of the encoded code stream is less than a volatility threshold;

in , the detecting module 2553 is further configured to, when detecting that abnormal information is generated in the process of decoding the encoded code stream, or,

In , in some embodiments, the detecting module 2553 is further configured to determine that a device type to which a device operating the playing client belongs does not have a hardware encoding capability compatible with the video standard to be tested when receiving abnormal information reported by the playing client in a process of encoding a decoded image of the received reference code stream.

Continuing with the description of the exemplary structure of the hardware coding capability detection apparatus 455 provided by the embodiment of the present invention, which is implemented as a software module, may be executed in the user terminal 400 shown in fig. 1, in embodiments, as shown in fig. 2B, the software module stored in the hardware coding capability detection apparatus 455 of the memory 450 may include a receiving module 4551, a decoding module 4552, an encoding module 4553, and a reporting module 4554.

The receiving module 4551 is configured to receive, by a playing client, a reference code stream that meets the video standard to be tested and is issued by a server;

the decoding module 4552 is configured to perform software decoding on the received reference code stream;

the encoding module 4553 is configured to encode a decoded image obtained through software decoding by the decoding module 4552 through a hardware encoder;

the reporting module 4554 is configured to report the encoded code stream obtained through encoding by the encoding module 4553 to a server, so that the server can report the encoded code stream

In , in some embodiments, the reporting module 4554 is further configured to report an encoded code stream composed of encoding results corresponding to each frame in the reference code stream to a server, so that the server performs the following operations:

and when detecting that the code rate fluctuation of the coding code stream is smaller than a fluctuation threshold value and the similarity between the decoding image of the coding code stream and the decoding image of the reference code stream is larger than a similarity threshold value, determining that the equipment type to which the equipment operating the playing client belongs has the hardware coding capability compatible with the video standard to be tested.

In , the reporting module 4554 is further configured to report, to a server, an encoded code stream composed of encoding results corresponding to each frames in the reference code stream and an average frame number of frames of images to be encoded that are accumulated in the hardware encoder buffer during an encoding process, so that the server performs the following operations:

In , in some embodiments, the reporting module 4554 is further configured to report the encoded code stream to a server, so that the server performs the following operations:

In , the reporting module 4554 is further configured to report, when the playback client generates abnormal information in a process of encoding a decoded image of the reference code stream, the abnormal information to the server, so that the server reports the abnormal information to the playback client, so that the playback client can decode the decoded image of the reference code stream

In , in some embodiments, the reporting module 4554 is further configured to report, to a server, an encoded code stream generated by a playing client in multiple devices of the same type with respect to the reference code stream and an average stocked frame number, so that the server performs the following operations:

It should be noted that, for the technical details that are not used in the hardware coding capability detection apparatus provided by the embodiment of the present invention, it can be understood from the descriptions of the drawings of fig. 3 to 7 or .

In the following, exemplary applications of the embodiments of the present invention in practical application scenarios will be described.

Aiming at the fragmentation of the systems of all android manufacturers at present, before the hardware coding capability is formally used, the hardware coding compatibility of the machine type needs to be verified, for example, whether the fluctuation of code control during hardware coding meets the requirement, whether the hard coded code stream meets the corresponding standard, whether the hard coded code stream is likely to generate screen splash, screen green, decoding failure and other problems during decoding of the hard coded code stream.

The method comprises the steps of adding a hard coding switch of a certain type of standard code to a client, continuously sending the gray level of the switch for starting the hard coding to a user according to the type class by the server, reporting the state of the hard coding to the server at the client, recording the model into a configured black list if the reported number of the hard coding failure of a certain standard reference test code stream (such as H264 and H265) exceeds a defined threshold value for the certain type of model, recording the model into the configured white list if the reported number of the hard coding failure of the certain type of model reference test code stream (such as H264 and H265) is directly below the defined threshold value, closing the function of the hard coding for the black list if the reported number of the hard coding failure of the model H264 or the H265 standard code stream is continuously below the defined threshold value, continuously continuing the white list until the full number of the gray level, and determining whether the hard coding capability of the model is compatible with the model of the model, or not necessary to use the hardware coding capability of the model.

The method comprises the steps of firstly, intercepting YUV ( image coding methods) to obtain data with different resolutions from test YUV ( image coding methods, namely gray values), and then, carrying out hard coding on YUV according to coding control parameters set by different resolutions, and finally, carrying out hard coding on YUV by using a software decoder to carry out soft decoding on the hard coded code stream, if the soft decoding is successful, judging that the model corresponding to the mobile phone has hard coding capability, and recording the model corresponding to the mobile phone into a white list with the hard coding capability and recording the model corresponding to the mobile phone into a standard hard coding success, and if the model corresponding to the mobile phone fails or the soft decoding is successful, recording the model corresponding to the mobile phone into a standard hard coding success white list with no more limitations on the hard coding capability, and even detecting that the model corresponding to the hard coding capability of the mobile phone fails or the model corresponding to the hard coding capability of the mobile phone does not have limitations on the hard coding capability of a hard coding standard, and even if the model corresponding to the hard coding capability of the mobile phone fails, the model corresponding to the hard coding capability of the mobile phone is not detected by using a black list with a hard coding switch, and the hard coding capability of the hard coding switch, if the model corresponding to the hard coding capability of the mobile phone fails, and the mobile phone fails, if the model corresponding to detect the hard coding capability of the hard coding of the mobile phone fails, and the mobile phone fails, if the mobile phone fails, and the mobile phone fails.

Step , if the above two schemes are adopted, for the model with the hard-coding compatibility problem, not only some users receiving the hard-coding start configuration will be affected, but also the code stream with the hard-coding problem will affect other users through the network.

The embodiment of the invention provides android hardware coding capacity detection method combining code control and similarity, a server randomly issues pieces of reference code streams to a user, a client performs soft decoding on the received reference code streams firstly, YUV data after the soft decoding is used as input of a hardware encoder, after the hard encoding is completed, the hard encoded streams are uploaded to the server for detection in steps, and meanwhile, average accumulated frame number in the hard encoding process is reported, at the server side, for each pieces of records reported by the client, the similarity of the YUV data after decoding the reference code streams and the YUV data after decoding the hard encoded streams reported by the client is calculated, meanwhile, the code control fluctuation of the hard encoded streams in the encoding process is evaluated, and only when the similarity and the code control fluctuation of all reported code streams of a certain model are within a threshold range determined by , the model is determined to have stably-used hardware coding capacity.

Referring to fig. 6, fig. 6 is a schematic flowchart of a hardware coding capability detection method provided by an embodiment of the present invention, and is applied to a client side. As shown in fig. 6, the method comprises the steps of:

step S601: and the server transmits the reference code stream to the client.

Here, the reference code stream refers to a standard reference test code stream conforming to H264 or H265.

Step S602: after confirming that the reference code stream issued by the server is received, the client sets corresponding code control parameters according to the file name of the reference code stream issued by the server to start hardware coding capability detection.

And step S603, traversing and reading every frames in the received reference code stream.

Step S604: judging whether the reading of the received reference code stream is finished, if so, executing the step S610; if no reading is finished, step S605 is executed.

Step S605: and sending the current frame in the read reference code stream to a software decoder for decoding.

Step S606: and sending the YUV data obtained by decoding the current frame through software into a hardware encoder for encoding.

Step S607: judging whether the encoding of the sent YUV data is finished, if the hardware encoder does not return failure, indicating that the encoding is finished, executing step S608; if the hardware encoder throws the exception information indicating that the encoding failed, step S609 is executed.

Step S608: and storing the coding result of the hardware coding of the YUV data obtained by decoding the current frame, and updating the number of frames currently stocked in a buffer area of a hardware coder.

Step S609: an encoding error code is determined.

Step S610: and finishing hard weaving.

Here, the hard-coding end means that each decoded images in the reference code stream issued by the server can be encoded until the end.

Step S611: and storing a hard coded stream generated by hardware coding of the image decoded by the reference code stream, and calculating the average accumulation frame number in the hard coding process.

Step S612: and reporting the hard coding stream after the hard coding or the error code generated in the hard coding process to a server.

And the server determines whether a certain model has the hardware coding capability aiming at the standard code stream according to the hard coding stream reported by the client.

Referring to fig. 7, fig. 7 is a schematic flowchart of a hardware coding capability detection method provided in an embodiment of the present invention, and is applied to a server side. As shown in fig. 7, the method comprises the steps of:

step S701: and after confirming that the hard coding stream reported by the client is received, the server starts the detection of the received hard coding stream.

Step S702, traversing every frames in the read reference code stream.

Step S703: judging whether the reading of the reference code stream is finished, if not, executing the step S704; if so, step S712 is performed.

Step S704: and performing soft decoding on the current frame in the read reference code stream.

Step S705, go through every frames in the hard coded stream reported by the reading client.

Step S706: judging whether the reading of the hard coded stream is finished, if not, executing the step S707; if so, step S711 is performed.

Step S707: and performing soft solution on the current frame in the read hard coded stream.

Step S708: judging whether the current frame in the hard coded stream is decoded successfully or not, and if the decoding is successful, executing step S709; if the decoding fails, step S711 is performed.

Step S709: and updating the similarity between the decoded YUV data of the current frame in the hard coded stream and the decoded YUV data of the current frame in the corresponding reference code stream.

Step S710: and updating the code rate of the current hard coded stream.

Step S711: and reporting the coding error information of the hard coded stream to a report.

Here, the hard coded stream coding error information includes that the hard coded stream fails to decode, that is, the hard coded stream does not meet the corresponding standard, and that the hard coded stream has already read the end of the file before the end of the reference code stream, that is, the number of frames of the hard coded stream is less than the number of frames of the reference code stream.

Step S712: and finishing the soft solution.

Here, the soft decoding end means that the frame number of the hard coded stream is the same as the frame number of the reference code stream, and each frames in the hard coded stream can be normally soft decoded until the decoding end.

Step S713: and determining the similarity between the YUV data decoded by the hard coded stream and the YUV data decoded by the reference code stream and the code rate fluctuation of the hard coded stream in the decoding process.

Step S714: judging whether the similarity between the YUV data decoded by the hard coded stream and the YUV data decoded by the reference code stream is larger than a defined threshold (for example: 85%) and whether the code rate volatility of the hard coded stream in the decoding process is smaller than a volatility threshold (for example: 30%), if so, executing a step S715; if not, step S716 is performed.

Step S715: and reporting the successful information of the hard-coded detection to a report.

Step S716: and reporting the hard-coded detection failure information to a report.

Step S717: and summarizing and analyzing the data in the report, and adding the type into a hard-coded configuration white list corresponding to the standard code stream only when the hard-coded power of the H264 or H265 standard code stream reported by the type is above a defined threshold value and the average accumulated frame number reported by the client in the hard-coded process meets the delay requirement of the product.

The android hardware coding capability detection method combining code control and similarity provided by the embodiment of the invention randomly issues pieces of reference code streams to a client of a user through a server, the client performs soft decoding on the reference code streams at first, sends YUV data after the soft decoding to a hardware encoder for hard encoding, reports the hard encoded streams to the server for detection in step after the hard encoding is completed, and reports average accumulated frame number in the hard encoding process at the same time, at the server, calculates similarity of decoded YUV data of the reference code streams and the hard encoded streams for each records reported by the client, evaluates code control fluctuation of the hard encoded streams in the encoding process, only when similarity of all reported hard encoded streams of a certain model and decoded YUV data of the reference code streams and code control fluctuation are within a threshold range determined by , determines that models have stably used hardware coding capability, and in practical application, can perform analysis on hardware coding capability of hardware coding streams of a certain model after the hardware coding capability of the hardware encoder and hardware decoder of the hardware encoder are used in model 2 function modules of a product application, when the hardware decoder of the hardware coding capability of the user is started, and when the hardware decoder of the hardware encoder is started, the hardware encoder of the hardware encoder is performed on a hardware decoder, and when the hardware decoder, the hardware decoder of the hardware decoder, the hardware encoder of the hardware decoder, and the hardware decoder of the hardware encoder is performed on a hardware decoder.

The present invention provides storage media storing executable instructions, which when executed by a processor will cause the processor to perform any of the hardware coding capability detection methods provided by embodiments of the present invention, for example, the hardware coding capability detection methods shown in fig. 3-7.

In embodiments, the storage medium may be FRAM, ROM, PROM, EPROM, EE PROM, flash memory, magnetic surface memory, optical disk, or CD-ROM, or any combination of these memories .

In embodiments, the executable instructions may be in the form of a program, software module, script, or code written in any form of programming language, including compiled or interpreted languages, or declarative or procedural languages, and it may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

By way of example, executable instructions may, but are not limited to , correspond to files in a file system, may be stored in the portion of a file that holds other programs or data, such as in or more scripts in a hypertext markup Language (H TML) document, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files that store or more modules, sub-programs, or portions of code).

By way of example, executable instructions may be deployed to be executed on computing devices, or on multiple computing devices located in sites, or distributed across multiple sites and interconnected by a communication network.

In summary, the embodiment of the invention has the following beneficial effects:

the method comprises the steps that a server issues a reference code stream meeting a video standard to be tested to a playing client of a user, the playing client firstly carries out soft decoding on the received reference code stream, images after the soft decoding are sent to a hardware encoder to be encoded, after the hard encoding is finished, a hard encoding stream is reported to the server to carry out steps of detection, and meanwhile, an average accumulated frame number in the hard encoding process is reported, after the server receives the hard encoding stream reported by the playing client, the similarity between the decoded images of the reference code stream and the hard encoding stream is calculated, the code control volatility of the hard encoding stream in the encoding process is evaluated, further, the type of equipment which the playing client belongs to can be determined according to the similarity, the code control volatility and the average accumulated frame number, the steps are carried out, the reference code stream can be issued to a plurality of playing clients of the same type, detection results reported by the plurality of playing clients can be statistically analyzed, and accordingly, the types of equipment which the playing client belongs to the equipment to be tested can be more accurately determined to provide a reliable ecological coding capability for manufacturers.

The above description is only an example of the present invention, and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, and improvement made within the spirit and scope of the present invention are included in the protection scope of the present invention.

Claims

1, hardware coding capability detection method, characterized in that, the method includes:

2. The method of claim 1,

the code control detection result comprises the code rate fluctuation of the coding code stream;

the similarity detection result comprises the similarity of the decoded image of the coding code stream and the decoded image of the reference code stream;

the determining, according to the corresponding detection result, the hardware encoding capability of the device type to which the device running the playing client belongs for the video standard to be tested includes:

3. The method of claim 2,

the code stream is further configured to cause the server to perform the following operations:

4. The method of claim 2, further comprising:

the playing client reports the average frame number of the image frames to be coded accumulated in the cache region of the hardware coder in the coding process to the server so as to ensure that the playing client can play the image frames to be coded in the cache region of the hardware coder in the coding process

And when detecting that the code rate fluctuation of the coded code stream is smaller than a fluctuation threshold, the similarity between the decoded image of the coded code stream and the decoded image of the reference code stream is larger than a similarity threshold, and the average frame number is smaller than a frame number threshold, the server determines that the equipment type to which the equipment operating the playing client belongs has the hardware coding capability compatible with the video standard to be tested.

5. The method of claim 4, further comprising:

the encoded code stream and the average frame number reported to the server by the playing clients operating in a plurality of devices of the same type are used for enabling the server to execute the following operations:

6, hardware coding capability detection method, characterized in that, the method includes:

7. The method of claim 6,

8. The method of any one of claims 6 to 7 and , further comprising:

receiving the average number of frames of the images to be coded, which are accumulated in a cache region of a hardware coder in the coding process and reported by the playing client;

and when detecting that the average frame number is smaller than a frame number threshold, the similarity between the decoded image of the coding code stream and the decoded image of the reference code stream is larger than a similarity threshold, and the code stream fluctuation of the coding code stream is smaller than a fluctuation threshold, determining that the equipment type to which the equipment operating the playing client belongs has the hardware coding capability compatible with the video standard to be tested.

9, hardware coding ability detection device, characterized in that, the device includes:

10, hardware coding ability detection device, characterized in that, the device includes: