CN114866840A

CN114866840A - VMAF image quality evaluation method, terminal, host, system and storage medium

Info

Publication number: CN114866840A
Application number: CN202210345879.8A
Authority: CN
Inventors: 鲍琦
Original assignee: Guangzhou Cubesili Information Technology Co Ltd
Current assignee: Guangzhou Cubesili Information Technology Co Ltd
Priority date: 2022-03-31
Filing date: 2022-03-31
Publication date: 2022-08-05

Abstract

The application discloses a VMAF image quality evaluation method, a terminal, a host, a system and a storage medium, wherein the method comprises the following steps: receiving at least two video files to be coded with different resolutions sent by a host, wherein the at least two video files to be coded with different resolutions are obtained by zooming a preset video source by the host; calling a hardware encoder to respectively encode at least two video files to be encoded with different resolutions by using at least one group of video encoding parameters, and correspondingly obtaining at least two encoded video files; and sending the at least two encoded video files to the host, so that the host performs VMAF image quality evaluation by using the at least two encoded video files and a preset video source. Through the mode, the applicability of the VMAF can be improved.

Description

VMAF image quality evaluation method, terminal, host, system and storage medium

Technical Field

The application relates to the technical field of network live broadcast, in particular to a VMAF image quality evaluation method, a terminal, a host, a system and a storage medium.

Background

With the rapid development of the live broadcast industry, the requirement of a user on the picture quality in the live broadcast process is higher and higher, and the high-quality broadcast picture quality can enable the user to obtain better viewing experience. In order to determine what code rate interval can be used to obtain the optimal choice of the image quality, the image quality under different resolutions of different code rates can be evaluated, so as to determine the code rate interval suitable for playing under the resolution to obtain the optimal image quality.

VMAF (Video multi-method Assessment Fusion) can use a large amount of subjective data as a training set, and perform "Fusion" on algorithms with different Assessment dimensions by means of machine learning to obtain an image quality evaluation standard capable of accurately reflecting subjective will, but VMAF has certain limitations in practical application.

Disclosure of Invention

The technical problem mainly solved by the application is to provide a VMAF image quality evaluation method, a terminal, a host, a system and a storage medium, which can improve the applicability of VMAF.

In order to solve the technical problem, the application adopts a technical scheme that: provided is a VMAF image quality evaluation method of a hardware encoder, which comprises the following steps: receiving at least two video files to be coded with different resolutions sent by a host, wherein the at least two video files to be coded with different resolutions are obtained by zooming a preset video source by the host; calling a hardware encoder to respectively encode at least two video files to be encoded with different resolutions by using at least one group of video encoding parameters, and correspondingly obtaining at least two encoded video files; and sending the at least two encoded video files to the host, so that the host performs VMAF image quality evaluation by using the at least two encoded video files and a preset video source.

In order to solve the above technical problem, another technical solution adopted by the present application is: provided is a VMAF image quality evaluation method of a hardware encoder, which comprises the following steps: sending at least two video files to be coded with different resolutions obtained by zooming a preset video source to a mobile terminal; receiving at least two coded video files which are obtained by respectively coding at least two video files to be coded with different resolutions by calling a hardware coder by a mobile terminal by using at least one group of video coding parameters; and performing VMAF image quality evaluation by utilizing at least two coded video files and a preset video source.

In order to solve the above technical problem, another technical solution adopted by the present application is: there is provided a mobile terminal including: the receiving module is used for receiving at least two video files to be coded with different resolutions sent by the host, wherein the at least two video files to be coded with different resolutions are obtained by scaling a preset video source by the host; the encoding module is used for calling a hardware encoder to respectively encode at least two video files to be encoded with different resolutions by using at least one group of video encoding parameters to correspondingly obtain at least two encoded video files; and the sending module is used for sending the at least two encoded video files to the host so that the host can perform VMAF image quality evaluation by utilizing the at least two encoded video files and a preset video source.

In order to solve the above technical problem, another technical solution adopted by the present application is: providing a host, the host comprising: the device comprises a sending module, a receiving module and a processing module, wherein the sending module is used for sending at least two video files to be coded with different resolutions obtained by zooming a preset video source to a mobile terminal; the mobile terminal comprises a receiving module, a processing module and a processing module, wherein the receiving module is used for receiving at least two coded video files which are obtained by respectively coding at least two video files to be coded with different resolutions by using at least one group of video coding parameters by a mobile terminal calling hardware encoder; and the image quality evaluation module is used for performing VMAF image quality evaluation by utilizing at least two coded video files and a preset video source.

In order to solve the above technical problem, another technical solution adopted by the present application is: there is provided a mobile terminal including: a processor, memory, and communication circuitry; the memory and the communication circuit are coupled to the processor, the memory stores a computer program, and the processor can execute the computer program to implement the method for evaluating the VMAF image quality of the hardware encoder as provided in the present application.

In order to solve the above technical problem, another technical solution adopted by the present application is: providing a host, the host comprising: a processor, memory, and communication circuitry; the memory and the communication circuit are coupled to the processor, the memory stores a computer program, and the processor can execute the computer program to implement the method for evaluating the VMAF image quality of the hardware encoder as provided in the present application.

In order to solve the technical problem, the other technical scheme adopted by the application is as follows: the VMAF image quality evaluation system of the hardware encoder comprises the mobile terminal and the host computer, wherein the mobile terminal is in communication connection with the host computer.

In order to solve the above technical problem, another technical solution adopted by the present application is: there is provided a computer-readable storage medium storing a computer program executable by a processor to implement the method for evaluating the VMAF image quality of a hardware encoder as provided in the present application.

The beneficial effect of this application is: different from the prior art, the method comprises the steps of receiving at least two video files to be coded with different resolutions sent by a host, wherein the at least two video files to be coded with different resolutions are obtained by scaling a preset video source by the host, because the host uses a software coding algorithm when coding the preset video source before using a VMAF (virtual media access function) for image quality evaluation, a live application scene also comprises a mobile terminal, and most of coded data of the mobile terminal is based on a hardware encoder, when the VMAF is used for image quality evaluation of the mobile terminal, a coding part can be carried out on the mobile terminal, the hardware encoder in the mobile terminal is called to respectively code the at least two video files to be coded with different resolutions received from the host by using at least one group of video coding parameters, correspondingly obtaining at least two coded video files, and sending the obtained at least two coded video files to the host, therefore, the host can utilize at least two coded video files and a preset video source to perform VMAF image quality evaluation, the coding link is transferred to the mobile terminal to replace the original link of coding by using software coding on the host, the data of VAMF image quality evaluation can be closer to the real situation of a specific hardware encoder, the applicability of VMAF is improved, the operation complexity can be reduced, and the authenticity and the accuracy of VMAF image quality evaluation data are improved.

Drawings

FIG. 1 is a schematic diagram of a system of an embodiment of a VMAF image quality evaluation system of the present application;

FIG. 2 is a timing diagram illustrating an embodiment of a VMAF image quality evaluation system of the present application;

FIG. 3 is a schematic flow chart illustrating a VMAF image quality evaluation method according to an embodiment of the present application, with a mobile terminal as an execution subject;

FIG. 4 is a schematic diagram of VMAF change with code rate in the VMAF image quality evaluation method of the present application;

FIG. 5 is a schematic diagram of a mobile terminal according to an embodiment of the present application;

FIG. 6 is a schematic flow chart illustrating a method for VMAF image quality evaluation according to an embodiment of the present application, with a host computer as an execution subject;

FIG. 7 is a schematic diagram of the components of an embodiment of the host computer of the present application;

FIG. 8 is a schematic diagram of a circuit configuration of an embodiment of the computer electronic device of the present application;

FIG. 9 is a schematic circuit diagram of an embodiment of a computer-readable storage medium according to the present application.

Detailed Description

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

With the rapid development of the live broadcast industry, the requirement of a user on the picture quality in the live broadcast process is higher and higher, and the high-quality broadcast picture quality can enable the user to obtain better viewing experience. In order to determine what code rate interval can be used to obtain the optimal choice of the image quality, the image quality under different resolutions of different code rates can be evaluated, so as to determine the code rate interval suitable for playing under the resolution to obtain the optimal image quality. VMAF (video Multi-method Assessment fusion) can use a large amount of subjective data as a training set, and carry out 'fusion' on algorithms with different Assessment dimensions by means of machine learning to obtain an image quality evaluation standard capable of accurately reflecting subjective will. Specifically, after an input video source of a specific scene is selected, code rates can be traversed according to a certain step length for different coding modes, different resolutions and different frame rates, and then VMAF scores of corresponding parameters are calculated, so that the VMAF score in a code rate interval is determined to be the highest, and the higher the VMAF score is, the better the image quality is represented. For example, the VMAF can evaluate what code rate interval can obtain the optimal image quality when the 720p, 15fps, 540p, and 15fps are broadcast respectively, so as to determine what code rate range is suitable for the 720p broadcast, and what code rate range is suitable for the 540p broadcast, and further apply the method in the process of configuring the direct broadcast image quality, thereby providing the most suitable broadcast image quality for the user, improving the viewing experience of the user, and being beneficial to improving the user stickiness.

The inventor finds that before VMAF quality evaluation is performed on an input source to be evaluated, the processes of scaling, coding and decoding the input source are all completed on the host 200 by using software coding algorithms such as x264 or x265 and the like carried by ffmpeg (well-known open source audio and video processing tool), the prepared input source is only required to be scaled, coded and decoded by the ffmpeg on the host 200, and then the decoded input source and an uncoded video source are used for contrast quality evaluation to output VMAF scores. However, in the live broadcast process, the application of the mobile terminal 100 needs to be involved, and most of the encoded data of the mobile terminal 100 is based on hardware coding, and because the encoded data obtained by hardware coding and software coding are different, the real image quality evaluation result cannot be accurately reflected. In order to improve the above technical problem, the present application proposes the following embodiments.

As shown in fig. 1 and fig. 2, the VMAF image quality evaluation system of the hardware encoder of the present application may include a mobile terminal 100 and a host 200, where the mobile terminal 100 and the host 200 may be a computer electronic device 1, and reference may be made to the following description related to the computer electronic device 1 in terms of a schematic circuit structure. The mobile terminal 100 and the host 200 are communicatively connected. In the process of performing the VMAF image quality evaluation, the encoding of the preset video source to be subjected to the image quality evaluation is completed by the mobile terminal 100, after the mobile terminal 100 completes encoding of the preset video source, the encoded video file is sent to the host 200, and the VMAF image quality evaluation is performed on the encoded video file by the host 200.

The mobile terminal 100 may be a terminal with a hardware encoder, and the preset video source is encoded by the hardware encoder in the mobile terminal 100 to obtain an encoded video file. For example, the mobile terminal 100 may be a mobile phone, a tablet computer, a smart wearable device, and the like.

The host 200 may be a terminal with a VMAF, and outputs VMAF scores at different code rate intervals by receiving an encoded video file of the mobile terminal 100, and comparing the VMAF quality evaluation with a preset video source and the encoded video file. For example, the host 200 may be a PC computer, a server, or the like.

The embodiment of the method for evaluating the VMAF image quality of the hardware encoder of the present application may be applied to the VMAF image quality evaluation system of the hardware encoder described in the embodiment of the system for evaluating the VMAF image quality of the hardware encoder, where the mobile terminal 100 is used as an execution perspective of the embodiment, and the method may include: s100: receiving at least two video files to be encoded with different resolutions sent by a host, wherein the at least two video files to be encoded with different resolutions are obtained by scaling a preset video source by the host. S200: and calling a hardware encoder to respectively encode at least two video files to be encoded with different resolutions by using at least one group of video encoding parameters, and correspondingly obtaining at least two encoded video files. S300: and sending the at least two encoded video files to the host, so that the host performs VMAF image quality evaluation by using the at least two encoded video files and a preset video source.

In the process of evaluating the quality of the VMAF image, the mobile terminal 100 receives at least two video files to be encoded with different resolutions sent by the host 200, the at least two video files to be encoded with different resolutions are obtained by scaling the preset video source by the host 200, since the host 200 uses a software encoding algorithm when encoding the preset video source before using the VMAF to evaluate the quality of the image, and the live application scenario further includes the mobile terminal 100, and most of the encoded data of the mobile terminal 100 is based on a hardware encoder, when evaluating the quality of the image of the mobile terminal 100 using the VMAF, an encoding part can be performed in the mobile terminal 100, the hardware encoder in the mobile terminal 100 is invoked to encode the at least two video files to be encoded with different resolutions received from the host 200 by using at least one set of video encoding parameters, so as to obtain at least two encoded video files correspondingly, the obtained at least two encoded video files are sent to the host 200, so that the host 200 can utilize the at least two encoded video files and a preset video source to perform VMAF image quality evaluation, and thus, the encoding link is transferred to the mobile terminal 100 to replace the original link of encoding by using software encoding on the host 200, so that the VAMF image quality evaluation data can be closer to the actual situation of a specific hardware encoder, the VMAF applicability is improved, the operation complexity is reduced, and the authenticity and the accuracy of the VMAF image quality evaluation data are improved.

The method described in the present embodiment can be applied to a VMAF quality evaluation scenario, and the following describes the present embodiment in detail with the mobile terminal 100 as an execution subject.

As shown in fig. 3, the present embodiment may include the following steps:

s100: and receiving at least two video files to be encoded with different resolutions sent by the host.

The at least two video files to be encoded with different resolutions may be obtained by scaling a preset video source by the host 200. Since the VMAF can only process video files with a resolution of 1080p and a YUV format, the preset video source and the video file to be encoded are both video files with the YUV format, however, in the actual playing process, different resolutions need to be used for playing according to the needs of a user, and therefore, the playing code rate which is most suitable for the image quality in each resolution needs to be determined, so that the preset video source needs to be scaled into at least two video files to be encoded with different resolutions, and the resolution of the video file to be encoded is less than or equal to the resolution of the preset video source. Specifically, in the process of receiving at least two video files to be encoded of different resolutions transmitted by the host 200, the at least two video files to be encoded of different resolutions in the host 200 may be copied into the mobile terminal 100.

In one implementation, S100 may include the steps of:

s110: and receiving at least two video files to be encoded with different resolutions sent by a host through a preset application program.

The preset application may be an application preset by the mobile terminal 100 for receiving and processing a video file to be encoded. Specifically, after the host 200 scales the preset video source to obtain at least two video files to be encoded, a list of the video files to be encoded with different resolutions is formed. After copying at least two video files to be encoded to the mobile terminal 100, a preset application is opened, and the application selects a list of the video files to be encoded copied from the host 200 to access the video files to be encoded with different resolutions.

For example, when the selected preset video source in the host 200 is a YUV video file with a resolution of 1080p, the preset video source may be respectively scaled to a resolution of 540p and a resolution of 720p in the host 200, so as to obtain a video file to be encoded with the resolution of 540p and a video file to be encoded with the resolution of 720p, then the video file to be encoded with the resolution of 540p and the video file to be encoded with the resolution of 720p are copied to the mobile terminal 100, and the video file to be encoded with the resolution of 540p and the video file to be encoded with the resolution of 720p are received through a preset application program. By scaling the preset video source into at least two video files to be encoded with different resolutions, the video files to be encoded with different resolutions can be simultaneously sent to the mobile terminal 100 for encoding processing, which is beneficial to improving the processing efficiency.

S200: and calling a hardware encoder to respectively encode at least two video files to be encoded with different resolutions by using at least one group of video encoding parameters, and correspondingly obtaining at least two encoded video files.

The hardware encoder may be an encoder of the mobile terminal 100 itself for encoding the video signal.

The video encoding parameters may be parameters used in encoding by a hardware encoder based on which the video file to be encoded is encoded. Specifically, in the process of encoding a video file to be encoded based on video encoding parameters, one or more groups of video files to be encoded with different resolutions may be encoded respectively, so as to facilitate fast obtaining of encoded video files corresponding to a plurality of different video encoding parameters.

The encoded video file may be a video file obtained by encoding a video file to be encoded based on the video encoding parameter.

In one implementation, S200 may include the steps of:

s210: and traversing at least two video files to be coded with different resolutions in sequence.

Sequentially traversing at least two video files to be encoded with different resolutions may be accessing all the video files to be encoded with different resolutions that are sequentially received through a preset application program in the mobile terminal 100. Specifically, in the process of traversing, all the video files to be encoded may be sequentially accessed according to the ordering of the video file list to be encoded generated in the host 200.

S220: and calling a hardware encoder to encode each video file to be encoded according to at least one group of video encoding parameters respectively until the encoding of all the video files to be encoded is completed.

The video coding parameters may be preset in a hardware encoder, and in the process of traversing the video files to be coded one by one, the hardware encoder is called to code each video file to be coded based on at least one group of video coding parameters until the coding of all the video files to be coded is completed. Specifically, in the process of encoding each video file to be encoded, encoding may be performed based on multiple sets of video encoding parameters, so as to obtain multiple encoded video files corresponding to the video file to be encoded.

In one implementation, the following steps may be included before S220:

s221: and acquiring at least one coding mode, at least one frame rate and at least one code rate and combining to obtain at least one group of video coding parameters.

The coding mode, the frame rate and the code rate can be that a hardware encoder sets a plurality of different selected data for each data in advance, and in the process of obtaining video coding parameters, one selected data can be selected from each data to be combined to obtain a plurality of groups of video coding parameters, so that the method is beneficial to coding based on different video coding parameters to obtain a plurality of coded video files, further carrying out VMAF image quality evaluation based on the plurality of video coding files, and improving the accuracy of the image quality evaluation. Specifically, in the process of obtaining the video coding parameters, at least one coding mode, at least one frame rate, at least one code rate, and at least one resolution may also be obtained and combined. For example, as shown in fig. 4, if the coding method in the hardware encoder has 2 selection data, x264 hard coding and x265 hard coding, the frame rate has 3 selection data, 15fps to 25fps, the code rate has 11 selection data, 500 bits/s to 2500 bits/s, and the resolution has 2 selection data, 720p and 540p, the number of video coding parameters that can be combined is 2 × 3 × 11 × 2 ═ 132, one or more groups of video coding parameters can be selected from the video coding parameters to be coded, for example, the coding method can be selected as 264 hard coding, the frame rate is 15fps, the code rate is 1100 bits/s, and the resolution is 720p, and the combined video coding parameters encode the video file to be coded with the resolution of 720 p.

In one implementation, S220 may include the following steps:

s222: and automatically calling a hardware encoder by a preset application program in the background, and respectively encoding at least two video files to be encoded with different resolutions by using at least one group of video encoding parameters to correspondingly obtain at least two encoded video files.

In the process of encoding a video file to be encoded by using the video encoding parameters, the hardware encoder may be automatically invoked in the background through a preset application program in the mobile terminal 100, and the hardware encoder encodes the video file by using the video encoding parameters. Specifically, at least two video files to be encoded with different resolutions can be encoded by using at least one set of video encoding parameters, so as to obtain at least two encoded video files correspondingly.

In one implementation, after S222, the following steps may be included:

s230: and writing at least two coded video files into a preset memory.

The preset memory may be preset in the mobile terminal 100. After the video file to be encoded is encoded to obtain the encoded video file, the encoded video file can be written into the preset memory. By writing the encoded video file into the preset memory as a transfer station in the video encoding process, if the speed of generating the encoded video file by encoding is too fast and the speed of sending the encoded video file to the host 200 is too slow, the video encoding process is slowed down by directly sending the encoded video file, and the encoded video file is written into the preset memory, so that the encoding speed is not influenced, and the acquisition of the host 200 is not influenced. By setting the preset memory, frequent communication between the host 200 and the mobile terminal 100 and inquiry about whether coding is completed can be avoided, and the host 200 only needs to inquire the preset memory, and when a coded video file is inquired, the coded video file can be received and read from the mobile terminal 100.

S300: and sending the at least two encoded video files to the host, so that the host performs VMAF image quality evaluation by using the at least two encoded video files and a preset video source.

After the video file to be encoded is encoded to obtain the encoded video file, the encoded video file may be sent to the host 200, so that the host 200 may perform VMAF image quality evaluation by using at least two encoded video files and a preset video source. Specifically, in the process of evaluating the quality of the VMAF image, the encoded video file after encoding is compared with the preset video source which is not encoded, so as to output a corresponding VMAF score, thereby evaluating the quality of the image.

In one implementation, S300 may include the steps of:

s310: and sending at least two encoded video files in a preset memory to the host.

Since the encoded video file can be written into the preset memory after the encoded video file is acquired, the encoded video file can be called out from the preset memory and sent to the host 200 in the process of sending the encoded video file to the host 200.

For example, when the encoding mode is x264 hard encoding, the frame rate is 15fps, the code rate is 1100bit/s, the resolution is 720p, the combined video encoding parameters encode the video file to be encoded with the resolution of 720p to obtain a first encoded video file, the first encoded video file is sent to the host 200 and compared with the preset video source in the host 200 for VMAF quality evaluation, the output first VMAF score is 89 points, and when the encoding mode is x264 hard encoding, the frame rate is 15fps, the code rate is 1300 bits/s, the resolution is 720p, the combined video encoding parameters encode the video file to be encoded with the resolution of 720p to obtain a second encoded video file, the second video encoded file is sent to the host 200 and compared with the preset video source in the host 200 for VMAF quality evaluation, the output second VMAF score is 91 points, it can be shown that the code rate of 1300bit/s is more suitable for video playback with the resolution of 720p and better image quality can be obtained compared with the code rate of 1100bit/s, so that in the live broadcast process, when the resolution is 720p, the code rate of 1300bit/s can be selected for playback.

As shown in fig. 5, the mobile terminal 100 according to the embodiment of the present application may include a receiving module 10, an encoding module 20, and a transmitting module 30. Specifically, the receiving module 10 is configured to receive at least two to-be-encoded video files with different resolutions sent by the host 200, where the at least two to-be-encoded video files with different resolutions are obtained by scaling a preset video source by the host 200. The encoding module 20 is configured to invoke a hardware encoder to encode at least two video files to be encoded with different resolutions respectively by using at least one group of video encoding parameters, so as to obtain at least two encoded video files correspondingly. The sending module 30 is configured to send the at least two encoded video files to the host 200, so that the host 200 performs VMAF quality evaluation by using the at least two encoded video files and a preset video source.

The embodiment of the method for evaluating the VMAF image quality of the hardware encoder of the present application can be applied to the hardware encoder VMAF image quality evaluation system described in the above hardware encoder VMAF image quality evaluation system embodiment, where the host 200 is used as an execution perspective, the method can include: s400: the video files to be encoded of at least two different resolutions obtained by scaling the preset video source are sent to the mobile terminal 100. S500: and receiving at least two coded video files which are obtained by respectively coding at least two video files to be coded with different resolutions by calling a hardware coder by the mobile terminal by using at least one group of video coding parameters. S600: and performing VMAF image quality evaluation by utilizing at least two coded video files and a preset video source.

The method comprises the steps of zooming a preset video source in a host 200 to obtain at least two video files to be coded with different resolutions, then sending the at least two video files to be coded to a mobile terminal 100, coding the video files to be coded through a hardware encoder in the mobile terminal 100 to obtain coded video files, then receiving the coded video files sent back by the mobile terminal 100, and performing VMAF image quality evaluation by using the coded video files and the preset video source, so that the obtained image quality evaluation result is closer to the real situation of hardware encoding without being based on software coded data in the host 200, and the authenticity and accuracy of the VMAF image quality evaluation are favorably improved.

The method described in the present embodiment can be applied to the VMAF image quality evaluation scenario, and the following describes the present embodiment in detail with the host 200 as an execution subject.

As shown in fig. 6, the present embodiment may include the following steps:

s400: and sending the at least two video files to be coded with different resolutions obtained by scaling the preset video source to the mobile terminal.

Since the image quality loss under the condition of different resolutions needs to be evaluated in the actual playing process, and the preset video source is a video file with a fixed resolution, the preset video source needs to be scaled into files to be encoded with different resolutions, so that the video files to be encoded with different resolutions can be simultaneously sent to the mobile terminal 100, the efficiency of processing the preset video source is improved, after the mobile terminal 100 receives at least two video files to be encoded with different resolutions, the video files to be encoded with different resolutions can be respectively encoded, the encoded video files with different resolutions can be obtained, the video files do not need to be received from the host 200 for multiple times, and the encoding efficiency is further improved.

S500: and receiving at least two coded video files which are obtained by respectively coding at least two video files to be coded with different resolutions by calling a hardware coder by the mobile terminal by using at least one group of video coding parameters.

After receiving the at least two video files to be encoded, the mobile terminal 100 invokes a hardware encoder preset in the mobile terminal 100 to encode the video files to be encoded based on the video encoding parameters, so as to obtain at least two encoded video files correspondingly, and the specific encoding process may refer to the process in S200, which is not described herein again. After obtaining the encoded video file, the mobile terminal 100 sends the encoded video file back to the host 200, and the host 200 receives the encoded video file and performs VMAF image quality evaluation. Specifically, an encoded video file in the mobile terminal 100 may be copied to the host 200.

S600: and performing VMAF image quality evaluation by utilizing at least two coded video files and a preset video source.

After receiving the encoded video files sent back by the mobile terminal 100, the host 200 may perform a comparison analysis using at least two encoded video files and a preset video source to perform a VMAF quality evaluation.

In one implementation, S600 may include the following steps:

s610: and respectively decoding the at least two coded video files to correspondingly obtain at least two decoded video files.

Since the encoded video file received by the host 200 is obtained by encoding by calling a hardware encoder, if the encoded video file needs to be processed, the encoded video file needs to be decoded, so as to obtain a decoded video file.

S620: and respectively scaling the at least two decoded video files to preset resolution, wherein the preset resolution is consistent with the resolution of a preset video source.

Since the VMAF can only process a video file with a resolution of 1080p, and the resolution of the decoded video file is a resolution obtained after scaling before encoding, the decoded video file needs to be scaled to a preset resolution, that is, 1080 p. By scaling the decoded video file to a preset resolution, the scaled decoded video file can be input to the VMAF for image quality evaluation.

S630: and inputting the at least two scaled decoded video files and a preset video source into the VMAF model for image quality evaluation.

Because the decoded video file after being encoded, decoded and zoomed is distorted, the decoded video file and a preset video source which is not encoded are input into the VMAF model together for comparison and evaluation, and a VMAF score is output, thereby evaluating the image quality.

As shown in fig. 7, the host 200 according to the embodiment of the present application may include a sending module 40, a receiving module 50, and a quality evaluation module 60. The sending module 40 is configured to send at least two video files to be encoded with different resolutions, which are obtained by scaling a preset video source, to the mobile terminal 100. The receiving module 50 is configured to receive at least two encoded video files obtained by using the mobile terminal 100 to invoke the hardware encoder to encode at least two video files to be encoded with different resolutions respectively by using at least one set of video encoding parameters. The image quality evaluation module 60 is configured to perform VMAF image quality evaluation by using at least two encoded video files and a preset video source.

As shown in fig. 8, the computer electronic device 1 described in the embodiment of the computer electronic device of the present application includes a processor 110, a memory 120 and a communication circuit, and the memory 120 and the communication circuit are coupled to the processor 110.

The memory 120 is used for storing computer programs, and may be a RAM, a ROM, or other types of storage devices. In particular, the memory may include one or more computer-readable storage media, which may be non-transitory. The memory may also include high speed random access memory, as well as non-volatile memory, such as one or more magnetic disk storage devices, flash memory storage devices. In some embodiments, a non-transitory computer readable storage medium in a memory is used to store at least one program code.

The processor 110 is used for controlling the operation of the computer electronic device 1, and the processor 110 may also be referred to as a Central Processing Unit (CPU). The processor 110 may be an integrated circuit chip having signal processing capabilities. The processor 110 may also be a general purpose processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components. A general purpose processor may be a microprocessor or the processor 110 may be any conventional processor or the like.

The processor 110 is configured to execute a computer program stored in the memory 120 to implement the VMAF image quality evaluation method described in the embodiment of the VMAF image quality evaluation method of the hardware encoder of the present application.

In some embodiments, the computer electronics 1 may further comprise: a peripheral interface 130 and at least one peripheral. The processor 110, memory 120, and peripheral interface 130 may be connected by buses or signal lines. Various peripheral devices may be connected to peripheral interface 130 via a bus, signal line, or circuit board. Specifically, the peripheral device includes: at least one of radio frequency circuitry 140, display 150, audio circuitry 160, and power supply 170.

The peripheral interface 130 may be used to connect at least one peripheral related to I/O (Input/output) to the processor 110 and the memory 120. In some embodiments, processor 110, memory 120, and peripheral interface 130 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 110, the memory 120, and the peripheral interface 130 may be implemented on a separate chip or circuit board, which is not limited in this embodiment.

The Radio Frequency circuit 140 is used for receiving and transmitting RF (Radio Frequency) signals, also called electromagnetic signals. The radio frequency circuit 140 communicates with a communication network and other communication devices via electromagnetic signals. The rf circuit 140 converts an electrical signal into an electromagnetic signal to transmit, or converts a received electromagnetic signal into an electrical signal. Optionally, the radio frequency circuit 140 comprises: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a subscriber identity module card, and so forth. The radio frequency circuit 140 may communicate with other terminals via at least one wireless communication protocol. The wireless communication protocols include, but are not limited to: the world wide web, metropolitan area networks, intranets, generations of mobile communication networks (2G, 3G, 4G, and 5G), Wireless local area networks, and/or WiFi (Wireless Fidelity) networks. In some embodiments, the rf circuit 140 may further include NFC (Near Field Communication) related circuits, which are not limited in this application.

The display 150 is used to display a UI (User Interface). The UI may include graphics, text, icons, video, and any combination thereof. When the display screen 150 is a touch display screen, the display screen 150 also has the ability to capture touch signals on or over the surface of the display screen 150. The touch signal may be input to the processor 110 as a control signal for processing. At this point, the display screen 150 may also be used to provide virtual buttons and/or a virtual keyboard, also referred to as soft buttons and/or a soft keyboard. In some embodiments, the display 150 may be one, provided on the front panel of the computer electronic device 1; in other embodiments, the display screen 150 may be at least two, respectively disposed on different surfaces of the mobile terminal 100 or in a folded design; in other embodiments, the display 150 may be a flexible display disposed on a curved surface or a folded surface of the mobile terminal 100. Even further, the display 150 may be arranged in a non-rectangular irregular pattern, i.e., a shaped screen. The Display 150 may be made of LCD (Liquid Crystal Display), OLED (Organic Light-emitting diode), and the like.

Audio circuitry 160 may include a microphone and a speaker. The microphone is used for collecting sound waves of a user and the environment, converting the sound waves into electric signals, and inputting the electric signals to the processor 110 for processing or inputting the electric signals to the radio frequency circuit 140 to realize voice communication. The microphones may be provided in a plurality for stereo sound collection or noise reduction, and are respectively disposed at different portions of the mobile terminal 100. The microphone may also be an array microphone or an omni-directional pick-up microphone. The speaker is used to convert electrical signals from the processor 110 or the radio frequency circuit 140 into sound waves. The loudspeaker can be a traditional film loudspeaker or a piezoelectric ceramic loudspeaker. When the speaker is a piezoelectric ceramic speaker, the speaker can be used for purposes such as converting an electric signal into a sound wave audible to a human being, or converting an electric signal into a sound wave inaudible to a human being to measure a distance. In some embodiments, audio circuitry 160 may also include a headphone jack.

The power supply 170 is used to supply power to the various components in the computer electronic device 1. The power source 170 may be alternating current, direct current, disposable or rechargeable. When power source 170 comprises a rechargeable battery, the rechargeable battery may be a wired rechargeable battery or a wireless rechargeable battery. The wired rechargeable battery is a battery charged through a wired line, and the wireless rechargeable battery is a battery charged through a wireless coil. The rechargeable battery may also be used to support fast charge technology.

For a detailed description of the functions and execution processes of each functional module or component in the embodiment of the computer electronic device, reference may be made to the description in the embodiment of the VMAF image quality evaluation method of the hardware encoder of the present application, which is not described herein again.

In the embodiments provided in the present application, it should be understood that the disclosed method for evaluating the VMAF image quality of the hardware encoder may be implemented in other manners. For example, the various embodiments of computer electronics described above are merely illustrative, and for example, a division of modules or units is merely a logical division, and an actual implementation may have additional divisions, for example, multiple units or components may be combined or integrated into another system, or some features may be omitted, or not implemented. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.

Units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.

Referring to fig. 9, the integrated unit may be stored in a computer-readable storage medium 300 if it is implemented in the form of a software functional unit and sold or used as a separate product. Based on such understanding, the technical solution of the present application may be substantially implemented or contributed by the prior art, or all or part of the technical solution may be embodied in a software product, which is stored in a storage medium and includes instructions/computer programs for causing a computer device (which may be a personal computer, a server, a network device, or the like) or a processor (processor) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: various media such as a usb disk, a mobile hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and terminal devices such as a computer, a mobile phone, a notebook computer, a tablet computer, and a camera having the storage medium.

For the description of the execution process of the program data in the computer-readable storage medium, reference may be made to the above description of the embodiment of the method for evaluating the VMAF image quality of the hardware encoder of the present application, and details are not repeated herein.

The above description is only an example of the present application and is not intended to limit the scope of the present application, and all modifications of equivalent structures and equivalent processes, which are made by the contents of the specification and the drawings, or which are directly or indirectly applied to other related technical fields, are intended to be included within the scope of the present application.

Claims

1. A VMAF image quality evaluation method is characterized by comprising the following steps:

receiving at least two video files to be coded with different resolutions sent by a host, wherein the at least two video files to be coded with different resolutions are obtained by zooming a preset video source by the host;

calling a hardware encoder to respectively encode the at least two video files to be encoded with different resolutions by using at least one group of video encoding parameters, and correspondingly obtaining at least two encoded video files;

and sending the at least two encoded video files to the host, so that the host performs VMAF image quality evaluation by using the at least two encoded video files and the preset video source.

2. The method of claim 1, wherein:

the calling hardware encoder respectively encodes the at least two video files to be encoded with different resolutions by using at least one group of video encoding parameters, and correspondingly obtains at least two encoded video files, including:

sequentially traversing the at least two video files to be coded with different resolutions;

and calling the hardware encoder to encode each video file to be encoded according to the at least one group of video encoding parameters respectively until the encoding of all the video files to be encoded is completed.

3. The method according to claim 1 or 2, characterized in that:

before the calling hardware encoder uses at least one group of video coding parameters to respectively code the at least two video files to be coded with different resolutions and correspondingly obtains at least two coded video files, the method comprises the following steps:

and acquiring and combining at least one coding mode, at least one frame rate and at least one code rate to obtain at least one group of video coding parameters.

4. The method of claim 3, wherein:

the video files to be coded with at least two different resolutions sent by the receiving host comprise:

receiving the at least two video files to be coded with different resolutions sent by the host through a preset application program;

and automatically calling the hardware encoder through the preset application program in the background to encode the at least two video files to be encoded with different resolutions by using the at least one group of video encoding parameters respectively, and correspondingly obtaining the at least two encoded video files.

5. The method of claim 4, wherein:

after the calling hardware encoder respectively encodes the at least two video files to be encoded with different resolutions by using at least one group of video encoding parameters, and correspondingly obtains the at least two encoded video files, the method includes:

writing the at least two encoded video files into a preset memory;

the sending the at least two encoded video files to the host includes:

and sending the at least two encoded video files in the preset memory to the host.

6. The method of claim 1, wherein:

the video file to be coded and the preset video source are video files in YUV format, and the resolution of the video file to be coded is smaller than or equal to that of the preset video source.

7. A VMAF image quality evaluation method is characterized by comprising the following steps:

sending at least two video files to be coded with different resolutions obtained by zooming a preset video source to a mobile terminal;

receiving at least two coded video files obtained by respectively coding the at least two video files to be coded with different resolutions by the mobile terminal calling hardware coder by utilizing at least one group of video coding parameters;

and performing VMAF image quality evaluation by using the at least two coded video files and the preset video source.

8. The method of claim 7, wherein:

the VMAF picture quality evaluation by utilizing the at least two coded video files and the preset video source comprises the following steps:

decoding the at least two coded video files respectively to obtain at least two decoded video files correspondingly;

respectively scaling the at least two decoded video files to preset resolution, wherein the preset resolution is consistent with the resolution of the preset video source;

and inputting the at least two scaled decoded video files and the preset video source into a VMAF model for image quality evaluation.

9. A mobile terminal, comprising:

the video coding device comprises a receiving module, a decoding module and a processing module, wherein the receiving module is used for receiving at least two video files to be coded with different resolutions sent by a host, and the at least two video files to be coded with different resolutions are obtained by zooming a preset video source by the host;

the coding module is used for calling a hardware coder to respectively code the at least two video files to be coded with different resolutions by utilizing at least one group of video coding parameters to correspondingly obtain at least two coded video files;

and the sending module is used for sending the at least two encoded video files to the host computer so as to evaluate the VMAF image quality by utilizing the at least two encoded video files and the preset video source.

10. A host, comprising:

the device comprises a sending module, a receiving module and a processing module, wherein the sending module is used for sending at least two video files to be coded with different resolutions obtained by zooming a preset video source to a mobile terminal;

a receiving module, configured to receive at least two encoded video files obtained by the mobile terminal calling a hardware encoder to encode the at least two to-be-encoded video files with different resolutions respectively by using at least one group of video encoding parameters;

and the image quality evaluation module is used for performing VMAF image quality evaluation by utilizing the at least two coded video files and the preset video source.

11. A computer electronic device, comprising: a processor, memory, and communication circuitry; the memory and the communication circuit are coupled to the processor, the memory storing a computer program executable by the processor to implement the method of any one of claims 1-8.

12. A VMAF picture quality evaluation system of a hardware encoder is characterized in that: comprising a mobile terminal according to claim 9 and a host according to claim 10, said mobile terminal and said host being communicatively coupled.

13. A computer-readable storage medium, in which a computer program is stored which can be executed by a processor to implement the method according to any one of claims 1 to 8.