CN112866682A - Audio and video quality evaluation system and method, computer equipment and storage medium - Google Patents
Audio and video quality evaluation system and method, computer equipment and storage medium Download PDFInfo
- Publication number
- CN112866682A CN112866682A CN202011633361.1A CN202011633361A CN112866682A CN 112866682 A CN112866682 A CN 112866682A CN 202011633361 A CN202011633361 A CN 202011633361A CN 112866682 A CN112866682 A CN 112866682A
- Authority
- CN
- China
- Prior art keywords
- evaluation
- terminal
- audio
- task
- execution
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000013441 quality evaluation Methods 0.000 title claims abstract description 111
- 238000000034 method Methods 0.000 title claims abstract description 35
- 238000011156 evaluation Methods 0.000 claims abstract description 670
- 230000002452 interceptive effect Effects 0.000 claims description 27
- 230000003287 optical effect Effects 0.000 claims description 22
- 239000013074 reference sample Substances 0.000 claims description 14
- 238000004891 communication Methods 0.000 claims description 8
- 238000001303 quality assessment method Methods 0.000 claims description 8
- 230000007246 mechanism Effects 0.000 claims description 7
- 230000006855 networking Effects 0.000 claims description 6
- 230000005236 sound signal Effects 0.000 claims description 4
- 238000004590 computer program Methods 0.000 claims description 3
- 238000012544 monitoring process Methods 0.000 claims description 3
- 230000008569 process Effects 0.000 abstract description 8
- 230000003993 interaction Effects 0.000 abstract description 2
- 230000002829 reductive effect Effects 0.000 abstract description 2
- 238000012360 testing method Methods 0.000 description 12
- 230000006735 deficit Effects 0.000 description 6
- 230000006870 function Effects 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- 238000005457 optimization Methods 0.000 description 4
- 238000011161 development Methods 0.000 description 3
- 238000004088 simulation Methods 0.000 description 3
- 230000008859 change Effects 0.000 description 2
- 238000012854 evaluation process Methods 0.000 description 2
- 238000009434 installation Methods 0.000 description 2
- 230000036961 partial effect Effects 0.000 description 2
- 230000002860 competitive effect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 230000000670 limiting effect Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 230000008707 rearrangement Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N17/00—Diagnosis, testing or measuring for television systems or their details
- H04N17/004—Diagnosis, testing or measuring for television systems or their details for digital television systems
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- General Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)
Abstract
The embodiment of the invention discloses an audio and video quality evaluation system and method, computer equipment and a storage medium. The system comprises an evaluation service platform and an evaluation execution platform; an evaluation server contained in the evaluation service platform forms an evaluation task of the audio and video quality according to the received evaluation parameters, and adds the evaluation task to a task cache table; and after receiving a target evaluation task correspondingly issued by the evaluation server, each evaluation execution room of the evaluation execution platform acquires corresponding audio and video quality evaluation data through executing the target evaluation task. By utilizing the system, the intelligent generation of the task to be evaluated is realized through the evaluation service platform, the automatic evaluation of the item to be evaluated in the task to be evaluated is realized through the evaluation execution platform, the user only needs to select the parameters required by the evaluation through the front-end interaction interface provided by the evaluation service platform in the whole process, the follow-up evaluation operation is not needed, the operation is simple and easy, the evaluation cost is reduced, and the accuracy of the evaluation result is greatly improved.
Description
Technical Field
The embodiment of the invention relates to the technical field of audio and video, in particular to an audio and video quality evaluation system, an audio and video quality evaluation method, computer equipment and a storage medium.
Background
With the rapid development of computer network technology, application software such as audio and video social contact and the like is more and more popular, and great convenience is brought to life entertainment, learning and working of people. In practical application, the popularity of the application product in the public range is influenced by the good and bad experience of the user on the application product, and for audio and video social application software, the influence on the user experience mainly lies in the audio and video quality in the audio and video call process. Therefore, a software developer can provide help for optimizing the audio and video application software for evaluating the audio and video quality of the audio and video application software in a project development stage.
The indexes needing to be evaluated in the audio and video quality evaluation mainly comprise: audio frequency sound quality average opinion score (MOS), audio frequency end-to-end time delay, audio frequency sending code rate, fluency, definition, terminal resource occupation condition and the like. Currently, the evaluation of the indexes mainly comprises subjective evaluation and objective evaluation, wherein the subjective evaluation needs to manually start audio and video conversation or live broadcast application on a terminal, and then the evaluation personnel scores the indexes through subjective feeling; the objective evaluation is mainly to use corresponding evaluation tools to evaluate each index.
The evaluation mode of the audio and video quality needs a large amount of professional evaluation personnel to participate, is time-consuming, labor-consuming, has various repeated operation steps, and has too high evaluation cost; and partial evaluation results of the audio and video quality evaluation are strong in subjectivity, and valuable reference data cannot be provided for optimization of audio and video application products.
Disclosure of Invention
In view of this, embodiments of the present invention provide an audio and video quality evaluation system, method, computer device, and storage medium, so as to implement effective evaluation of audio and video quality.
In a first aspect, an embodiment of the present invention provides an audio and video quality evaluation system, including: the system comprises an evaluation service platform and an evaluation execution platform;
the evaluation service platform comprises an evaluation server, the evaluation server is used for forming an evaluation task of the audio and video quality according to the received evaluation parameters and adding the evaluation task to a task cache table, wherein the evaluation parameters are selected by a user in a front-end interactive interface provided by the evaluation service platform;
the evaluation execution platform comprises at least one evaluation execution room, and each evaluation execution room is used for executing the target evaluation task to obtain corresponding audio and video quality evaluation data after receiving the target evaluation task correspondingly issued by the evaluation server. .
In a second aspect, an embodiment of the present invention provides an audio and video quality assessment method, which is executed by the audio and video quality assessment system provided in the first aspect of the present invention, and includes:
forming an evaluation task of audio and video quality according to the received evaluation parameters through an evaluation server in an evaluation service platform, and adding the evaluation task to a task cache table, wherein the evaluation parameters are selected by a user in a front-end interactive interface provided by the evaluation service platform;
and after receiving a target evaluation task correspondingly issued by the evaluation server through an evaluation execution room in the evaluation execution platform, executing the target evaluation task to obtain corresponding audio and video quality evaluation data. .
In a third aspect, an embodiment of the present invention provides a computer device, which is used as an evaluation server on an evaluation service platform in an audio and video quality evaluation system provided in the first aspect of the present invention, and is configured to execute corresponding method steps; or, as an evaluation execution terminal in an evaluation execution room on an evaluation execution platform in the audio/video quality evaluation system provided in the embodiment of the first aspect of the present invention, the evaluation execution terminal is configured to execute corresponding method steps.
In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the audio and video quality assessment method provided in the second aspect of the present invention.
In the audio and video quality evaluation system, the audio and video quality evaluation method, the computer equipment and the storage medium provided by the embodiment of the invention, an evaluation service platform in the system comprises an evaluation server, wherein the evaluation server is used for forming an evaluation task of the audio and video quality according to a received evaluation parameter and adding the evaluation task to a task cache table, wherein the evaluation parameter is selected by a user in a front-end interactive interface provided by the evaluation service platform; the evaluation execution platform in the system comprises at least one evaluation execution room, and each evaluation execution room can obtain corresponding audio and video instruction evaluation data through the execution of a target evaluation task after receiving the target evaluation task correspondingly issued by the evaluation server. The audio and video quality evaluation system provided by the embodiment of the invention is equivalent to an integrated audio and video quality evaluation system, and can be used for unifying and integrating the original independent evaluation items relative to the audio and video quality evaluation items; the intelligent generation of the task to be tested is realized through the evaluation service platform, the automatic evaluation of the item to be tested in the task to be tested is realized through the evaluation execution platform, and in the whole process, a user only needs to select parameters required by the evaluation through a front-end interactive interface provided by the evaluation service platform, does not need to participate in subsequent evaluation operation, is simple and easy to operate, does not need to limit professional evaluation personnel to participate, greatly reduces the steps of repeated operation of the evaluation personnel, and better reduces the evaluation cost; meanwhile, the whole audio and video quality evaluation is realized through an evaluation execution platform, so that the artificial subjective evaluation is avoided, the accuracy of the evaluation result is greatly improved, and effective reference data is provided for the optimization of audio and video application products.
Drawings
Fig. 1 is a block diagram of an audio/video quality evaluation system according to a first embodiment of the present invention;
fig. 1a is a diagram showing an architecture of an evaluation execution room in an audio/video quality evaluation system according to an embodiment of the present invention;
fig. 2 is a schematic flow chart of an audio/video quality evaluation method according to a second embodiment of the present invention;
fig. 2a shows an implementation example diagram of implementing audio and video quality evaluation by an audio and video quality evaluation system in the second embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, embodiments of the present invention will be described in detail with reference to the accompanying drawings. It should be understood that the described embodiments are only some embodiments of the invention, and not all embodiments. In addition, the embodiments and features of the embodiments in the present invention may be combined with each other without conflict.
In the description of the present application, it is to be understood that the terms "first," "second," "third," and the like are used solely to distinguish one from another and are not necessarily used to describe a particular order or sequence, nor are they to be construed as indicating or implying relative importance. The specific meaning of the above terms in the present application can be understood by those of ordinary skill in the art as appropriate.
Example one
Fig. 1 shows a structural block diagram of an audio/video quality evaluation system according to a first embodiment of the present invention, where the audio/video quality evaluation system is suitable for evaluating audio/video quality of an audio/video application product, and as shown in fig. 1, the audio/video quality evaluation system includes: the system comprises an evaluation service platform 1 and an evaluation execution platform 2;
the evaluation service platform 1 comprises an evaluation server 11, wherein the evaluation server 11 is used for forming an evaluation task of audio and video quality according to received evaluation parameters and adding the evaluation task to a task cache table 111, and the evaluation parameters are selected by a user in a front-end interactive interface 12 provided by the evaluation service platform 1;
the evaluation execution platform 2 comprises at least one evaluation execution room 21, and each evaluation execution room 21 is used for acquiring corresponding audio/video quality evaluation data through the execution of a target evaluation task correspondingly issued by the evaluation server 11 after receiving the target evaluation task.
In this embodiment, the evaluation service platform 1 may be specifically considered as a platform for providing an evaluation interaction interface for a user and performing management of an evaluation task, and the evaluation execution platform 2 may be specifically considered as an actual evaluation platform for performing audio/video quality evaluation according to the evaluation task issued by the evaluation service platform 1. The user can preferably be a technician with the audio and video application software evaluation requirement.
For the evaluation service platform 1, the system mainly includes a front-end interactive interface 12 for a user to input parameter information to be evaluated or query related evaluation information data, and an evaluation server 11 for generating evaluation tasks according to various parameter information input by the user and performing distribution, scheduling and management on the evaluation tasks. For example, the evaluation parameters input by the user through the front-end interactive interface 12 may include attribute information of the software to be evaluated, such as a software name, evaluation items (such as fluency, definition, end-to-end delay degree, and the like) that need to be evaluated on the software to be evaluated, and network environment parameters (such as a network packet loss rate, a network speed limit, a network jitter frequency, and the like) of a network environment in which each evaluation item is located.
The software to be evaluated provided for the user on the front-end interactive interface 12 may at least include audio/video application software developed by a software developer, and may also include audio/video application software issued by other software developers as a competitive product. The functions which various audio and video application software may have include: real-time audio and video calls, voice chatting, live broadcast services and the like. The task cache list comprises all the created evaluation tasks arranged according to the task creation time.
In this embodiment, it is preferable to construct the evaluation service platform 1 by using a framework combining an open source software project Jenkins and a software development framework spring boot, where Jenkins can be regarded as a continuous integration tool based on object-oriented programming language Java development, and can implement continuous management of an evaluation task (such as creation of an evaluation task, queuing and issuing of a task, stopping of a task, and the like), and both the added spring boot framework and vue.
In this embodiment, the evaluation task may be specifically considered as a software evaluation request including various evaluation parameters selected by the user, the evaluation server 11 may create an evaluation task through the evaluation parameters after receiving the evaluation parameters submitted by the user through the front-end interactive interface 12, and the newly created evaluation task is not directly issued to the evaluation execution platform 2, but is first cached in a preset task cache list to wait for the issuance of the evaluation execution platform 2.
For the evaluation execution platform 2, the evaluation execution platform includes at least one evaluation execution room 21, each evaluation execution room 21 is in communication connection with the evaluation server 11 in the evaluation service platform 1, and each evaluation execution room 21 has an ability to perform audio/video quality evaluation operation, and starts evaluation of audio/video quality based on an evaluation task issued by the evaluation server 11, and can obtain evaluation data related to each evaluation item in the evaluation task in an evaluation process.
In the embodiment, each evaluation execution room 21 in the evaluation execution platform 2 is deployed in a distributed manner, so that each evaluation execution room 21 is independent in a physical space, and the deployment manner can solve the problems of many evaluation requirements, queuing waiting time process of evaluation tasks and the like, and can also effectively cope with the subsequently increased evaluation requirements, thereby providing higher expandability; in addition, in the embodiment, the distributed deployment of each evaluation execution room 21 is an independent deployment in a physical space, so that mutual interference of wireless network signals in different evaluation execution rooms 21 on a channel can be effectively avoided, and interference caused by an evaluation network environment can be further avoided.
It should be noted that, in this embodiment, it is preferable that the evaluation task received by the evaluation execution room 21 is a target evaluation task, and the evaluation server 11 may issue the evaluation tasks in the task cache list to the corresponding evaluation execution platforms 2 one by one through a certain allocation policy. For an evaluation execution room 21, after receiving a target evaluation task, if there is no other target evaluation task waiting to be executed or being executed before the target evaluation task, the execution of the target evaluation task may be directly started, and if there is another target evaluation task before the target evaluation task, the target evaluation task may be cached in a task queue to wait for a corresponding execution opportunity.
The audio and video quality evaluation system provided by the embodiment of the invention is equivalent to an integrated audio and video quality evaluation system, and can be used for unifying and integrating the original individual evaluation items relative to the audio and video quality evaluation items; the intelligent generation of the task to be tested is realized through the evaluation service platform, the automatic evaluation of the item to be tested in the task to be tested is realized through the evaluation execution platform, and in the whole process, a user only needs to select parameters required by the evaluation through a front-end interactive interface provided by the evaluation service platform, does not need to participate in subsequent evaluation operation, is simple and easy to operate, does not need to limit professional evaluation personnel to participate, greatly reduces the steps of repeated operation of the evaluation personnel, and better reduces the evaluation cost; meanwhile, the whole audio and video quality evaluation is realized through an evaluation execution platform, so that the artificial subjective evaluation is avoided, the accuracy of the evaluation result is greatly improved, and effective reference data is provided for the optimization of audio and video application products.
Further, fig. 1a shows a structure of an evaluation execution room in the audio/video quality evaluation system according to an embodiment of the present invention; as shown in fig. 1a, the evaluation execution room 21 includes: an evaluation execution terminal 211, a network switch 212, a network impairment instrument 213, a first router 214, and a second router 215;
the evaluation execution terminal 211 is connected to the network switch 212 through a network cable;
the network damage apparatus 213 is connected to the network switch 212 through a private network and is connected to the first router 214 through a network cable;
the evaluation execution terminal 211 establishes communication with the evaluation server 11, and is configured to receive a target evaluation task issued by the evaluation server 11, and control the network impairment instrument 213 to perform network environment simulation according to network environment parameters in the target evaluation task, so that the first router 214 provides a first wireless network signal with a lossy signal;
the second router 215 is connected to the network switch 212 through a network cable; for providing second wireless network signals with normal signals.
In this embodiment, each evaluation execution room 21 specifically includes an evaluation execution terminal 211 serving as an evaluation execution control device, and the evaluation execution terminal 211 establishes a communication connection with the evaluation server 11 to the outside, and can receive an evaluation task (denoted as an objective evaluation task) issued by the evaluation server 11; in the pair, the evaluation execution terminal 211 is connected to a network switch 212 included in the evaluation execution room 21 to provide a local area network for the evaluation execution room 21 through the network switch 212. In addition, the evaluation execution room 21 further includes a first router 214 and a second router 215, which are respectively connected to the network switch 212 and are used for forming wireless network signals, wherein, in order to simulate a weak network environment, a network impairment instrument 213 disposed between the network switch 212 and the first router 214 is further added in the present embodiment.
Specifically, when the evaluation execution terminal 211 starts an evaluation operation with respect to a target evaluation task, the target evaluation task may be analyzed first to obtain a network environment parameter included in the target evaluation task, so as to control the network impairment instrument 213 to simulate a network signal output of a weak network environment required by the target evaluation task based on the network environment parameter, and the first router 214 may convert the network signal in the weak network environment into a wireless network signal output, where the wireless network signal is recorded as a first wireless network signal, and a problem of the weak network environment simulated by the network impairment instrument 213 may be packet loss, speed limit, or network jitter.
In addition, the present embodiment also sets a wireless network signal in a normal network environment, which is denoted as a second wireless network signal, in the evaluation execution room 21 through the second router 215 connected to the network switch 212.
Further, the evaluation execution room 21 further includes: a first evaluation terminal 216 and a second evaluation terminal 217 connected to the evaluation execution terminal 211 via a Universal Serial Bus (USB);
the first evaluation terminal 216 is in a networking state by establishing a wireless connection with the first router 214, and the second evaluation terminal 217 is in a networking state by establishing a wireless connection with the second router 215, so that the first evaluation terminal 216 has a network communication state matching the network environment parameters.
In this embodiment, in order to realize the evaluation of the audio/video quality, the evaluation execution room 21 needs to include the above network device for performing the network environment simulation, and further includes an evaluation terminal capable of integrating, installing and operating the audio/video application software to be evaluated, and in consideration of that the evaluation of the audio/video quality at least relates to an audio/video sending terminal and an audio/video receiving terminal, a first evaluation terminal 216 and a second evaluation terminal 217 are provided in the evaluation execution room 21 in this embodiment.
Specifically, the first evaluation terminal 216 and the second evaluation terminal 217 may access the first router 214 and the second router 215 respectively through equipped wireless devices, and then are in a networking state, wherein the first evaluation terminal 216 has a network communication state matching with the network environment parameters because a first wireless network signal formed by the first router 214 is limited by the network impairment instrument 213; the second network signal formed by the second router 215 is a normal wireless signal, and the second evaluation office has a network communication state of a normal network environment.
Further, the evaluation execution terminal 211 includes:
and the evaluation configuration module is used for controlling the first evaluation terminal 216 and the second evaluation terminal 217 to download and install the to-be-evaluated software according to the link information of the to-be-evaluated software contained in the target evaluation task, and controlling the first evaluation terminal 216 and the second evaluation terminal 217 to operate the to-be-evaluated software through a pre-deployed evaluation frame so as to enter a to-be-evaluated mode of the target evaluation task.
In this embodiment, before the audio/video evaluation, the first evaluation terminal 216 and the second evaluation terminal 217 need to install software to be evaluated. Specifically, the evaluation execution terminal 211 may obtain link information of the software to be evaluated, which the user desires to evaluate, through analysis of the target evaluation task, so as to download the link information to the installation package of the software to be evaluated; meanwhile, in this embodiment, a system interface automation frame (preferably, an android interface automation frame uiautomator2) is configured on the first evaluation terminal 216 and the second evaluation terminal 217, and the installation package of the software to be evaluated can be automatically installed on the first evaluation terminal 216 and the second evaluation terminal 217 through the system interface automation frame, and the software to be evaluated can be automatically run to enter a to-be-evaluated mode (e.g., an audio/video call mode or an audio/video live broadcast mode) corresponding to the target evaluation task; in this embodiment, it is preferable that the first evaluation terminal 216 is an audio/video transmitting terminal and can transmit an application for audio/video call, and the second evaluation terminal 217 is an audio/video receiving terminal and can receive the application for audio/video call transmitted by the first evaluation terminal 216.
In addition, after the first evaluation terminal 216 and the second evaluation terminal 217 are connected with the evaluation execution terminal 211 through the USB, the evaluation execution terminal 211 can monitor information such as code rate, CPU occupation, and memory occupation when the first evaluation terminal 216 and the second evaluation terminal 217 operate in an evaluation mode, and use the information as one evaluation information in the audio/video quality evaluation; meanwhile, the evaluation execution terminal 211 may also serve as a power supply to supply power to the first evaluation terminal 216 and the second evaluation terminal 217.
Further, the evaluation execution room 21 further includes: a terminal display 218, an optical signal acquisition device 219, and an audio quality tester 210 connected to the evaluation execution terminal 211, respectively;
the optical signal acquisition device 219 is connected with the second evaluation terminal 217;
the audio quality tester 210 is connected to the first evaluation terminal 216 and the second evaluation terminal 217 through audio signal lines, respectively.
In this embodiment, the evaluation execution room 21 further includes an evaluation tool for audio/video quality evaluation, which may be subdivided into an audio quality evaluation tool, such as an audio quality tester 210, preferably a Spirent MOS machine, and a video quality evaluation tool, such as an optical signal acquisition device 219, in addition to a network device required for simulating a network environment for audio/video quality evaluation and an operation carrier (evaluation terminal) of software to be evaluated, and further includes an auxiliary tool for video quality evaluation, such as a terminal display 218.
Specifically, in order to ensure the normal evaluation of the audio/video quality, the receiving end (the second evaluation terminal 217) preferably performing the audio/video operation in this embodiment is connected to the optical signal acquisition device 219, so that the optical signal acquisition device 219 can acquire the light change signal of the video picture presented on the second evaluation terminal 217; and the preferred audio quality tester 210 is respectively connected with the first evaluation terminal 216 and the second evaluation terminal 217 serving as a sending end and a receiving end, so that the audio quality tester 210 can play audio signals to the first evaluation terminal 216 and can also collect the played audio signals from the second evaluation terminal 217 side, wherein the optical signal collection device can be preferably a photosensitive sensor.
The evaluation execution room in the audio and video quality evaluation system provided by the embodiment of the invention integrates various hardware devices required by audio and video quality evaluation, and by erecting various hardware devices and combining the evaluation server 11, the integrated evaluation and the automatic evaluation of the audio and video quality are realized, the evaluation work of technicians is greatly reduced, the evaluation execution steps are simplified, and meanwhile, the erected hardware devices contain accurate testing tools, and the accurate evaluation of evaluation items is realized.
Further, the evaluation execution terminal 211 further includes:
the information determining module is used for determining the to-be-evaluated items relative to the to-be-evaluated software according to the target evaluation task;
the video quality evaluation module is used for determining video quality evaluation data of each video quality evaluation item by controlling the terminal display 218, the first evaluation terminal 216, the second evaluation terminal 217 and the optical signal acquisition device 219 aiming at the video quality evaluation item in the evaluation items to be evaluated;
the audio quality evaluation module is used for determining audio quality evaluation data of each audio quality evaluation item by controlling the first evaluation terminal 216, the second evaluation terminal 217 and the audio quality tester 210 aiming at the audio quality evaluation item in the evaluation items to be tested;
the video quality to-be-evaluated item comprises one of the following items: video fluency, video definition and video end-to-end delay;
the audio quality item to be evaluated comprises one of the following items: mean opinion score and audio end-to-end delay.
In this embodiment, based on the evaluation execution room 21 including the network device, the evaluation terminal, and the evaluation tool, the response and execution of the target evaluation task can be realized. In the specific execution process of the target evaluation task, the target evaluation task can be analyzed through the information determination module, so that an evaluation item to be evaluated, which is expected by a user to evaluate software to be evaluated, is obtained. It can be known that the evaluation item to be evaluated includes a video quality evaluation item and an audio quality evaluation item; in this embodiment, the video quality evaluation module may be used in combination with the terminal display 218, the first evaluation terminal 216, the second evaluation terminal 217, and the optical signal acquisition device 219 to obtain video quality evaluation data corresponding to the video quality item to be evaluated; the audio quality evaluation data of the audio quality item to be evaluated can also be obtained through the audio quality evaluation module in combination with the first evaluation terminal 216, the second evaluation terminal 217 and the audio quality tester 210.
It can be known that before the above modules such as the information determining module, the video quality evaluation module and the audio quality evaluation module operate, a network environment corresponding to a target test task is already constructed through network equipment, and software to be tested is also installed in the two test terminals and automatically operates to enter a mode to be tested. Therefore, after the items to be evaluated (such as video fluency, video definition and video end-to-end delay in the items to be evaluated in video quality, and average opinion score and audio end-to-end delay in the items to be eliminated in audio quality) are determined, the video quality evaluation module and the audio quality evaluation module can be controlled to realize the acquisition of the evaluation data corresponding to the corresponding items to be evaluated.
Further, the video quality evaluation module is specifically configured to:
when the video quality evaluation item is determined to be contained in the evaluation item to be tested, the terminal display 218 is controlled to play a preset video evaluation sequence;
controlling the first evaluation terminal 216 to collect the video evaluation sequence played by the terminal display 218, so that the video evaluation sequence is transmitted from the first evaluation terminal 216 to the second evaluation terminal 217 in the evaluation state to be tested;
receiving fluency and/or definition evaluation data fed back by the second evaluation terminal 217 relative to the video evaluation sequence; and/or the presence of a gas in the gas,
and controlling the optical signal acquisition device 219 to receive the corresponding optical change signal when the second evaluation terminal 217 plays the video evaluation sequence, and determining video end-to-end delay evaluation data through the pulse signal fed back by the optical signal acquisition device 219.
In this embodiment, the evaluation execution terminal 211 includes a preset video evaluation sequence, and when a playing condition is satisfied, the video evaluation sequence may be presented through the terminal display 218, and the first evaluation terminal 216 may capture the video evaluation sequence through a camera provided therein (capture of a scene picture at a transmitting end side in an analog audio/video call scene); the second evaluation terminal 217 may receive the video evaluation sequence and perform presentation of the video content.
It can be known that, the second evaluation terminal 217 integrates a collection and analysis tool for video fluency and/or definition, and after receiving the video test sequence, the collection and analysis tool can determine the fluency and/or definition of the video test sequence in the network environment, and this embodiment records the fluency and/or definition as fluency and/or definition, and the second evaluation terminal 217 feeds back the fluency and/or definition to the evaluation execution terminal 211 through the USB data line.
In addition, when the video quality evaluation item further includes an evaluation item to be evaluated of video end-to-end delay, the video evaluation sequence picture output by the second evaluation terminal 217 may be captured by the optical signal acquisition device 219 in the form of a light variation signal, the optical signal acquisition device 219 may convert the light variation signal into a pulse signal which may be transmitted to the evaluation execution terminal 211, where the pulse signal may preferably be a current signal, and the evaluation execution terminal 211 may determine a duration of the video end-to-end delay according to the current signal and may determine the duration as video end-to-end delay evaluation data.
It should be noted that, for determining the video end-to-end delay evaluation data, the implementation process of the evaluation execution terminal 211 may be expressed as: analyzing the pulse signal fed back by the optical signal acquisition device 219 to obtain a characteristic value sequence and a receiving time stamp of each characteristic value (the second evaluation terminal 217 receives the time stamp of the video evaluation sequence); and comparing the receiving time stamp of each characteristic value with the sending time stamp of the standard characteristic value sequence corresponding to the video evaluation sequence, so as to determine the end-to-end video delay.
Further, the audio quality evaluation module is specifically configured to:
when the audio quality evaluation item is determined to contain the audio quality evaluation item, the audio quality tester 210 is controlled to play a preset audio reference sample to the first evaluation terminal 216, so that the audio reference sample is transmitted from the first evaluation terminal 216 to the second evaluation terminal 217 in the evaluation state to be tested;
controlling the audio quality tester 210 to record the audio reference sample played by the second evaluation terminal 217;
the mean opinion score and/or audio end-to-end delay fed back by the audio quality tester 210 with respect to the audio reference sample is received.
In this embodiment, the evaluation execution terminal 211 further includes an audio reference sample, the evaluation execution terminal 211 may transmit the audio reference sample to the first evaluation terminal 216 through the audio quality tester 210 for playing, and the audio reference sample may be transmitted to the second evaluation terminal 217 through the first evaluation terminal 216, so that the evaluation execution terminal 211 may control the audio quality tester 210 to record the audio reference sample played by the second evaluation terminal 217 again. The audio quality tester 210 can determine the evaluation data (such as the mean opinion score and/or the audio end-to-end delay) related to the audio quality in the software to be evaluated through the audio reference sample sent to the first evaluation terminal 216 and the audio reference sample collected from the second evaluation terminal 217, wherein the audio quality tester 210 can determine the audio end-to-end delay through the perceptual objective hearing quality evaluation POLQA algorithm integrated thereon.
It is to be understood that the fluency, clarity, etc. of the audio in the audio quality can also be directly analyzed and obtained by the second evaluation terminal 217 through the related evaluation software configured thereon, and further fed back to the evaluation execution terminal 211.
Further, the evaluation execution terminal 211 further includes:
the terminal performance evaluation module is configured to receive one of the following parameter information when the first evaluation terminal 216 and the second evaluation terminal 217 are in an audio/video call state, when the performance evaluation item is included in the to-be-evaluated item: the coding/decoding rate of the terminal, the processor occupancy rate and the memory occupancy rate of the terminal.
In this embodiment, the CPU resources and the memory resources occupied by the first evaluation terminal 216 and the second evaluation terminal 217 when running the software to be evaluated may be obtained through monitoring and analyzing related software configured on the corresponding evaluation terminals, and the obtained parameter information may be used to evaluate the compatibility of the software to be evaluated on the execution terminal.
Further, the front-end interactive interface 12 of the evaluation service platform 1 includes: the method comprises the following steps that a software to be evaluated is selected to be listed, a network environment configuration item, an evaluation item selection list, an evaluation terminal selection list and an evaluation information search entry are selected;
the evaluation parameters at least comprise: the software link information to be evaluated, the network environment parameters, the evaluation items to be evaluated and the evaluation terminal parameters.
In this embodiment, the front-end interactive interface 12 in the evaluation service platform 1 is equivalent to an operable interface for a user to perform evaluation, and the operable interface may include a software list of software to be evaluated, which is used for the user to select software to be evaluated, a network environment configuration item required by the network environment configuration, and an evaluation item selection list, which is also included, and provides a selection item of a test item related to the software to be evaluated for the user.
Meanwhile, the operable interface also comprises an evaluation terminal selection list, and the evaluation terminal selection list comprises terminal configuration information of each test terminal equipped in the evaluation execution room 21; it is to be noted that there are different models of test terminals or hardware devices provided in different evaluation execution rooms 21, and when a user wants to determine the operation performance of software to be evaluated on different models or hardware device type terminals, a desired test terminal can be selected.
In this embodiment, the evaluation information search entry corresponds to a query entry provided for the user, and the user can query the running state of the relevant test task or information such as evaluation data through the query entry. In addition, in this embodiment, the user may obtain each selected evaluation parameter by selecting different options in the front-end interactive interface 12, and specifically may include link information of software to be evaluated, network environment parameters (jitter, bandwidth, packet loss rate, or the like), an evaluation item to be evaluated (audio/video fluency, definition, or end-to-end delay, or the like), and evaluation terminal parameters (a model and a model number of a hard device that the evaluation terminal should have, or the like).
Further, the evaluation server 11 is further configured to:
determining an evaluation task to be issued from the task cache table 111 through a set task issuing mechanism;
according to the evaluation terminal parameters included in the to-be-issued evaluation task, the evaluation execution chamber 21 matched with the to-be-issued evaluation task is determined from the evaluation execution platform 2, and the to-be-issued task is taken as a target evaluation task and is issued to the evaluation execution chamber 21.
In this embodiment, the evaluation server 11 may cache each created evaluation task in the task cache table 111, and then may use a preset task issuing mechanism (such as a time sequence issuing mechanism, a test terminal matching issuing mechanism, and the like) to issue the evaluation task; in this embodiment, the preferred task issuing mechanism is a combination of matching issuing of the test terminal and time sequence issuing. Specifically, the evaluation server 11 may select, according to the time sequence, a current evaluation task to be issued from the task cache table 111; then, the evaluation terminal parameters included in the to-be-issued evaluation task may be analyzed, and then it may be determined in which evaluation execution room 21 the evaluation terminal on the evaluation execution platform 2 that matches the evaluation terminal parameters exists (it is preferable that a terminal combination that can include terminal devices popular in the market in each evaluation execution room 21 of the evaluation execution platform 2), and finally the to-be-issued task is issued as a target evaluation task to the evaluation execution room 21 that includes the matching evaluation terminal.
Further, the evaluation service platform 1 further includes: a database 13;
the evaluation server 11 is further configured to:
monitoring a current execution state corresponding to the issued target evaluation task, and storing the current execution state and the target evaluation task as a first association item to the database 13;
and receiving audio and video quality evaluation data fed back by the evaluation execution room 21 relative to the target evaluation task, and storing the audio and video quality evaluation data and the target evaluation task as a second association item in the database 13.
In the embodiment, a database 13 is further optimally added to the evaluation service platform 1, and the database 13 is preferably MySQL. In addition to the above, the present embodiment may further limit the function of the evaluation server 11 to store the target evaluation task and the current state of the task in the database 13 in a related manner. Specifically, the evaluation server 11 may monitor the status of each target evaluation task received in the evaluation execution room 21 on the evaluation execution platform 2, so as to determine which evaluation task is in a waiting state and which evaluation task is in a processing state; thereafter, the evaluation server 11 may store the obtained current execution state of each target evaluation task in association with the corresponding target evaluation task to the database 13.
Similarly, in this embodiment, the audio/video quality evaluation data fed back by the evaluation execution room 21 with respect to the target evaluation task being processed may also be received in real time, and the audio/video quality evaluation data may also be stored in the database 13 in association with the target evaluation task.
Further, the evaluation server 11 is further configured to:
when receiving the evaluation state query request, determining a state query result corresponding to the evaluation state query request and feeding the state query result back to the front-end interactive interface 12 for display;
when receiving the query request of the evaluation result, determining the state query result corresponding to the query request of the evaluation result and feeding the state query result back to the front-end interactive interface 12 for display;
the evaluation state query request and the evaluation result query request are generated by a user through an evaluation information search entry in the front-end interactive interface 12.
In this embodiment, corresponding to the information storage function of the evaluation server 11, the evaluation server 11 may execute a corresponding query function and give a corresponding information feedback when receiving a query request. The received query request may be generated by responding to information edited by the user by evaluating the information search portal.
Example two
Fig. 2 is a schematic flow chart of an audio/video quality evaluation method provided in the second embodiment of the present invention, and the method is suitable for evaluating audio/video quality of an audio/video application product, and can be specifically executed by the audio/video quality evaluation system provided in the first embodiment of the present invention.
As shown in fig. 2, the audio/video quality evaluation method provided in the second embodiment of the present invention may specifically include the following steps:
s201, forming an evaluation task of audio and video quality according to the received evaluation parameters through an evaluation server in an evaluation service platform, and adding the evaluation task to a task cache table.
And the evaluation parameters are selected by a user in a front-end interactive interface provided by the evaluation service platform.
Illustratively, the front-end interactive interface can be used for a user to input information of parameters to be evaluated or query data of related evaluation information; the evaluation server can generate evaluation tasks according to various parameter information input by a user and carry out distribution scheduling management on the evaluation tasks; the evaluation task can be specifically regarded as a software evaluation request containing various evaluation parameters selected by the user; the evaluation server can create an evaluation task through the evaluation parameters submitted by the user through the front-end interactive interface after receiving the evaluation parameters, and the evaluation task can be cached in a task cache table in advance so as to be issued as a target evaluation task to the corresponding evaluation execution after meeting the conditions set by the task issuing mechanism.
S202, after receiving a target evaluation task issued correspondingly by the evaluation server through an evaluation execution room in the evaluation execution platform, executing the target evaluation task to obtain corresponding audio and video quality evaluation data. .
Illustratively, the evaluation execution room is in communication connection with an evaluation server in the evaluation service platform, starts evaluation of audio and video quality based on an evaluation task issued by the evaluation server, and can obtain evaluation data related to each evaluation item in the evaluation task in the evaluation process.
According to the audio and video evaluation method provided by the embodiment of the invention, the intelligent generation of the task to be evaluated is realized through the evaluation service platform, the automatic evaluation of the item to be evaluated in the task to be evaluated is realized through the evaluation execution platform, and in the whole process, a user only needs to select parameters required by the evaluation through a front-end interactive interface provided by the evaluation service platform, does not need to participate in subsequent evaluation operation, is simple and easy to operate, does not need to limit professional evaluation personnel to participate, greatly reduces the steps of repeated operation of the evaluation personnel, and better reduces the evaluation cost; meanwhile, the whole audio and video quality evaluation is realized through an evaluation execution platform, so that the artificial subjective evaluation is avoided, the accuracy of the evaluation result is greatly improved, and effective reference data is provided for the optimization of audio and video application products.
In order to better understand the method implementation of the audio/video evaluation system provided by the embodiment, the embodiment is exemplified by the following description. Fig. 2a shows an implementation example diagram of implementing audio and video quality evaluation by an audio and video quality evaluation system in the second embodiment of the present invention. As shown in fig. 2a, each execution subject participating in audio/video quality evaluation is given, and specifically includes: the front-end interactive interface 31, the evaluation server 32 and the evaluation execution terminal 33, through each execution main body, the execution flow of the audio and video quality evaluation is as follows:
and S1, the front-end interactive interface acquires the link information of the software to be evaluated, the network environment parameters, the items to be evaluated and the evaluation terminal parameters selected by the user, forms evaluation parameters and submits the evaluation parameters to the evaluation server.
And S2, the evaluation server creates an evaluation task according to the received evaluation parameters and stores the evaluation task in a task cache table.
And S3, the evaluation server selects the current evaluation task to be issued from the task cache table, and issues the current evaluation task as the target evaluation task to the corresponding evaluation execution room.
And S4, the evaluation execution room acquires network environment parameters from the target evaluation task, controls the network damage instrument to simulate the network environment according to the network environment parameters, and controls the first evaluation terminal and the second evaluation terminal to be in a networking state.
And S5, the evaluation execution room controls the first evaluation terminal and the second evaluation terminal to download and install the to-be-evaluated software from the link information of the to-be-evaluated software contained in the target evaluation task, and controls the first evaluation terminal and the second evaluation terminal to operate the to-be-evaluated software through the pre-deployed evaluation framework so as to enter the to-be-evaluated mode of the target evaluation task.
And S6, the evaluation execution room determines the to-be-evaluated items of the to-be-evaluated software in the target evaluation task.
And S7, the evaluation execution room determines the video quality evaluation data of each video quality item to be evaluated through controlling the terminal display, the first evaluation terminal, the second evaluation terminal and the optical signal acquisition device.
And S8, the evaluation execution room determines the audio quality evaluation data of each audio quality item to be evaluated through controlling the first evaluation terminal, the second evaluation terminal and the audio quality tester.
Wherein, either S7 or S8, or both S7 and S8 may be selectively performed according to the actual contents of the item to be evaluated.
And S9, the evaluation server receives evaluation data containing the video quality evaluation data and/or the audio quality evaluation data and stores the evaluation data in a database in association with the target evaluation task.
Accordingly, when only S7 is executed in the actual application, only the video quality evaluation data can be received; only the audio quality evaluation data can be received when only S8 is executed; when both S7 and S8 are performed, video quality assessment data and audio quality assessment data may be received.
And S10, when the evaluation server receives the evaluation result query request, determining a state query result corresponding to the evaluation result query request and feeding the state query result back to the front-end interactive interface for display.
EXAMPLE III
The third embodiment of the present invention provides a computer device, which can be used as an evaluation server in an evaluation service platform in an audio and video quality evaluation system provided in the first embodiment of the present invention, and can execute method steps corresponding to the evaluation server, for example, an evaluation task of audio and video quality is formed according to a received evaluation parameter, and the evaluation task is added to a task cache table, wherein the evaluation parameter is selected by a user in a front-end interactive interface provided by the evaluation service platform.
In addition, the computer device may also be used as an evaluation execution terminal in each evaluation execution room in the evaluation execution platform in the audio/video quality evaluation system provided in the embodiment of the present invention, and may execute the method steps corresponding to the evaluation server, for example, obtaining a network environment parameter from a target evaluation task issued by the evaluation server, and controlling the network damage instrument to perform network environment simulation according to the network environment parameter, so that the first router provides a first wireless network signal with a damaged signal.
The third embodiment of the present invention further provides a computer-readable storage medium, where when a processor of a computer device executes a program in the storage medium, the computer device can execute the audio and video quality evaluation method according to the third embodiment. Illustratively, the audio/video quality evaluation method according to the above embodiment includes: forming an evaluation task of audio and video quality according to the received evaluation parameters through an evaluation server in an evaluation service platform, and adding the evaluation task to a task cache table, wherein the evaluation parameters are selected by a user in a front-end interactive interface provided by the evaluation service platform;
and after receiving a target evaluation task correspondingly issued by the evaluation server through an evaluation execution room in the evaluation execution platform, executing the target evaluation task to obtain corresponding audio and video quality evaluation data. .
It should be noted that, as for the embodiments of the apparatus, the computer device, and the storage medium, since they are basically similar to the embodiments of the method, the description is relatively simple, and for the relevant points, reference may be made to the partial description of the embodiments of the method, and the advantageous effects of the embodiments of the method may also be achieved.
From the above description of the embodiments, it is obvious for those skilled in the art that the present invention can be implemented by software and necessary general hardware, and certainly, can also be implemented by hardware, but the former is a better embodiment in many cases. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, and the computer software product may be stored in a computer-readable storage medium, such as a floppy disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a FLASH Memory (FLASH), a hard disk or an optical disk of a computer, and includes several instructions to enable a computer device (which may be a robot, a personal computer, a server, or a network device) to execute the audio/video quality evaluation method according to any embodiment of the present invention.
It should be noted that, in the above audio/video quality evaluation system, each unit and each module included in the system are only divided according to functional logic, but are not limited to the above division, as long as corresponding functions can be realized; in addition, specific names of the functional units are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present invention.
It should be understood that portions of the present invention may be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, the various steps or methods may be implemented in software or firmware stored in memory and executed by suitable instruction execution devices. For example, if implemented in hardware, as in another embodiment, any one or combination of the following techniques, which are known in the art, may be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application specific integrated circuit having an appropriate combinational logic gate circuit, a Programmable Gate Array (PGA), a Field Programmable Gate Array (FPGA), or the like.
It is to be noted that the foregoing is only illustrative of the preferred embodiments of the present invention and the technical principles employed. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, although the present invention has been described in greater detail by the above embodiments, the present invention is not limited to the above embodiments, and may include other equivalent embodiments without departing from the spirit of the present invention, and the scope of the present invention is determined by the scope of the appended claims.
Claims (13)
1. An audio/video quality evaluation system characterized by comprising: the system comprises an evaluation service platform and an evaluation execution platform;
the evaluation service platform comprises an evaluation server, the evaluation server is used for forming an evaluation task of the audio and video quality according to the received evaluation parameters and adding the evaluation task to a task cache table, wherein the evaluation parameters are selected by a user in a front-end interactive interface provided by the evaluation service platform;
the evaluation execution platform comprises at least one evaluation execution room, and each evaluation execution room is used for executing the target evaluation task to obtain corresponding audio and video quality evaluation data after receiving the target evaluation task correspondingly issued by the evaluation server.
2. The system of claim 1, wherein the evaluation execution room comprises: the system comprises an evaluation execution terminal, a network switch, a network damage instrument, a first router and a second router;
the evaluation execution terminal is connected with the network switch through a network cable;
the network damage instrument is connected with the network switch through a private network and is connected with the first router through a network cable;
the evaluation execution terminal is communicated with the evaluation server and used for acquiring network environment parameters from a target evaluation task issued by the evaluation server and controlling the network damage instrument to simulate the network environment according to the network environment parameters so that the first router provides a first wireless network signal with a damaged signal;
the second router is connected with the network switch through a network cable; for providing second wireless network signals with normal signals.
3. The system of claim 2, wherein the evaluation execution room further comprises: the first evaluation terminal and the second evaluation terminal are connected with the evaluation execution terminal through a Universal Serial Bus (USB);
the first evaluation terminal is in a networking state by establishing wireless connection with the first router, and the second evaluation terminal is in a networking state by establishing wireless connection with the second router, so that the first evaluation terminal has a network communication state matched with the network environment parameters.
4. The system according to claim 3, wherein the evaluation execution terminal includes:
and the evaluation configuration module is used for controlling the first evaluation terminal and the second evaluation terminal to download and install the software to be evaluated according to the link information of the software to be evaluated contained in the target evaluation task, and controlling the first evaluation terminal and the second evaluation terminal to operate the software to be evaluated through a pre-deployed evaluation frame so as to enter a to-be-evaluated mode of the target evaluation task.
5. The system of claim 4, wherein the evaluation execution room further comprises: the terminal display, the optical signal acquisition device and the audio quality tester are respectively connected with the evaluation execution terminal;
the optical signal acquisition device is connected with the second evaluation terminal;
the audio quality tester is respectively connected with the first evaluation terminal and the second evaluation terminal through audio signal lines.
6. The system according to claim 5, wherein the evaluation execution terminal further comprises:
the information determination module is used for determining the to-be-evaluated item relative to the to-be-evaluated software according to the target evaluation task;
the video quality evaluation module is used for determining video quality evaluation data of each video quality evaluation item through controlling the terminal display, the first evaluation terminal, the second evaluation terminal and the optical signal acquisition device aiming at the video quality evaluation item in the evaluation items;
the audio quality evaluation module is used for determining audio quality evaluation data of each audio quality evaluation item through controlling the first evaluation terminal, the second evaluation terminal and the audio quality tester aiming at the audio quality evaluation item in the evaluation items;
the terminal performance evaluation module is used for receiving one of the following parameter information when the first evaluation terminal and the second evaluation terminal are in an audio and video call state when the to-be-evaluated item comprises a performance evaluation item: encoding/decoding rate of the terminal, processor occupancy rate and memory occupancy rate of the terminal;
the video quality evaluation item comprises one of the following items: video fluency, video definition and video end-to-end delay;
the audio quality evaluation item comprises one of the following items: mean opinion score and audio end-to-end delay.
7. The system of claim 6, wherein the video quality assessment module is specifically configured to:
when the video quality to-be-evaluated item is determined to be contained in the to-be-evaluated item, controlling the terminal display to play a preset video evaluation sequence;
controlling the first evaluation terminal to collect a video evaluation sequence played by the terminal display so that the video evaluation sequence is transmitted from the first evaluation terminal to the second evaluation terminal in the state to be evaluated;
receiving fluency and/or definition evaluation data fed back by the second evaluation terminal relative to the video evaluation sequence; and/or the presence of a gas in the gas,
and controlling the optical signal acquisition device to receive the corresponding optical variation signal when the second evaluation terminal plays the video evaluation sequence, and determining video end-to-end delay evaluation data through the pulse signal fed back by the optical signal acquisition device.
8. The system of claim 6, wherein the audio quality assessment module is specifically configured to:
when the audio quality item to be evaluated is determined to contain the audio quality item to be evaluated, controlling the audio quality tester to play a preset audio reference sample to the first evaluation terminal, so that the audio reference sample is transmitted to the second evaluation terminal from the first evaluation terminal in the state to be evaluated;
controlling the audio quality tester to record an audio reference sample played by the second evaluation terminal;
receiving a mean opinion score and/or an audio end-to-end delay of the audio quality tester feedback relative to the audio reference sample.
9. The system according to any one of claims 1-8, wherein the evaluation server is further configured to:
determining an evaluation task to be issued from the task cache table through a set task issuing mechanism;
and determining an evaluation execution room matched with the evaluation task to be issued from the evaluation execution platform according to the evaluation terminal parameters included in the evaluation task to be issued, and issuing the evaluation execution room with the evaluation task to be issued as a target evaluation task.
10. The system according to any one of claims 1 to 8, wherein the evaluation service platform further comprises: a database;
the evaluation server is further configured to:
monitoring a current execution state corresponding to the issued target evaluation task, and storing the current execution state and the target evaluation task as a first association item to the database;
and receiving audio and video quality evaluation data fed back by an evaluation execution room relative to the target evaluation task, and storing the audio and video quality evaluation data and the target evaluation task in the database as a second association item.
11. An audio-visual quality evaluation method, performed by the system of any one of claims 1-10, comprising:
forming an evaluation task of audio and video quality according to the received evaluation parameters through an evaluation server in an evaluation service platform, and adding the evaluation task to a task cache table, wherein the evaluation parameters are selected by a user in a front-end interactive interface provided by the evaluation service platform;
and after receiving a target evaluation task correspondingly issued by the evaluation server through an evaluation execution room in the evaluation execution platform, executing the target evaluation task to obtain corresponding audio and video quality evaluation data.
12. A computer device, being an evaluation server on an evaluation service platform in a system according to any of claims 1-10, for performing the corresponding method steps; or,
an evaluation execution terminal in an evaluation execution room on an evaluation execution platform in the system of any one of claims 2 to 10, for executing the corresponding method steps.
13. A computer storage medium on which a computer program is stored, characterized in that the program, when being executed by a processor, implements the audio-video quality assessment method according to claim 11.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011633361.1A CN112866682A (en) | 2020-12-31 | 2020-12-31 | Audio and video quality evaluation system and method, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011633361.1A CN112866682A (en) | 2020-12-31 | 2020-12-31 | Audio and video quality evaluation system and method, computer equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112866682A true CN112866682A (en) | 2021-05-28 |
Family
ID=76000037
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011633361.1A Pending CN112866682A (en) | 2020-12-31 | 2020-12-31 | Audio and video quality evaluation system and method, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112866682A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113691799A (en) * | 2021-08-11 | 2021-11-23 | 广州华多网络科技有限公司 | Live stream interaction control method and corresponding device, equipment and medium |
| CN114071232A (en) * | 2021-11-04 | 2022-02-18 | 广州华多网络科技有限公司 | Audio and video quality visualization method and device, equipment, medium and product thereof |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6965597B1 (en) * | 2001-10-05 | 2005-11-15 | Verizon Laboratories Inc. | Systems and methods for automatic evaluation of subjective quality of packetized telecommunication signals while varying implementation parameters |
| US20120192242A1 (en) * | 2011-01-21 | 2012-07-26 | Wolfgang Kellerer | Method and evaluation server for evaluating a plurality of videos |
| CN102724544A (en) * | 2012-06-18 | 2012-10-10 | 清华大学 | System and method for subjective assessment of stereoscopic videos |
| CN203165007U (en) * | 2013-03-06 | 2013-08-28 | 张乔木 | Science and technology project evaluation system |
| JP2015050561A (en) * | 2013-08-30 | 2015-03-16 | ソフトバンクモバイル株式会社 | Voice quality measurement system |
| CN106780154A (en) * | 2017-01-23 | 2017-05-31 | 国网山东省电力公司电力科学研究院 | The electricity transmitting and converting construction process environmental practice monitoring system and method for multithreading information fusion |
| CN107133157A (en) * | 2017-04-11 | 2017-09-05 | 广东网金控股股份有限公司 | One kind automation cloud platform ease for use assessment method and system |
| CN109347726A (en) * | 2018-11-06 | 2019-02-15 | 西安海润通信技术有限公司 | A kind of surfing Internet with cell phone network aware quality assessment method based on wechat public platform |
| CN110505352A (en) * | 2018-05-17 | 2019-11-26 | 腾讯科技（深圳）有限公司 | Call quality test method, system, computer equipment and computer storage medium |
| CN111782543A (en) * | 2020-07-20 | 2020-10-16 | 王天宝 | Method, related device and system for evaluating in cloud |
-
2020
- 2020-12-31 CN CN202011633361.1A patent/CN112866682A/en active Pending
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6965597B1 (en) * | 2001-10-05 | 2005-11-15 | Verizon Laboratories Inc. | Systems and methods for automatic evaluation of subjective quality of packetized telecommunication signals while varying implementation parameters |
| US20120192242A1 (en) * | 2011-01-21 | 2012-07-26 | Wolfgang Kellerer | Method and evaluation server for evaluating a plurality of videos |
| CN102724544A (en) * | 2012-06-18 | 2012-10-10 | 清华大学 | System and method for subjective assessment of stereoscopic videos |
| CN203165007U (en) * | 2013-03-06 | 2013-08-28 | 张乔木 | Science and technology project evaluation system |
| JP2015050561A (en) * | 2013-08-30 | 2015-03-16 | ソフトバンクモバイル株式会社 | Voice quality measurement system |
| CN106780154A (en) * | 2017-01-23 | 2017-05-31 | 国网山东省电力公司电力科学研究院 | The electricity transmitting and converting construction process environmental practice monitoring system and method for multithreading information fusion |
| CN107133157A (en) * | 2017-04-11 | 2017-09-05 | 广东网金控股股份有限公司 | One kind automation cloud platform ease for use assessment method and system |
| CN110505352A (en) * | 2018-05-17 | 2019-11-26 | 腾讯科技（深圳）有限公司 | Call quality test method, system, computer equipment and computer storage medium |
| CN109347726A (en) * | 2018-11-06 | 2019-02-15 | 西安海润通信技术有限公司 | A kind of surfing Internet with cell phone network aware quality assessment method based on wechat public platform |
| CN111782543A (en) * | 2020-07-20 | 2020-10-16 | 王天宝 | Method, related device and system for evaluating in cloud |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113691799A (en) * | 2021-08-11 | 2021-11-23 | 广州华多网络科技有限公司 | Live stream interaction control method and corresponding device, equipment and medium |
| CN113691799B (en) * | 2021-08-11 | 2023-07-04 | 广州华多网络科技有限公司 | Live stream interaction control method and corresponding device, equipment and medium thereof |
| CN114071232A (en) * | 2021-11-04 | 2022-02-18 | 广州华多网络科技有限公司 | Audio and video quality visualization method and device, equipment, medium and product thereof |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110751940B (en) | Method, device, equipment and computer storage medium for generating voice packet | |
| CN111341296B (en) | Voice control response test method, tester and storage medium | |
| CN112866682A (en) | Audio and video quality evaluation system and method, computer equipment and storage medium | |
| CN110782962A (en) | Hearing language rehabilitation device, method, electronic equipment and storage medium | |
| CN113923441A (en) | Video quality evaluation method and device and electronic equipment | |
| JP2000196705A (en) | Automatic communication protocol test system having message/sequence compilation function and tet method | |
| CN110265004B (en) | Control method and device for target terminal in intelligent home operating system | |
| CN110503982B (en) | Voice quality detection method and related device | |
| CN113823334B (en) | Environment simulation method applied to vehicle-mounted equipment, related device and equipment | |
| CN108197213A (en) | Action performs method, apparatus, storage medium and electronic device | |
| CN111093069A (en) | Quality evaluation method and device for panoramic video stream | |
| CN116743958B (en) | Integrated operation and maintenance method and system for integrating management and control data of audio and video conference system | |
| CN111338865A (en) | Test method, test device and computer readable storage medium | |
| CN112866732A (en) | Music broadcasting method and device, equipment and medium thereof | |
| US10381008B1 (en) | Voice-based interactive network monitor | |
| CN113596381A (en) | Audio data acquisition method and device | |
| CN114019921B (en) | Production line equipment control method, device, equipment and medium based on combined model | |
| CN115623252B (en) | Method, device and storage medium for controlling restarting and stream pushing of automatic detection of online examination | |
| CN113691799B (en) | Live stream interaction control method and corresponding device, equipment and medium thereof | |
| CN109831353B (en) | Automatic testing device for multiple network services | |
| CA2502216A1 (en) | Network monitoring system | |
| JP2019074865A (en) | Conversation collection device, conversation collection system, and conversation collection method | |
| CN113628611A (en) | Voice service test system, method, device and equipment | |
| CN117292701A (en) | Audio information processing method and device, storage medium and electronic equipment | |
| CN111083137B (en) | Method and device for adjusting state of terminal equipment of Internet of things and operating system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |