CN111355949A - Method for constructing audio and video multimedia database and evaluating subjective quality of multimedia - Google Patents
Method for constructing audio and video multimedia database and evaluating subjective quality of multimedia Download PDFInfo
- Publication number
- CN111355949A CN111355949A CN202010172382.1A CN202010172382A CN111355949A CN 111355949 A CN111355949 A CN 111355949A CN 202010172382 A CN202010172382 A CN 202010172382A CN 111355949 A CN111355949 A CN 111355949A
- Authority
- CN
- China
- Prior art keywords
- audio
- video
- multimedia
- video multimedia
- subjective quality
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 36
- 238000012360 testing method Methods 0.000 claims abstract description 78
- 238000013441 quality evaluation Methods 0.000 claims abstract description 58
- 238000012549 training Methods 0.000 claims abstract description 23
- 238000011156 evaluation Methods 0.000 claims abstract description 9
- 238000001303 quality assessment method Methods 0.000 claims description 16
- 230000000007 visual effect Effects 0.000 claims description 16
- 230000006835 compression Effects 0.000 claims description 8
- 238000007906 compression Methods 0.000 claims description 8
- 230000002159 abnormal effect Effects 0.000 claims description 7
- 238000012545 processing Methods 0.000 claims description 6
- 230000002123 temporal effect Effects 0.000 claims description 4
- 238000012935 Averaging Methods 0.000 claims description 2
- 238000006243 chemical reaction Methods 0.000 claims description 2
- 238000012805 post-processing Methods 0.000 abstract description 2
- 238000010586 diagram Methods 0.000 description 3
- 230000005236 sound signal Effects 0.000 description 3
- 238000010276 construction Methods 0.000 description 2
- 238000012074 hearing test Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 239000012925 reference material Substances 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 239000000463 material Substances 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N17/00—Diagnosis, testing or measuring for television systems or their details
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/233—Processing of audio elementary streams
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
- H04N21/23418—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/439—Processing of audio elementary streams
- H04N21/4394—Processing of audio elementary streams involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/44008—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/47—End-user applications
- H04N21/475—End-user interface for inputting end-user data, e.g. personal identification number [PIN], preference data
- H04N21/4756—End-user interface for inputting end-user data, e.g. personal identification number [PIN], preference data for rating content, e.g. scoring a recommended movie
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- General Health & Medical Sciences (AREA)
- Human Computer Interaction (AREA)
- Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)
Abstract
The invention provides a method for constructing an audio and video multimedia database and evaluating the subjective quality of multimedia, which comprises the steps of establishing a joint quality subjective evaluation audio and video multimedia database specially used for audio and video multimedia, training a tester by building an audio and video multimedia quality evaluation environment and selecting data in the audio and video multimedia database for training, testing selected data to be tested from the audio and video multimedia database for testing and performing post-processing on a subjective quality evaluation result obtained by testing based on the audio and video multimedia database, thereby realizing the standardized and streamlined audio and video multimedia subjective quality evaluation.
Description
Technical Field
The invention relates to the technical field of subjective quality evaluation, in particular to a method for constructing an audio and video multimedia database and audio and video multimedia subjective quality evaluation based on the audio and video multimedia database.
Background
In recent years, visual quality evaluation has attracted attention from many researchers in the fields of image processing, computer vision, and the like. Visual quality assessment can be divided into two broad categories depending on whether there is a user evaluating directly: subjective quality assessment (subjective quality assessment) and objective quality assessment (objective quality assessment). In most visual communication systems, humans are the ultimate recipients of visual media information, and therefore the most reliable evaluation method should be subjective quality evaluation. Subjective quality assessment usually allows a large number of users to Score an assessment object according to a certain standard, and take the "Mean Opinion Score" (MOS) or "Difference of Mean Opinion scores" (DMOS) as the result of subjective quality assessment. The mean opinion score difference refers to an average value of differences between the scores of the reference material and the distortion material, and the influence of the image content can be effectively reduced by subtracting the scores of the reference material.
Subjective quality evaluation has long been considered to be the best method for quality evaluation because of its accuracy, and therefore subjective quality evaluation results are also generally used as criteria for evaluating objective evaluation methods. For common visual signals, such as images, the International Telecommunication Union (ITU) has proposed a subjective image Quality evaluation algorithm in the method for the objective Assessment of the Quality of images, document rec.itu-R bt.500-13, jan.2012, however, the method is only applicable to visual signals such as images and videos. Xumei et al propose a subjective quality evaluation method suitable for panoramic video in "a subjective quality evaluation method for panoramic video, CN 201711452823.8", however, the method is only suitable for visual signals such as panoramic video. Both of the above methods are only suitable for visual signals such as images and videos, but not for multimedia signals including video and audio.
At present, no standardized and flow audio and video multimedia subjective quality evaluation method similar to the method is available.
Disclosure of Invention
Aiming at the defects in the prior art, the invention aims to provide a method for constructing an audio and video multimedia database and subjectively evaluating the quality of the audio and video multimedia.
The invention is realized by the following technical scheme.
According to one aspect of the invention, a method for constructing an audio/video multimedia database is provided, which comprises the following steps: building an audio and video multimedia database for testing and building an audio and video multimedia database for training; wherein:
the audio and video multimedia database for testing and the audio and video multimedia database for training both comprise a plurality of multimedia signal data simultaneously containing video and audio and distortion audio and video multimedia signal data corresponding to the multimedia signal data; wherein each multimedia signal data has a plurality of different visual and auditory properties;
the visual characteristics include:
spatial complexity of the video:
wherein, I represents a video frame,
representing the edge strength of the video frame, std is used for calculating the standard deviation of all pixels in the two-dimensional edge map;
temporal complexity of the video:
TI＝std(It-It-1)，
wherein, ItAnd It-1Representing two adjacent frames of the video, and std is used for calculating the standard deviation of all pixels of the two-dimensional frame difference image;
the auditory properties include:
complexity of the audio:
AI＝std(At-At-1)，
wherein A istAnd At-1Adjacent sample values representing audio, std, are used to calculate the standard deviation of the one-dimensional audio adjacent sample difference values.
Preferably, the multimedia signal data contents in the audio/video multimedia database for testing and the audio/video multimedia data for training are not coincident, and the distortion types and the distortion degrees of the corresponding audio/video multimedia signal data for distortion are the same or similar.
Preferably, the audio-video-distortion multimedia signal data is obtained by:
and respectively carrying out 3-5-level compression distortion on the video and the audio in the audio and video multimedia signal data, obtaining all video and audio distortion combinations by video distortion and audio distortion obtained by any combination, and applying all video and audio distortion combinations to the audio and video multimedia signals to obtain distorted audio and video multimedia signals.
According to another aspect of the present invention, there is provided an audio/video multimedia subjective quality assessment method implemented based on any one of the above audio/video multimedia databases, including the following steps:
s1: the early preparation stage comprises: building an audio and video multimedia quality evaluation environment and selecting data in an audio and video multimedia database for training to train a tester;
s2: a testing phase comprising:
selecting data to be tested from an audio/video multimedia database for testing;
calculating the time length required by the test, and when the time length required by the test is less than or equal to the set time length, directly performing the test as a sub-test; when the time length required by the test exceeds the set time length, dividing the test into a plurality of sub-tests, and ensuring that the time interval between two adjacent sub-tests is at least T;
in each sub-test, testing all selected data to be tested in a random sequence, and carrying out subjective evaluation on the quality of the data to be tested to obtain a subjective quality evaluation result;
s3: a post-data processing stage comprising:
analyzing the subjective quality evaluation result to obtain audio and video subjective quality grading data, and eliminating abnormal values in the data to obtain an average score for describing the subjective quality of each group of audio and video multimedia signal data;
and (4) counting the average score of the subjective quality of each group of audio and video multimedia signal data to obtain the influence of the audio quality and the video quality on the audio and video joint quality.
Preferably, in S1, the established audio-video multimedia quality evaluation environment includes:
testing the environment;
a terminal computer;
and the video display and the audio player are arranged in the test environment and connected with the terminal computer.
Preferably, in S1, the test environment satisfies the conditions of no interference, low noise (noise less than 10 db), and normal lighting.
Preferably, the audio/video multimedia database is installed in a terminal computer, the playing of the video and audio in the database is controlled through the terminal computer, and subjective quality evaluation result data is collected.
Preferably, in step S2, the set time length is 30 minutes.
Preferably, in step S2, T is 24 hours.
Preferably, in S3, the method for obtaining an average score describing the subjective quality of each group of audio-video multimedia signals includes:
firstly, converting subjective quality scoring data of all testers after abnormal values are removed into a Z value;
then, linearly converting the Z value to a range of 0-100, and averaging subjective quality scoring data after linear conversion to obtain a mean opinion score MOS:
wherein: r isijIndicating the grade given to the jth video by the ith tester; mu.siMean score representing the ith tester; sigmaiRepresenting the corresponding standard deviation, and calculating the standard deviation of all pixels of the two-dimensional frame difference image; std is used for calculating the standard deviation of all pixels of the two-dimensional frame difference image; n is the total number of testers;
and (4) counting the average score of the subjective quality of each group of audio and video multimedia signal data to obtain the influence of the audio quality and the video quality on the audio and video joint quality.
Compared with the prior art, the invention has the following beneficial effects:
the audio and video multimedia database construction and audio and video multimedia subjective quality evaluation method comprises the steps of establishing a joint quality subjective evaluation audio and video multimedia database specially used for audio and video multimedia, training a tester by building an audio and video multimedia quality evaluation environment and selecting data in the audio and video multimedia database for training based on the database, testing selected to-be-tested data from the audio and video multimedia database for testing, and performing post-processing on a subjective quality evaluation result obtained by testing, so that standardized and streamlined audio and video multimedia subjective quality evaluation is realized, the problem that the audio and video multimedia subjective quality evaluation data cannot be effectively reproduced is solved, and large-scale and high-reliability acquisition of audio and video subjective quality is realized.
Drawings
Other features, objects and advantages of the invention will become more apparent upon reading of the detailed description of non-limiting embodiments with reference to the following drawings:
fig. 1 is an overall flow diagram of an audio/video multimedia subjective quality evaluation method according to an embodiment of the present invention;
FIG. 2 is an exemplary diagram of a graphical user interface for quality assessment of a master-godson video according to an embodiment of the present invention;
fig. 3 is an exemplary diagram of statistical results of questionnaire surveys provided by the embodiment of the present invention.
Detailed Description
The following examples illustrate the invention in detail: the embodiment is implemented on the premise of the technical scheme of the invention, and a detailed implementation mode and a specific operation process are given. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the inventive concept, which falls within the scope of the present invention.
The embodiment of the invention provides a method for constructing an audio and video multimedia database, which comprises the following steps: building an audio/video multimedia database for testing and building audio/video multimedia data for training; wherein:
the audio and video multimedia database for testing and the audio and video multimedia database for training both comprise a plurality of multimedia signal data simultaneously containing video and audio and distortion audio and video multimedia signal data corresponding to the multimedia signal data; wherein each multimedia signal data has a plurality of different visual and auditory properties.
The multimedia signal data contents in the audio and video multimedia database for testing and the audio and video multimedia data for training are not coincident, and the distortion types and the distortion degrees of the corresponding distorted audio and video multimedia signal data are the same or similar.
As shown in fig. 1, based on the audio/video multimedia database constructed in the foregoing embodiment, an embodiment of the present invention further provides an audio/video multimedia subjective quality evaluation method implemented based on the audio/video multimedia database, including the following steps:
s1: the early preparation stage comprises: building an audio and video multimedia quality evaluation environment and selecting data in an audio and video multimedia database for training to train a tester;
s2: a testing phase comprising:
selecting data to be tested from an audio/video multimedia database for testing;
calculating the time length required by the test, and when the time length required by the test is less than or equal to the set time length, directly performing the test as a sub-test; when the time length required by the test exceeds the set time length, dividing the test into a plurality of sub-tests, and ensuring that the time interval between two adjacent sub-tests is at least T;
in each sub-test, all selected data to be tested are tested by a tester in a random sequence, and the quality of the data to be tested is subjectively evaluated to obtain a subjective quality evaluation result;
s3: a post-data processing stage comprising:
analyzing the subjective quality evaluation result to obtain audio and video subjective quality grading data, and eliminating abnormal values in the data to obtain an average score for describing the subjective quality of each group of audio and video multimedia signal data;
and (4) counting the average score of the subjective quality of each group of audio and video multimedia signal data to obtain the influence of the audio quality and the video quality on the audio and video joint quality.
The technical solutions provided by the above embodiments of the present invention are further described in detail below with reference to the accompanying drawings.
In an embodiment of the invention, the audio/video multimedia database is constructed by the following steps:
firstly, multimedia signals containing high-quality videos and high-quality audios are collected, the visual and auditory characteristics of related audios and videos are calculated, and diversified and differentiated audios and videos are selected as the multimedia signals to be researched;
the visual characteristics of video refer to the spatial complexity of video:
wherein, I represents a video frame,
representing the edge strength of the video frame, std calculating the standard deviation of all pixels in the two-dimensional edge image;
the visual characteristics of video on the other hand refer to the temporal complexity of the video:
TI＝std(It-It-1)，
wherein ItAnd It-1Representing two adjacent frames of the video, and std calculating the standard deviation of all pixels of the two-dimensional frame difference image;
the auditory properties of audio refer to the complexity of the audio:
AI＝std(At-At-1)，
wherein A istAnd At-1Representing adjacent sample values of the audio, std calculating a standard deviation of a difference value of the adjacent samples of the one-dimensional audio;
secondly, simulating the distortion of the video and audio to be researched, and simultaneously applying the distortion to the selected high-quality audio and video signals so as to generate a series of audio and video distortion multimedia signals;
among them, the most common distortion of video is compression and scaling, so two types of video distortion can be included: compression and compression plus scaling, for which 3-5 levels of compression distortion are introduced; the most common distortion of audio is compression, and 3-5 levels of compression distortion can be introduced as well; then, randomly combining the video distortion and the audio distortion to obtain all audio and video distortion combinations, and applying all the distortion combinations to the high-quality multimedia signals to generate a series of audio and video distortion multimedia signals;
and finally, constructing a testing audio and video multimedia database in the audio and video multimedia database by using the high-quality audio and video multimedia signal and the distortion audio and video multimedia signal.
And referring to the steps, a small number of audio/video multimedia signals which are not coincident with the signal content, have the same or similar distortion types and distortion degrees in the audio/video multimedia database for testing are collected, and an audio/video multimedia database for training is constructed.
In another embodiment of the present invention, a subjective quality evaluation method for audio/video multimedia is provided, and the method is performed by using the audio/video multimedia database constructed in the above embodiment. As shown in fig. 1, the audio/video multimedia subjective quality evaluation method comprises the following detailed steps:
the first step, early preparation phase, includes: building an audio and video multimedia quality evaluation environment and selecting data in an audio and video multimedia database for training to train a tester;
specifically, the method comprises the following steps:
(1) building an audio and video multimedia quality evaluation environment
The audio and video multimedia quality evaluation environment is set up to meet the following requirements:
the audio and video multimedia quality evaluation environment is built in an indoor place which is not easy to interfere, normally illuminates and has no excessive noise; the fact that each tester is not easily interfered means that each tester is guaranteed not to be interfered by external human factors and interrupted in the whole subjective quality evaluation process; the lighting condition can refer to a subjective quality evaluation method of the related video; in addition, the environment should be as quiet as possible to avoid noise from affecting the tester's judgment on audio quality;
the playing control of the audio and Video and the collection of subjective quality evaluation data can be realized by a computer and a corresponding graphical user interface, wherein the graphical user interface is shown as figure 2, a user can subjectively evaluate the watched audio and Video by a left sliding bar, and the Next Video button can be clicked to synchronously play the Next audio and Video signal after the evaluation is finished;
the video signal can be displayed by a display, wherein the display can be a consumer-grade display used daily, and any video should be played at the original resolution of the video when played, so that the quality of experience of the video is not affected by the zooming operation of the video;
the audio signal can be played by a sound box or a headset, wherein the sound box or the headset preferably uses high-quality sound boxes or headsets facing common consumers so as to prevent the sound box or the headset from excessively influencing the evaluation of the audio signal quality;
the method has the advantages that smooth playing of video signals and audio signals is guaranteed, the problems that delay, frame loss, asynchrony of audio and video and the like affect experience quality cannot occur, all audio and video multimedia signals to be tested are played by experimenters before testing, and all audio and video cannot occur.
(2-1) selecting data in an audio/video multimedia database for training to train the tester, wherein the following modes can be adopted:
firstly, performing vision test on a subjective tester to ensure that the tester corrects normal vision, wherein the vision test can be performed by using a Snellen vision test chart;
secondly, performing vision test on the subjective testers to ensure normal hearing of the testers, wherein the hearing test can be performed by using a binaural hearing test;
finally, the audio and video subjective quality evaluation test is introduced to a tester integrally, wherein all contents to be explained can be printed and handed to the tester for reading, and important contents are explained to the tester in a way of oral expression;
the part of the content to be described to the tester should contain as much as possible all the information that the tester should be informed of.
(2-2) subjective quality evaluation training of audio and video multimedia of testers
And by adopting the data in the audio and video multimedia database for training, a tester can perform audio and video joint quality evaluation on the training audio and video multimedia signal, so that the tester is familiar with test contents, distortion types, distortion degrees, graphical user interfaces used for testing and the like.
Second, testing stage, selecting data to be tested from audio/video multimedia database for testing, and formally evaluating subjective quality of audio/video multimedia
Firstly, calculating and estimating the time length required by the formal test, if the required test time is longer than 30 minutes, uniformly dividing the formal test into a plurality of tests, and ensuring that the time interval of each test is at least 24 hours;
secondly, in each test, the tester experiences all the audio and video multimedia signals with the test in a random sequence, and subjectively evaluates the experience quality of the audio and the video to obtain a subjective quality evaluation result.
The third step, the later data processing stage, which processes and analyzes the subjective quality evaluation result
Firstly, audio and video subjective quality grading data obtained by formal subjective quality evaluation are analyzed and processed, abnormal values are removed, and average scores describing the subjective quality of each group of audio and video multimedia signals are obtained;
generally, the scores of all testers after the abnormal testers are eliminated are converted into a Z value, then the Z value is linearly transformed to a range of 0-100, and then all scores are averaged to obtain a Mean Opinion Score (MOS):
wherein r isijRepresents the score, μ, of the ith tester to the jth videoiIs the mean score, σ, of the ith test subjectiAnd calculating the standard deviation of all pixels of the two-dimensional frame difference image according to the corresponding standard deviation, calculating the standard deviation of all pixels of the two-dimensional frame difference image according to std, wherein N is the total number of the testers.
Secondly, counting the average scores of the subjective quality of each group of audio and video multimedia signal data, and knowing the influence of the audio quality and the video quality on the audio and video joint quality.
The result of counting the subjective quality evaluation scores is shown in fig. 3, and it is known from the figure that: the tester believes that the ratio of the video and audio to the overall quality of experience weight is about 57% to 43%; most testers consider the overall experience quality of the audio and video to be more difficult to evaluate than the video quality or the audio quality of a single mode; most testers evaluate the overall experience quality and the like by sequentially judging the video quality and the audio quality. According to the statistical results, the method provided by the invention can effectively enable a tester to consider the audio quality and the video quality at the same time, thereby giving reasonable and reliable evaluation to the overall audio and video quality.
The audio and video multimedia subjective quality evaluation method provided by the embodiment of the invention can provide standardized and flow subjective audio and video multimedia quality evaluation through the steps of audio and video multimedia quality evaluation environment construction, audio and video multimedia data preparation to be evaluated, preparation in the early stage of main kwan-yin video quality evaluation test, audio and video multimedia subjective quality evaluation training of testers, formal audio and video multimedia subjective quality evaluation, questionnaire survey after subjective test, subjective quality evaluation and questionnaire survey data processing and analysis and the like.
The foregoing description of specific embodiments of the present invention has been presented. It is to be understood that the present invention is not limited to the specific embodiments described above, and that various changes and modifications may be made by one skilled in the art within the scope of the appended claims without departing from the spirit of the invention.
Claims (10)
1. A method for constructing an audio/video multimedia database is characterized by comprising the following steps: building an audio and video multimedia database for testing and building an audio and video multimedia database for training; wherein:
the audio and video multimedia database for testing and the audio and video multimedia database for training both comprise a plurality of audio and video multimedia signal data simultaneously containing video and audio and distortion audio and video multimedia signal data corresponding to the audio and video multimedia signal data; each audio/video multimedia signal data has various different visual characteristics and auditory characteristics;
the visual characteristics comprise spatial domain complexity SI of the video and temporal domain complexity TI of the video;
spatial domain complexity SI of the video:
wherein, I represents a video frame,
representing the edge strength of the video frame, std is used for calculating the standard deviation of all pixels in the two-dimensional edge map;
temporal complexity TI of the video:
TI＝std(It-It-1)，
wherein, ItAnd It-1Representing two adjacent frames of the video, and std is used for calculating the standard deviation of all pixels of the two-dimensional frame difference image;
the auditory properties include the complexity AI of the audio:
AI＝std(At-At-1)，
wherein A istAnd At-1Adjacent sample values representing audio, std, are used to calculate the standard deviation of the one-dimensional audio adjacent sample difference values.
2. The method for constructing the audio/video multimedia database according to claim 1, wherein the audio/video multimedia signal data contents in the audio/video multimedia database for testing and the audio/video multimedia data for training are not coincident, and the distortion types and the distortion degrees of the corresponding audio/video multimedia signal data are similar.
3. The method for constructing an audio-video multimedia database according to claim 1, wherein the distorted audio-video multimedia signal data is obtained by:
and respectively carrying out 3-5-level compression distortion on the video and the audio in the audio and video multimedia signal data, obtaining all video and audio distortion combinations by video distortion and audio distortion obtained by any combination, and applying all video and audio distortion combinations to the audio and video multimedia signals to obtain distorted audio and video multimedia signals.
4. An audio-video multimedia subjective quality assessment method implemented based on the audio-video multimedia database according to any one of claims 1 to 3, characterized by comprising:
s1: the early preparation stage comprises: building an audio and video multimedia quality evaluation environment and selecting data in an audio and video multimedia database for training to train a tester;
s2: a testing phase comprising:
selecting data to be tested from an audio/video multimedia database for testing;
calculating the time length required by the test, and when the time length required by the test is less than or equal to the set time length, directly performing the test as a sub-test; when the time length required by the test exceeds the set time length, dividing the test into a plurality of sub-tests, and ensuring that the time interval between two adjacent sub-tests is at least T;
in each sub-test, testing all selected data to be tested in a random sequence, and carrying out subjective evaluation on the quality of the data to be tested to obtain a subjective quality evaluation result;
s3: a post-data processing stage comprising:
analyzing the subjective quality evaluation result to obtain audio and video subjective quality grading data, and eliminating abnormal values in the data to obtain an average score for describing the subjective quality of each group of audio and video multimedia signal data;
and (4) counting the average score of the subjective quality of each group of audio and video multimedia signal data to obtain the influence of the audio quality and the video quality on the audio and video joint quality evaluation.
5. The audio/video multimedia subjective quality evaluation method according to claim 4, wherein in S1, the set up audio/video multimedia quality evaluation environment includes:
testing the environment;
a terminal computer;
and the video display and the audio player are arranged in the test environment and connected with the terminal computer.
6. The subjective quality assessment method of audio-visual multimedia according to claim 5, wherein in S1, the test environment meets the conditions of no interference, noise less than 10 db, and normal lighting.
7. The audio/video multimedia subjective quality assessment method according to claim 5, wherein the audio/video multimedia database is installed in a terminal computer and controls the playing of video and audio in the database through the terminal computer, and collects subjective quality assessment result data.
8. The subjective quality assessment method for audio-visual multimedia according to claim 4, wherein in step S2, the set time length is 30 minutes.
9. The subjective quality assessment method for audio-visual multimedia according to claim 4, wherein in step S2, T is 24 hours.
10. The subjective quality assessment method of audio-visual multimedia according to claim 4, wherein in S3, the method for obtaining the average score describing the subjective quality of each group of audio-visual multimedia signals comprises:
firstly, converting subjective quality scoring data of all testers after abnormal values are removed into a Z value;
then, linearly converting the Z value to a range of 0-100, and averaging subjective quality scoring data after linear conversion to obtain a mean opinion score MOS:
wherein: r isijIndicating the grade given to the jth video by the ith tester; mu.siMean score representing the ith tester; sigmaiRepresenting the corresponding standard deviation, and calculating the standard deviation of all pixels of the two-dimensional frame difference image; std is used for calculating the standard deviation of all pixels of the two-dimensional frame difference image; n is the total number of testers;
and (4) counting the average score of the subjective quality of each group of audio and video multimedia signal data to obtain the influence of the audio quality and the video quality on the audio and video joint quality.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010172382.1A CN111355949B (en) | 2020-03-12 | 2020-03-12 | Method for constructing audio and video multimedia database and evaluating subjective quality of multimedia |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010172382.1A CN111355949B (en) | 2020-03-12 | 2020-03-12 | Method for constructing audio and video multimedia database and evaluating subjective quality of multimedia |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111355949A true CN111355949A (en) | 2020-06-30 |
| CN111355949B CN111355949B (en) | 2021-05-25 |
Family
ID=71194372
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010172382.1A Active CN111355949B (en) | 2020-03-12 | 2020-03-12 | Method for constructing audio and video multimedia database and evaluating subjective quality of multimedia |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111355949B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113382232A (en) * | 2021-08-12 | 2021-09-10 | 北京微吼时代科技有限公司 | Method, device and system for monitoring audio and video quality and electronic equipment |
| CN115396695A (en) * | 2022-08-18 | 2022-11-25 | 上海哔哩哔哩科技有限公司 | Method executed by mobile terminal, medium, and computer system |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101783971A (en) * | 2010-01-26 | 2010-07-21 | 深圳市融创天下科技发展有限公司 | Method for subjective testing of video quality |
| CN102496372A (en) * | 2011-12-15 | 2012-06-13 | 中国传媒大学 | Low-code-rate audio quality objective evaluation method based on nonlinear parameter fitting |
| CN108055533A (en) * | 2017-01-04 | 2018-05-18 | 北京航空航天大学 | A kind of subjective quality assessment method for panoramic video |
| CN208400483U (en) * | 2018-10-31 | 2019-01-18 | 中国铁道科学研究院集团有限公司 | Video and audio quality subjective evaluation laboratory |
-
2020
- 2020-03-12 CN CN202010172382.1A patent/CN111355949B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101783971A (en) * | 2010-01-26 | 2010-07-21 | 深圳市融创天下科技发展有限公司 | Method for subjective testing of video quality |
| CN102496372A (en) * | 2011-12-15 | 2012-06-13 | 中国传媒大学 | Low-code-rate audio quality objective evaluation method based on nonlinear parameter fitting |
| CN108055533A (en) * | 2017-01-04 | 2018-05-18 | 北京航空航天大学 | A kind of subjective quality assessment method for panoramic video |
| CN208400483U (en) * | 2018-10-31 | 2019-01-18 | 中国铁道科学研究院集团有限公司 | Video and audio quality subjective evaluation laboratory |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113382232A (en) * | 2021-08-12 | 2021-09-10 | 北京微吼时代科技有限公司 | Method, device and system for monitoring audio and video quality and electronic equipment |
| CN115396695A (en) * | 2022-08-18 | 2022-11-25 | 上海哔哩哔哩科技有限公司 | Method executed by mobile terminal, medium, and computer system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111355949B (en) | 2021-05-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN100559880C (en) | A kind of highly-clear video image quality evaluation method and device based on self-adapted ST area | |
| Almohammad et al. | Stego image quality and the reliability of PSNR | |
| Series | Methodology for the subjective assessment of the quality of television pictures | |
| CN111355949B (en) | Method for constructing audio and video multimedia database and evaluating subjective quality of multimedia | |
| CN101312544A (en) | Video quality automatic evaluation system oriented to wireless network and evaluation method thereof | |
| CN111510710B (en) | Video quality evaluation method based on electroencephalogram signals and space-time distortion | |
| CN111696078B (en) | Ultra-high definition video detection method and system | |
| CN113473117B (en) | Non-reference audio and video quality evaluation method based on gated recurrent neural network | |
| Corriveau | Video quality testing | |
| CN113554597A (en) | Image quality evaluation method and device based on electroencephalogram characteristics | |
| Voran et al. | The development and evaluation of an objective video quality assessment system that emulates human viewing panels | |
| CN112927720B (en) | Audio anomaly detection method and device | |
| CN101895787B (en) | Method and system for subjectively evaluating video coding performance | |
| Pastor et al. | Comparison of conditions for omnidirectional video with spatial audio in terms of subjective quality and impacts on objective metrics resolving power | |
| CN111479106A (en) | Two-dimensional quality descriptor fused audio and video joint quality evaluation method and terminal | |
| Series | Methodology for the subjective assessment of the quality of television pictures | |
| CN115457029B (en) | Underwater image quality measurement method based on perception characteristics | |
| CN113593536B (en) | Device and system for detecting voice recognition accuracy | |
| Martinez et al. | Perceived quality of audio-visual stimuli containing streaming audio degradations | |
| US20050105802A1 (en) | Method and an arrangement for objective assessment of video quality | |
| Zou et al. | Methods for evaluation of digital television picture quality | |
| Wei et al. | Non-invasive image quality assessment based on eye-tracking | |
| Ouni et al. | A new spatial colour metric for perceptual comparison | |
| Zhiming et al. | Quality of experience models for network video quality | |
| CN117607808A (en) | Method and system for realizing interference effectiveness evaluation by using radar original video data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |