WO2014198062A1 - Procédé et appareil pour la mesure de la qualité vidéo - Google Patents

Procédé et appareil pour la mesure de la qualité vidéo Download PDF

Info

Publication number
WO2014198062A1
WO2014198062A1 PCT/CN2013/077257 CN2013077257W WO2014198062A1 WO 2014198062 A1 WO2014198062 A1 WO 2014198062A1 CN 2013077257 W CN2013077257 W CN 2013077257W WO 2014198062 A1 WO2014198062 A1 WO 2014198062A1
Authority
WO
WIPO (PCT)
Prior art keywords
frame
pixels
video sequence
error
quality
Prior art date
Application number
PCT/CN2013/077257
Other languages
English (en)
Inventor
Debing Liu
Zhibo Chen
Xiaodong Gu
Original Assignee
Thomson Licensing
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Thomson Licensing filed Critical Thomson Licensing
Priority to PCT/CN2013/077257 priority Critical patent/WO2014198062A1/fr
Publication of WO2014198062A1 publication Critical patent/WO2014198062A1/fr

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N17/00—Diagnosis, testing or measuring for television systems or their details
    • H04N17/004—Diagnosis, testing or measuring for television systems or their details for digital television systems
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/154—Measured or subjectively estimated visual quality after decoding, e.g. measurement of distortion
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/85—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression
    • H04N19/89—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression involving methods or arrangements for detection of transmission errors at the decoder
    • H04N19/895—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression involving methods or arrangements for detection of transmission errors at the decoder in combination with error concealment

Definitions

  • This invention relates to video quality measurement, and more particularly, to a method and apparatus for determining an objective video quality metric.
  • the determined video quality metric can then be used, for example, to adjust encoding parameters, or to provide required video quality at the receiver side.
  • a video bitstream may suffer from transmission losses when it is transmitted over unreliable channels such as wireless network or Internet. Consequently, in addition to the quality loss from video compression, the video quality is further degraded when a video is transmitted through unreliable networks.
  • a successful video quality modeling tool needs to rate the quality degradation caused by network transmission impairment (for example, packet losses, transmission delays, and transmission jitters), in addition to quality degradation caused by video compression.
  • the present principles provide a method for estimating visual quality of a video sequence, comprising the steps of: accessing a first set of pixels that is used to conceal a block of a frame in the video sequence; accessing a second set of pixels that is used to approximate the block; determining differences responsive to the first and second sets of pixels; and estimating the visual quality of the video sequence responsive to the determined differences as described below.
  • the present principles also provide an apparatus for performing these steps.
  • the present principles also provide a method for estimating visual quality of a video sequence, comprising the steps of: accessing a first set of pixels that is used to conceal a block of a frame in the video sequence; accessing a second set of pixels that is used to approximate the block; determining differences responsive to the first and second sets of pixels; estimating a respective set of differences for each of a plurality of spatial neighboring blocks of the block; and estimating the visual quality of the video sequence responsive to the determined differences and the respective sets of differences for the plurality of spatial neighboring blocks as described below.
  • the present principles also provide an apparatus for performing these steps.
  • the present principles also provide a computer readable storage medium having stored thereon instructions for estimating visual quality of a video sequence according to the methods described above.
  • FIG. 1 is a pictorial example depicting that a macroblock (EC-MB B) in frame t+k is used to conceal a macroblock (Error MB A) in current frame t.
  • EC-MB B macroblock
  • Error MB A macroblock
  • FIG. 2A is a pictorial example depicting artifacts caused by spatial error concealment
  • FIG. 2B is a pictorial example depicting artifacts caused by temporal error concealment
  • FIGs. 2C and 2D are pictorial examples depicting artifacts caused by error propagation.
  • FIG.3 is a flow diagram depicting an example of estimating initial visible artifact level, in accordance with an embodiment of the present principles.
  • FIG. 4 is a pictorial example depicting an error macroblock and its four neighboring macroblocks.
  • FIG. 5 is a pictorial example depicting a decoded frame after error
  • FIG. 6 is a block diagram depicting an example of a video quality monitor, in accordance with an embodiment of the present principles.
  • FIG. 7 is a block diagram depicting an example of a video processing system that may be used with one or more implementations.
  • MB macroblock
  • the principles may be adapted to use a block at a different size, for example, an 8x8 block, a 1 6x8 block, a 32x32 block, or a 64x64 block.
  • macroblocks corresponding to the lost portions are denoted as lost macroblocks and they are undecodable.
  • a decoder may adopt error concealment techniques to conceal macroblocks that are not properly reconstructed.
  • the goal of error concealment is to restore undecodable macroblocks in order to reduce perceptual quality degradation.
  • the perceived strength of artifacts in restored macroblocks depends heavily on employed error concealment techniques.
  • a spatial or temporal approach may be used for error concealment.
  • spatial approach spatial correlation between pixels is exploited, and undecodable macroblocks can be recovered from neighboring pixels using interpolation techniques.
  • temporal coherence and spatial smoothness of pixels can be exploited to estimate an undecodable macroblock or pixel.
  • undecodable pixels can be estimated using collocated pixels in a previously decoded frame (i.e., using pixels in the same positions as the
  • undecodable pixels in the previously decoded frame can be concealed using reference pixels in previous frames.
  • FIG. 1 illustrates that an EC-MB B in a previous decoded frame t+k can be used to conceal an Error MB A in the current frame t in a temporal error concealment approach.
  • FIGs. 2A and 2B illustrate exemplary decoded frames, where some packets of the coded bitstream are lost during transmission. Specifically, for FIG. 2A, spatial error concealment is used and obvious artifact can still be noticed.
  • FIG. 2B temporal error concealment is used, and some macroblocks are restored properly and some other macroblocks still have visible artifacts.
  • FIGs. 2C and 2D illustrate artifacts caused by error propagation.
  • visual artifacts may still be perceived after error concealment.
  • the visual artifacts may occur at an error MB because error concealment is not effective therein.
  • Such visual artifacts caused by undecodable macroblocks are denoted as initial visible artifacts. If a block having initial visible artifacts is used as a reference, for example, for intra prediction or inter prediction, the initial visible artifacts may propagate spatially or temporally to other blocks in the same or other frames through prediction. Such propagated artifacts are denoted as propagated visible artifacts.
  • the overall artifacts, caused by initial and propagated visible artifacts are denoted as overall visible artifacts.
  • the present principles are directed to measuring initial visible artifact level, which can then be used to measure the overall visible artifact level.
  • FIG. 3 illustrates an exemplary video quality modeling method 300 for estimating initial visible artifact levels.
  • Method 300 starts at step 305.
  • step 310 we locate error macroblocks in the current frame, for example, from the bitstream information.
  • step 320 we estimate pixels that are used for performing error concealment of a current error MB.
  • step 325 we locate pixels that can be used to approximate the current error MB.
  • step 330 we estimate the error (also referred to as residual) caused by transmission errors which remains after error concealment in the current error MB.
  • the error also referred to as residual
  • the initial visible artifact level for the current error MB Based on the estimated error, at step 340, we calculate an initial visible artifact level for the current error MB.
  • step 350 it checks whether more error MBs need to be processed. If yes, the control returns to step 325.
  • method 300 ends at step 399.
  • an error MB can be the restored by copying its collocated MB in a previous decoded frame as shown in FIG. 1 , wherein a collocated MB in a previous decoded frame refers to a macroblock in the same position as the error MB in the previous decoded frame.
  • a collocated MB in a previous decoded frame refers to a macroblock in the same position as the error MB in the previous decoded frame.
  • dij ( k ) (t ⁇ j (t + /c)
  • , pixel (i, j) e error MBs, k —3,—2,—1,1,2,3, (1 )
  • Ijj(t) is the luminance intensity of the pixel in the i th row and f h column of current frame t
  • j(t+k) is the luminance intensity of the pixel in the / row and / column of neighboring frame t+k
  • d,j- (k) is the difference.
  • k EC argmin fc (d(/c)), (3) where t+k E c denotes the detected error concealment frame.
  • error concealment frame t+k E c there may be other neighboring frames whose d(/c)are close or equal to d(k EC ).
  • the preceding frame is selected as the error
  • frame t-1 is selected as the frame used for error
  • Locate pixels to approximate an error MB As discussed above, when an error MB is detected, an EC-MB is used to conceal the error MB. For ease of notation, we denote the error MB after error concealment as the restored error MB (that is, the restored error MB is identical to the corresponding EC-MB if an EC-MB is correctly detected and copied for error concealment), and denote the error MB as if it had been correctly decoded as the original error MB.
  • the difference between the original error MB and the restored error MB may cause visible artifacts.
  • a macroblock in a temporal neighboring frame is used to approximate what should have been correctly decoded at the error MB.
  • video characteristics for example, but not limited to, scene cut around current frame t, the distance between the current frame and the error concealment frame, and choose a representative macroblock in the temporal neighborhood that can be used to approximate the error MB.
  • the error MB is marked as ineffective and it will not be used in estimating the initial visible artifact level. Otherwise, the error MB is marked as effective.
  • , pixel(i, j) e error MB; m 1 or— 1.
  • the pixel differences calculated for the error MB may also estimate the differences at spatial neighboring MBs of the error MB (for example, MBu, MB
  • neighboring macroblock is an error MB
  • the neighboring macroblock is marked as ineffective and will not be used to estimate the initial visible artifact level.
  • a neighboring MB is considered effective if it is not marked as ineffective.
  • N 0
  • we may estimate the original error MB by using its spatio-temporal neighboring macroblocks, and calculate residual R based on pixel differences between the restored error MB and the estimated original error MB. Estimate the initial visible artifact level for an error MB
  • the initial visible artifact level of the error MB may be calculated as:
  • initial visible artifact level of the image/video can be estimated, for example, by averaging the initial visible artifact levels of error MBs. Because of intra or inter prediction used in video compression, initial visible artifacts of error MBs may propagate spatially and temporally to other macroblocks or other frames.
  • Propagated visible artifact levels can be estimated, for example, using the method and apparatus described in a commonly owned PCT application, entitled "Video quality assessment at a bitstream level" by N. Liao, Z. Chen, and K. Xie
  • the present principles can not only detect spatial artifact, but also temporal artifact.
  • the present principles can be more efficient in detecting artifacts. For example, in a decoded frame as shown in FIG. 5, the spatial discontinuity is not very high. But when the video is played frame by frame, obvious temporal discontinuity can be observed. Our method can provide a high artifact level which indicates that there exist spatial/temporal artifact in the frame.
  • FIG. 6 depicts a block diagram of an exemplary video quality monitor 600.
  • the input of apparatus 600 may include a transport stream that contains the bitstream.
  • the input may be in other formats that contain the bitstream.
  • Demultiplexer610 obtains packet layer information, for example, which packets are lost, from the bitstream. Decoder 620 decodes the input stream to reconstruct the video and perform error concealment if necessary. Decoder 620 also determines which macroblock is an error MB. In other embodiments, the decoder may perform the functions of the demultiplexer.
  • initial visible artifact levels for error macroblocks are estimated at initial visible artifact level estimator 630, for example, using method 300.
  • the overall artifact levels for individual macroblocks are estimated at overall artifact level estimator 640.
  • a quality predictor 650 then pools the macroblock level artifact levels into a quality score. The quality predictor 650 may consider other types of artifacts and the property of human visual property.
  • the video quality monitor 600 may be used, for example, in ITU-T P.NBAMS (parametric non-intrusive bitstream assessment of video media streaming quality) standard, which works on video quality assessment models in two application scenarios, namely, IPTV and mobile video streaming, also called HBR scenario and LBR scenario respectively.
  • ITU-T P.NBAMS parametri non-intrusive bitstream assessment of video media streaming quality
  • the input to the P.NBAMS VQM Video Quality Model
  • the output is an objective MOS (Mean Opinion Score) score.
  • MOS Magnetic Opinion Score
  • STB set-top box
  • P.NBAMS mode 1 model only uses bitstream information by parsing the H.264 syntax
  • mode 2 model may decode parts or all of the video sequence, and the pixel information is used for visual quality prediction in addition to parsing the bitstream information in order to improve the prediction accuracy.
  • a processor 705 processes the video and the encoder 710 encodes the video.
  • the bitstream generated from the encoder is transmitted to a decoder 730 through a distribution network 720.
  • a video quality monitor for example, the video quality monitor 600, may be used at different stages.
  • a video quality monitor 740 may be used by a content creator.
  • the estimated video quality may be used by an encoder in deciding encoding parameters, such as mode decision or bit rate allocation.
  • the content creator uses the video quality monitor to monitor the quality of encoded video. If the quality metric does not meet a pre-defined quality level, the content creator may choose to re-encode the video to improve the video quality. The content creator may also rank the encoded video based on the quality and charges the content accordingly.
  • a video quality monitor 750 may be used by a content distributor.
  • a video quality monitor may be placed in the distribution network. The video quality monitor calculates the quality metrics and reports them to the content distributor. Based on the feedback from the video quality monitor, a content distributor may improve its service by adjusting bandwidth allocation and access control.
  • the content distributor may also send the feedback to the content creator to adjust encoding.
  • improving encoding quality at the encoder may not necessarily improve the quality at the decoder side since a high quality encoded video usually requires more bandwidth and leaves less bandwidth for transmission protection. Thus, to reach an optimal quality at the decoder, a balance between the encoding bitrate and the bandwidth for channel protection should be considered.
  • a video quality monitor 760 may be used by a user device. For example, when a user device searches videos in Internet, a search result may return many videos or many links to videos corresponding to the requested video content. The videos in the search results may have different quality levels. A video quality monitor can calculate quality metrics for these videos and decide to select which video to store.
  • the user device may have access to several error concealment techniques.
  • a video quality monitor can calculate quality metrics for different error concealment techniques and automatically choose which concealment technique to use based on the calculated quality metrics.
  • the implementations described herein may be implemented in, for example, a method or a process, an apparatus, a software program, a data stream, or a signal. Even if only discussed in the context of a single form of implementation (for example, discussed only as a method), the implementation of features discussed may also be implemented in other forms (for example, an apparatus or program).
  • An apparatus may be implemented in, for example, appropriate hardware, software, and firmware.
  • the methods may be implemented in, for example, an apparatus such as, for example, a processor, which refers to processing devices in general, including, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device.
  • processors also include communication devices, such as, for example, computers, cell phones, portable/personal digital assistants ("PDAs"), and other devices that facilitate communication of information between end-users.
  • PDAs portable/personal digital assistants
  • the appearances of the phrase “in one embodiment” or “in an embodiment” or “in one implementation” or “in an implementation”, as well any other variations, appearing in various places throughout the specification are not necessarily all referring to the same embodiment.
  • this application or its claims may refer to "determining" various pieces of information. Determining the information may include one or more of, for example, estimating the information, calculating the information, predicting the information, or retrieving the information from memory. Further, this application or its claims may refer to "accessing" various pieces of information. Accessing the information may include one or more of, for example, receiving the information, retrieving the information (for example, from memory), storing the information, processing the information, transmitting the information, moving the information, copying the information, erasing the information, calculating the information, determining the information, predicting the information, or estimating the information.
  • Receiving is, as with “accessing”, intended to be a broad term.
  • Receiving the information may include one or more of, for example, accessing the information, or retrieving the information (for example, from memory). Further,
  • receiving is typically involved, in one way or another, during operations such as, for example, storing the information, processing the information, transmitting the information, moving the information, copying the information, erasing the information, calculating the information, determining the information, predicting the information, or estimating the information.
  • implementations may produce a variety of signals formatted to carry information that may be, for example, stored or transmitted.
  • the information may include, for example, instructions for performing a method, or data produced by one of the described implementations.
  • a signal may be formatted to carry the bitstream of a described embodiment.
  • Such a signal may be formatted, for example, as an electromagnetic wave (for example, using a radio frequency portion of spectrum) or as a baseband signal.
  • formatting may include, for example, encoding a data stream and modulating a carrier with the encoded data stream.
  • the information that the signal carries may be, for example, analog or digital information.
  • the signal may be transmitted over a variety of different wired or wireless links, as is known.
  • the signal may be stored on a processor-readable medium.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • General Health & Medical Sciences (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

Un macrobloc d'une séquence vidéo peut être indécodable parce que les données compressées correspondantes sont perdues ou ne peuvent pas être analysées correctement. Un macrobloc indécodable peut avoir été caché au moyen de techniques de dissimulation d'erreurs. Le niveau des artefacts initiaux visibles provoqués par des macroblocs indécodables est estimé d'après les macroblocs voisins spatio-temporels du macrobloc indécodable. En particulier, des différences de pixel entre le macrobloc utilisé pour la dissimulation d'erreurs et le macrobloc indécodable qui existerait s'il avait été correctement décodé peuvent être calculées. Combiné avec les différences calculées pour les macroblocs voisins, le niveau des artefacts initiaux visibles peut être estimé. Les artefacts initiaux visibles peuvent se propager dans l'espace ou dans le temps vers d'autres macroblocs par prédiction. Si l'on considère à la fois les artefacts initiaux visibles et les artefacts propagés, des niveaux d'artefacts globaux peuvent être estimés pour les différents macroblocs. On peut alors estimer la qualité visuelle de la séquence vidéo en assemblant les niveaux des artefacts de niveau macrobloc.
PCT/CN2013/077257 2013-06-14 2013-06-14 Procédé et appareil pour la mesure de la qualité vidéo WO2014198062A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/CN2013/077257 WO2014198062A1 (fr) 2013-06-14 2013-06-14 Procédé et appareil pour la mesure de la qualité vidéo

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2013/077257 WO2014198062A1 (fr) 2013-06-14 2013-06-14 Procédé et appareil pour la mesure de la qualité vidéo

Publications (1)

Publication Number Publication Date
WO2014198062A1 true WO2014198062A1 (fr) 2014-12-18

Family

ID=52021587

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2013/077257 WO2014198062A1 (fr) 2013-06-14 2013-06-14 Procédé et appareil pour la mesure de la qualité vidéo

Country Status (1)

Country Link
WO (1) WO2014198062A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108696751A (zh) * 2017-04-11 2018-10-23 中国移动通信有限公司研究院 一种视频处理方法和装置

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2007130389A2 (fr) * 2006-05-01 2007-11-15 Georgia Tech Research Corporation Procédé et système de mesure de la qualité vidéo automatique reposant sur des mesures de cohérence spatio-temporelles
WO2011082719A1 (fr) * 2010-01-11 2011-07-14 Telefonaktiebolaget L M Ericsson (Publ) Technique d'estimation de la qualité d'une vidéo
CN102959976A (zh) * 2010-04-30 2013-03-06 汤姆森特许公司 评估视频流质量的方法及设备
US20130114002A1 (en) * 2011-10-25 2013-05-09 Microsoft Corporation Estimating Quality of a Video Signal

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2007130389A2 (fr) * 2006-05-01 2007-11-15 Georgia Tech Research Corporation Procédé et système de mesure de la qualité vidéo automatique reposant sur des mesures de cohérence spatio-temporelles
WO2011082719A1 (fr) * 2010-01-11 2011-07-14 Telefonaktiebolaget L M Ericsson (Publ) Technique d'estimation de la qualité d'une vidéo
CN102959976A (zh) * 2010-04-30 2013-03-06 汤姆森特许公司 评估视频流质量的方法及设备
US20130114002A1 (en) * 2011-10-25 2013-05-09 Microsoft Corporation Estimating Quality of a Video Signal

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108696751A (zh) * 2017-04-11 2018-10-23 中国移动通信有限公司研究院 一种视频处理方法和装置
CN108696751B (zh) * 2017-04-11 2020-07-28 中国移动通信有限公司研究院 一种视频处理方法和装置

Similar Documents

Publication Publication Date Title
KR101414435B1 (ko) 비디오 스트림 품질 평가 방법 및 장치
US10075710B2 (en) Video quality measurement
JP5981561B2 (ja) シーンカットアーチファクトを考慮するビデオ品質評価
US9769501B2 (en) Video quality assessment at a bitstream level
US9723301B2 (en) Method and apparatus for context-based video quality assessment
US10536703B2 (en) Method and apparatus for video quality assessment based on content complexity
US9716881B2 (en) Method and apparatus for context-based video quality assessment
WO2014198062A1 (fr) Procédé et appareil pour la mesure de la qualité vidéo
US20150170350A1 (en) Method And Apparatus For Estimating Motion Homogeneity For Video Quality Assessment

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 13886880

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 13886880

Country of ref document: EP

Kind code of ref document: A1