JP3835683B2 - Transmission image quality monitoring device - Google Patents

Description

である。
αは上記行列演算により、１ブロックあたり、８×８個の係数となる。係数抽出部１３は、このうち、いずれかの係数を抽出して、中央監視室８へ向けて伝送する。もちろん、伝送可帯域幅に応じて、１ブロックあたり複数の係数を伝送しても、あるいは逆に、1個の係数も伝送しないブロックがあっても良い。
【００２７】
この実施形態によれば、次のようなメリットがある。以前に、「伝送による劣化は、白色、つまり、確率的には各周波数成分に均等に影響する傾向がある」と説明したが、これはあくまで傾向であって、劣化によっては周波数上で低域や高域側などに偏りを持つものもある。
ＰＮ系列乗算は、画像にPN系列（疑似ランダム系列、＋１と−１がランダムに並んでいる）を掛け合わせることによって、このノイズを統計的に各周波数成分に均等にばらつかせる効果を有する。すなわち、第1の実施形態における直交変換を行なう直前に、PN系列を画像に掛け合わせ、しかる後に直交変換することによって、(5)式による推定の精度を向上させることができる。
【００２８】
また、本実施形態において、直交変換としてウォルシュアダマール変換（WHT）を使用することのメリットについて説明する。
WHTは、直交変換のうちでは、最も計算の簡単なものの一つである。すなわち、全ての演算が和演算または差演算であるため、装置化する際に、この演算をハードウェアで実現するにしてもソフトウェアで実現するにしても、非常に簡易にできる（フーリエ変換の場合、実数演算が必要になる）。これがWHTを直交変換として利用するメリットである。
【００２９】
次に、発明の第３実施形態を、図５のブロック図を参照して説明する。図５は、前記第１、第２特徴量抽出部４，５の第３の具体例を示すブロック図である。図５が図４と異なる所は、直交変換部１２としてフーリエ変換ＦＦＴを使用するようにした点、および係数抽出部１３の後段に自乗和演算部１５を設けた点であり、図４と同符号は同一または同等物を示す。
この実施形態では、直交変換部１２としてフーリエ変換ＦＦＴを用いるようにした点に第１の特徴がある。フーリエ変換ＦＦＴは、周波数スペクトルという意味でもっとも物理的に正確で適した変換方法である。これがフーリエ変換を直交変換として利用するメリットである。なお、実現に当たっては、計算量を削減するため、高速フーリエ変換を利用するのが適当といえる。
【００３０】
次に、直交変換としてフーリエ変換ＦＦＴを使用する場合に、任意の周波数成分を取り出す際、実数部と虚数部ではなく振幅値のみを取り出して処理するようにした点に第２の特徴がある。本実施形態では、中央監視室８へ送る情報量を、約半分（実数部と虚数部計2種類の代わりに振幅値のみを伝送するので）にして、より低速な回線で中央監視を可能とすることができる。この際、重要なことは、このように情報を半分にしても、MSEの推定精度を低下させずに済むということである。その理由は次の通りである。
【００３１】
上記のX(f)、Y(f)をフーリエ変換係数とし、振幅と位相分に分けて表現したものを次の(6)式、(7)式のようにおく。
X(f)=A_X(f)ｅ^{-j φ X(f)} ・・・(6)
Y(f)=A_Y(f)ｅ^{-j φ Y(f)} ・・・(7)
そうすると、前記(5)式の右辺の｜X(f i)−Y(f i)｜² は、次の式のようになる。
｜X(f i)−Y(f i)｜² ＝｜A_X（ｆi）ｅ^{-j φ X(fi)}−A_Y（ｆi）ｅ^{-j φ Y(fi)}｜²
【００３２】
さらに仮定として、次のように考える。すなわち、 伝送劣化は、各周波数成分の位相には影響を与えない。振幅成分にのみ影響を与える。実際、位相に影響を与える劣化とは、画面のずれなどのことを意味するので、これは妥当な仮定といえる
そうすると、前記(5)式は、次の(8)式のように変形できる。

つまり、ある周波数成分の振幅値（＝エネルギースペクトル密度の平方根）の入出力差分の2乗で、劣化が推定できることになる。実数部と虚数部を伝送する必要はなくなる。
【００３３】
図６は、本発明の第４実施形態を示すブロック図である。本実施形態では、特徴量抽出部４、５の前段に、人間の視覚特性を模した時空間フィルタ９、１０が適用されており、これらのフィルタ９、１０の後の信号に対して、第１〜第３実施形態で行ったのと同様の処理を行うことにより、視覚特性の重みつきのMSEを推定する。このフィルタとしては、2次元ディジタルフィルタを用いることができる。該2次元ディジタルフィルタの一例としては、例えば、本出願人による発明である特開平７−３０７９２５号公報「画像の動き推定装置」に記されたものを用いることができる。
【００３４】
以上の第１〜４の実施形態によれば、例えば画像圧縮符号化伝送装置等による符号化ノイズに基づく受信映像の信号対雑音比の劣化を正確に推定することが可能になる。その理由は、前記ＰＮ系列乗算により、直交変換した後のスペクトルがランダムになり、ある任意の抽出係数が統計的に見て全体を代表する傾向が強まるからである。
【００３５】
ところが、デジタル伝送路においてエラーが発生した場合には、原画像と受信画像との差異は画像全体には散らばらず、局所的なブロック状のエラーとなって現れる。すなわち、受信画面上のある部分だけ大きな劣化が発生し、その他の部分では殆ど劣化がないという状態になる。
このような場合には、前記のＰＮ系列乗算（画像を構成する各画素の符号をランダムに変化させること）だけでは、十分な拡散効果が得られない。すなわち、直交変換係数が、互いに独立にならず、ある程度の相関が残留してしまうことになる。その結果、信号対雑音比の推定精度が低くなることになる。
【００３６】
本発明の第５の実施形態は、上記の考察に基づいてなされたものであり、例えば、デジタル伝送路においてエラーが発生した場合に、信号対雑音比の推定精度が低くならない伝送画質監視装置を提供するものである。
この実施形態を、図７を参照して説明する。図中の、図１、図２と同じ符号は同一または同等物を示す。
【００３７】
送信側２の入力画像は、まずブロック分割部１１に入力し、任意のサイズのブロックに分割される。ブロックに分割された画像は、直交変換部１２に入力し、直交変換される。
直交変換により得られた変換係数はＰＮ系列乗算部３１に入力し、この変換領域においてＰＮ系列を乗算されて、スペクトル拡散される。この処理により、各変換係数はランダムに変更される。この処理の詳細は、前記第２、３の実施形態と同様である。なお、前記第２、３の実施形態では、直交変換の前に各画素がＰＮ系列乗算されるが、この実施形態では直交変換の後にＰＮ系列乗算するようにした点に特徴がある。
【００３８】
次に、ＰＮ系列乗算された各変換係数は、逆直交変換部３２に送られ、逆直交変換処理を行われる。前記直交変換処理がＦＥＴであった場合には逆ＦＥＴが行われ、ＷＨＴであった場合には逆ＷＨＴが行われる。なお、ＷＨＴの場合は、逆変換行列は順変換行列と同一になり、回路作成規模の削減が可能である。前記逆直交変換により得られた画素は、画素抽出部３３で、変換ブロック毎に任意の個数抽出され、中央監視室８へ送られる。
【００３９】
一方、受信側３の受信画像も、ブロック分割部１１ａ、直交変換部１２ａ、ＰＮ系列乗算部３１ａ、および逆直交変換部３２ａにより、前記入力側画像と同じ処理をされる。そして、画素抽出部３３ａにて、画素抽出部３３で抽出された前記と同じ位置でかつ同じ個数の画素が抽出され、これらの画素は中央監視室８へ入力される。
中央監視室８は、送受信側それぞれからの抽出画素に基づいて、図３に示されている処理を行い、ＭＳＥの推定を行う。
【００４０】
デジタル伝送路においてエラーが発生した場合のように、画像領域上で局所的な異常な値をもつ劣化は、直交変換スペクトル上では、広い範囲に渡って異常な値を持つといえる。したがって、直交変換スペクトル上で符号をランダムに変化させると、効果的な拡散（空間拡散）が可能になる。これにより、逆変換画素の選択抽出によるＭＳＥの推定が効果的に行えることになる。
【００４１】
次に、前記第５の実施形態の変形例を、図８を参照して説明する。この変形例は、前記ブロック分割部１１，１１ａの前に、それぞれ帯域制限フィルタ３４，３４ａ、サブサンプリング部３５，３５ａを設けた点に特徴がある。該帯域制限フィルタ３４，３４ａとして、例えば人間の視覚特性を模した時空間フィルタを使用し、該フィルタ適用後の信号に対して前記した直交変換、ＰＮ系列乗算等の処理を行うと、視覚特性の重みを付された画質劣化を推定することができるようになり、人間の主観画質評価に近い画質劣化に基づく、伝送画質の自動遠隔監視が実現できる。また、サブサンプリング部３５，３５ａの処理を行うと、計算量の負荷を軽減できるようになる。なお、前記フィルタ３４，３４ａの一例としては、本出願人による特許出願、特願平６−１２１７４８号「画像の動き推定装置」に開示されている２次元ディジタルフィルタを用いることができる。
【００４２】
次に、本発明の第６の実施形態を、図９を参照して説明する。この実施形態は、図７の直交変換部１２、１２ａの前段に、それぞれＰＮ系列乗算部４１，４１ａを付加した点に特徴があり、他の構成は図７と同一または同等である。
本実施形態では、ブロック分割部１１で分割されたブロックの画像は、第２、第３実施形態と同様に、ＰＮ系列乗算部４１に入り、スペクトル拡散される。次いで、直交変換部１２に入力し、直交変換される。次に、直交変換された符号は、第５の実施形態と同様に、ＰＮ系列乗算部３１に入り、スペクトル拡散される。以降の動作は前記第５の実施形態と同様であるので、説明を省略する。
この実施形態によれば、直交変換の前後でＰＮ系列を乗算するようにしているので、符号化ノイズなどの広範囲な劣化に対しても、また伝送路エラー等に起因する局所的な劣化に対しても、有効に拡散が可能になり、受信映像の信号対雑音比の劣化を正確に推定できるようになる。
【００４３】
なお、本実施形態の変形例として、ブロック分割部１１，１１ａの前段に、第８図に示されているように、帯域制限フィルタ３４，３４ａ、サブサンプリング部３５，３５ａを入れてもよい。これらを入れると、視覚特性の重みを付された画質劣化を推定することができるようになると共に、計算量の負荷を軽減できるようになる。
【００４４】
【発明の効果】
以上の説明から明らかなように、本発明によれば、チェーン各箇所と中央監視室の間の伝送路が細くても有効にMSEを推定することができ、正確な伝送画質の遠隔監視が可能となる。
【００４５】
また、本発明では、ＰＮ系列乗算したのち直交変換するようにしたので、符号化ノイズなどの広範囲な劣化に対して、正確な伝送画質の遠隔監視が可能となる。
また、本発明では、特に、直交変換としてWHT（ウォルシュアダマール変換）を用いることによって、伝送画質の推定精度を落さずに計算量を少なくできる。すなわち、正確な伝送画質の遠隔監視が簡易に実現できる。また、本発明では、実数部と虚数部を伝送する必要はなくなり、少ない抽出量有効にMSEを推定することができるようになる。
【００４６】
また、本発明では、直交変換の後にＰＮ系列乗算するようにしたので、伝送路エラー等に起因する局所的な劣化に対して、正確な伝送画質の遠隔監視が可能となる。
また、本発明では、直交変換の前後にＰＮ系列乗算するようにしたので、、符号化ノイズなどの広範囲な劣化に対しても、また伝送路エラー等に起因する局所的な劣化に対しても、正確な伝送画質の遠隔監視が可能となる。
【００４７】
また、本発明では、視覚特性を模したフィルタを映像に対して適用してから処理を行なうようにしたので、視覚特性重み付きのMSEを推定することができる。すなわち人間の主観評価に基づいた伝送画質の遠隔監視が実現できる。
【図面の簡単な説明】
【図１】 本発明の一実施形態の構成を示すブロック図である。
【図２】 図１の特徴量抽出部の一具体例のブロック図である。
【図３】 図１のMSE推定部の一具体例のブロック図である。
【図４】 本発明の第２実施形態の特徴量抽出部のブロック図である。
【図５】 本発明の第３実施形態の特徴量抽出部のブロック図である。
【図６】 本発明の第４実施形態の構成を示すブロック図である。
【図７】 本発明の第５実施形態の構成を示すブロック図である。
【図８】 第５実施形態の変形例を示すブロック図である。
【図９】 本発明の第６実施形態の構成を示すブロック図である。
【符号の説明】
１・・・伝送路、４、５・・・特徴量抽出部、６、７・・・低速回線、８・・・中央監視室、８ａ・・・MSE推定部、１１・・・ブロック分割部、１２，１２ａ・・・直交変換部、１３・・・係数抽出部、１４・・・ＰＮ系列乗算部、１５・・・自乗和演算部、２１・・・減算部、２２・・・自乗演算部、２３・・・加算部、２４・・・テーブル参照部、３１，３１ａ、４１，４１ａ・・・ＰＮ系列乗算部、３２，３２ａ・・・逆直交変換部、３３，３３ａ・・・画素抽出部。[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a transmission image quality monitoring device, and in particular, a transmission image quality in video transmission suitable for a case where a plurality of transmission processing devices are interposed in a transmission line in a column and the transmission image quality in each processing place is collectively monitored remotely. It relates to a monitoring device.
[0002]
[Prior art]
Conventional automatic image quality monitoring / evaluation methods include (1) a method for comparing a processed image with its original image, (2) a method for evaluating the image quality using only the processed image, and (3) an original image and a processed image. There is a method of comparing feature quantities.
[0003]
Examples of the method (1) are disclosed in detail in the following document, for example.
T. Hamada, et al .: `` Picture quality assessment system by three-layered bottom-up noise weighting considering human visual perception, '' SMPTE Journal, Vol.108, No.1, Jan 1999.
[0004]
However, the method (1) cannot be used in actual video transmission sites. The reason is that only the processed image can be obtained during actual transmission, and the original image cannot be used.
[0005]
Therefore, in the past, only the method (2) or (3) could be used to monitor and evaluate the image quality during transmission. However, since the method (2) performs image quality evaluation only from the processed image, the evaluation accuracy is low in principle. On the other hand, the method (3) transmits a certain feature amount of the original image from the transmission side via a low-speed line and compares it with the same feature amount on the reception side, thereby enabling higher-accuracy image quality monitoring than the method (2). Can be realized. The invention relating to the method (3) has been filed as a patent application by the inventor as Japanese Patent Application No. 2000-256015 “Transmission image quality evaluation device and transmission image quality remote monitoring device” (August 25, 2000). .
[0006]
[Problems to be solved by the invention]
In the method (3), it is important to accurately evaluate the image quality with a small amount of features. In particular, it is important to estimate the MSE (mean square error) that is the basis of image quality evaluation, but no apparatus or method for accurately estimating the MSE with a small amount of features has been published so far.
Here, MSE is defined as follows. Assuming that the signal on the transmission side is x (t), the signal on the reception side (degraded) is y (t), and the total number of pixels is N, MSE can be expressed by the following equation (1).
MSE = 1 / N ・ Σ [x (t) −y (t)] ² (1)
Here, t is a two-dimensional coordinate composed of horizontal and vertical coordinates. The sum is taken for all pixels.
[0007]
The present invention has been made in view of the above-described prior art, and an object of the present invention is to reduce the amount of extraction (and thus central monitoring) in transmission image quality monitoring in which remote monitoring is performed by extracting feature amounts from the transmission side and the reception side. It is an object of the present invention to provide a transmission image quality monitoring device that effectively estimates the MSE when the line to the room is slower).
[0008]
Another object of the present invention is to provide a transmission image quality monitoring device capable of accurately estimating the degradation of the signal-to-noise ratio of a received video based on coding noise by an image compression coding transmission device or an error in a transmission path. There is to do.
[0009]
[Means for Solving the Problems]
In order to achieve the above-described object, the present invention provides a transmission image quality monitoring apparatus for monitoring transmission image quality on a video transmission path composed of a cascade connection of a plurality of transmission apparatuses, wherein the video image quality is determined at a plurality of points on the video transmission path. A feature amount extracting means for extracting a feature amount of the image data, and an MSE estimating means for estimating an MSE (mean square error) from the feature amount extracting means. Block dividing means for dividing the image in the block, orthogonal transform means for orthogonally transforming the video in the block, and frequency component value extracting means for extracting an arbitrary frequency component value of the orthogonal transform, wherein the MSE estimating means comprises the plurality of points From any of the frequency component values sent from, select any two points, square calculation means for obtaining the square value of the difference between the corresponding frequency component values, addition means for adding the square value by an arbitrary number, There is first characterized in that consists from the addition value obtained by the adding means and the image quality degradation estimation means for estimating the image quality degradation.
[0010]
According to this feature, since the video is divided into blocks of an arbitrary size and then subjected to orthogonal transformation for each block, the amount of calculation of orthogonal transformation per operation can be reduced. In addition, by extracting a certain frequency component from the frequency components obtained by orthogonal transformation and using it for estimation, degradation can be estimated with higher accuracy than when extracting the same amount of signals before orthogonal transformation by thinning out. It can be made easier. This is because image quality degradation such as encoding noise to be measured often exists over a wide frequency band.
[0011]
Further, in the present invention, the feature amount extraction unit includes a PN sequence multiplication unit that multiplies the signal in the block by a random sequence of +1 and −1 after the block division unit, and after the PN sequence multiplication, A second feature is that the orthogonal transformation is performed by the orthogonal transformation means.
[0012]
According to this feature, by performing PN sequence multiplication on an image before performing orthogonal transform, it is possible to frequency spread noise with biased frequency characteristics. Thereafter, orthogonal transformation is performed to extract a value of a certain frequency component, and a difference is obtained between the transmission side and the reception side, so that the average value of the noise is more suitable as a representative example. In other words, automatic remote monitoring of transmission image quality can be realized with higher accuracy.
[0013]
According to a third aspect of the present invention, there is provided PN sequence multiplication means for performing PN multiplication on a signal in a block after the orthogonal transformation means, and multiplying the coefficient after the orthogonal transformation by PN series. There is. According to this feature, accurate remote monitoring of transmission image quality is possible with respect to local degradation caused by a transmission path error or the like.
[0014]
Further, the present invention provides PN sequence multiplication means for performing PN multiplication on the signal in the block before and after the orthogonal transformation means, performs orthogonal transformation after PN sequence multiplication, and multiplies the coefficient after orthogonal transformation by PN sequence. There is a fourth feature in the point of doing so. According to this feature, accurate remote monitoring of transmission image quality is possible even for wide-ranging degradation such as coding noise and local degradation due to transmission path errors and the like.
[0015]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, the present invention will be described in detail with reference to the drawings. FIG. 1 is a block diagram showing a schematic configuration of an embodiment of the present invention applied to a system (referred to as a video transmission chain) in which a plurality of transmission processing devices are connected in cascade to a transmission path. FIG. 1 shows a partial configuration of the video transmission chain.
[0016]
In FIG. 1, it is assumed that a transmission image is transmitted from the transmission side 2 to the reception side 3 via the transmission path 1. The first feature amount extraction unit 4 extracts feature amounts from the transmission image on the transmission side 2. On the other hand, the second feature amount extraction unit 5 extracts the feature amount of the received image on the receiving side 3. The feature quantities extracted by the first and second feature quantity extraction units 4 and 5 are sent to the central monitoring room 8 through the low speed lines 6 and 7, respectively. The data relating to the feature amount is a frequency component value described later. In the central monitoring room 8, the MSE (mean square error) is estimated from these data by the MSE estimating unit 8a.
[0017]
In the present invention, the point is how to accurately estimate the following amount between the transmitted image and the received image, that is, MSE, from a small amount of data. MSE can be expressed by the following formula (2).
MSE = 1 / N · Σ [x (t) −y (t)] ² (2)
Here, the signal on the transmission side 2 is x (t), and the (deteriorated) signal on the reception side 3 is y (t).
[0018]
Although an image is originally a two-dimensional signal, it is represented here as a one-dimensional signal for convenience. If all x (t) and y (t) are obtained in the central monitoring room 8, a true MSE can be obtained, but this is usually not possible. Accordingly, the first and second feature quantity extraction units 4 and 5 extract information (feature quantities) of the transmission image and the reception image, respectively, and transmit them through the low bit rate lines 6 and 7 such as a telephone line and a LAN line, MSE is estimated in the MSE estimation part 8a of the central monitoring room 8 using those feature-values.
[0019]
Here, the present embodiment uses the feature that noise components that cause deterioration are superimposed over a wide frequency band. In other words, if the original signal is orthogonally transformed by some method and expressed as a set of frequency components, only arbitrary frequency components are sampled and sent to the monitoring room, the overall degradation can be estimated to some extent from the degradation of the frequency components. This principle can be said from Parseval's formula as follows.
[0020]
Signals obtained by orthogonal transformation of the signal x (t) on the transmission side 2 and the signal y (t) on the reception side 3 are set as X (f) and Y (f), respectively. If x (t) −y (t) ≡z (t) and the orthogonal transformation of z (t) is Z (f), the following expression (3) is established from Parseval's formula.
1 / NΣz (t) ² = 1 / N · Σ | Z (f) | ² (3)
Therefore, the following equation (4) is established from the equations (2) and (3).
MSE = 1 / N · Σ [x (t) −y (t)] ² = 1 / N · Σz (t) ² = 1 / N · Σ | Z (f) | ²
That is, MSE = 1 / N · Σ | X (f) −Y (f) | ² (4)
That is, the sum of squares of the differences between the frequency components after orthogonal transformation is MSE.
[0021]
Here, deterioration due to transmission tends to affect white, that is, each frequency component evenly in terms of probability. If this is used, one of the arbitrary frequency components f i can be used as an estimated amount of MSE (≡e ² ).
If this estimated amount is set to ^ e _i ² , the ^ e _i ² can be placed as in the following equation (5).
^ e _i ² = N | X (fi) −Y (fi) | ² (5)
[0022]
That is, deterioration can be estimated by the square of the input / output difference of a certain frequency component.
Furthermore, assuming that the deterioration of each frequency component is independent, the estimation accuracy is further improved by averaging the values of the formula (5) by using a plurality of frequency components to be used.
That is, MSE can be estimated by transmitting only frequency components X (fi) and Y (fi) for each block from the transmission side and the reception side.
[0023]
FIG. 2 is a block diagram showing a specific example of the configuration of the first and second feature quantity extraction units 4 and 5.
The input image is first input to the block dividing unit 11 and divided into blocks. However, the definition of “block” includes the entire screen. In addition, a three-dimensional block (cuboid) including the time axis direction is also included. That is, it is naturally possible to set the unit of orthogonal transformation to the entire screen or to three-dimensional orthogonal transformation including the time axis direction.
Next, the block is orthogonally transformed by the orthogonal transformation unit 12. Next, the coefficient extraction unit 13 extracts an appropriate coefficient from the coefficients obtained by the orthogonal transformation, and outputs it to the central monitoring room 8.
[0024]
FIG. 3 is a block diagram showing a specific example of the MSE estimation unit 8 a of the central monitoring room 8.
In the MSE estimator 8a, the difference between the coefficient values (amplitude values) at the same position in the corresponding blocks on the transmission side and the reception side is calculated by the subtraction unit 21 and squared by the square calculation unit 22. The square value is added by the adding unit 23 for an arbitrary number, for example, for each coefficient or / and for each block. The added value is applied to a table indicating the relationship between the added value and the MSE by the table reference unit 24, and the MSE is estimated.
[0025]
Next, a second embodiment of the present invention will be described with reference to FIG. FIG. 4 is a block diagram showing a second specific example of the configuration of the first and second feature quantity extraction units 4 and 5.
In FIG. 4, first, an input image is input to the block dividing unit 11 and divided into blocks. However, the definition of “block” is the same as described above. Next, the PN sequence multiplier 14 performs spectrum spreading on the signals in these blocks. That is, the block is multiplied by a PN sequence. The PN sequence may be an arbitrary sequence in which +1 and −1 are randomly generated, such as (+1, +1, -1, +1, -1, -1, +1, -1, +1, ...). Thereafter, WHT is applied in the orthogonal transform unit 12. As the WHT, for example, in the case of an image G having a block size of 8 × 8, the following α is obtained.
[0026]
α = UGU
here,

It is.
α is 8 × 8 coefficients per block by the above matrix calculation. The coefficient extraction unit 13 extracts one of these coefficients and transmits it to the central monitoring room 8. Of course, depending on the transmittable bandwidth, a plurality of coefficients may be transmitted per block, or conversely, there may be a block that does not transmit one coefficient.
[0027]
According to this embodiment, there are the following merits. Previously, it was explained that “degradation due to transmission tends to be white, that is, probabilistically affects each frequency component evenly”, but this is only a tendency, and depending on the degradation, it is a low frequency range. Some have a bias on the high frequency side.
The PN sequence multiplication has an effect of statistically evenly distributing the noise to each frequency component by multiplying the image by a PN sequence (a pseudo-random sequence, +1 and −1 are randomly arranged). That is, the accuracy of estimation according to the equation (5) can be improved by multiplying the image with the PN sequence immediately before performing the orthogonal transform in the first embodiment and then performing the orthogonal transform.
[0028]
Further, in the present embodiment, a merit of using Walsh Hadamard transform (WHT) as orthogonal transform will be described.
WHT is one of the simplest computations among orthogonal transforms. In other words, since all operations are sum operations or difference operations, it is very easy to implement this operation in hardware, whether it is realized by hardware or software (in the case of Fourier transform). , Real number operations are required). This is the merit of using WHT as orthogonal transform.
[0029]
Next, a third embodiment of the invention will be described with reference to the block diagram of FIG. FIG. 5 is a block diagram showing a third specific example of the first and second feature quantity extraction units 4 and 5. FIG. 5 differs from FIG. 4 in that a Fourier transform FFT is used as the orthogonal transform unit 12 and a square sum calculation unit 15 is provided after the coefficient extraction unit 13. The symbols indicate the same or equivalent.
This embodiment has a first feature in that a Fourier transform FFT is used as the orthogonal transform unit 12. The Fourier transform FFT is the most physically accurate and suitable transform method in the sense of frequency spectrum. This is an advantage of using the Fourier transform as an orthogonal transform. It should be noted that it is appropriate to use a fast Fourier transform in order to reduce the amount of calculation.
[0030]
Next, when Fourier transform FFT is used as orthogonal transform, there is a second feature in that when an arbitrary frequency component is extracted, only an amplitude value is extracted and processed instead of a real part and an imaginary part. In this embodiment, the amount of information sent to the central monitoring room 8 is approximately halved (since only the amplitude value is transmitted instead of the real part and the imaginary part), central monitoring can be performed with a lower speed line. can do. At this time, what is important is that even if the information is halved in this way, it is not necessary to reduce the estimation accuracy of the MSE. The reason is as follows.
[0031]
The above X (f) and Y (f) are used as Fourier transform coefficients, and expressed by dividing into amplitude and phase as shown in the following equations (6) and (7).
X (f) = A _X (f) e ^{-j φ X (f)} (6)
Y (f) = A _Y (f) e ^{-j φ Y (f)} (7)
Then, | X (fi) −Y (fi) | ^{2 on} the right side of the equation (5) is expressed by the following equation.
| X (fi) -Y (fi ) | 2 = | A X (fi) e -j φ X (fi) -A Y (fi) e -j φ Y (fi) | 2
[0032]
As a further assumption, consider the following. That is, transmission degradation does not affect the phase of each frequency component. It affects only the amplitude component. Actually, the deterioration affecting the phase means a screen shift or the like. Therefore, if this can be said to be a reasonable assumption, the equation (5) can be transformed into the following equation (8).

That is, deterioration can be estimated by the square of the input / output difference of the amplitude value of a certain frequency component (= square root of energy spectral density). There is no need to transmit the real part and the imaginary part.
[0033]
FIG. 6 is a block diagram showing a fourth embodiment of the present invention. In the present embodiment, spatio-temporal filters 9 and 10 simulating human visual characteristics are applied in front of the feature quantity extraction units 4 and 5. The weighted MSE of visual characteristics is estimated by performing the same processing as in the first to third embodiments. As this filter, a two-dimensional digital filter can be used. As an example of the two-dimensional digital filter, for example, the one described in Japanese Patent Laid-Open No. 7-307925 “Image Motion Estimation Device” which is an invention by the present applicant can be used.
[0034]
According to the first to fourth embodiments described above, it is possible to accurately estimate the degradation of the signal-to-noise ratio of the received video based on the coding noise, for example, by the image compression coding transmission device or the like. This is because the spectrum after orthogonal transformation becomes random due to the PN sequence multiplication, and an arbitrary extraction coefficient tends to be statistically representative of the whole.
[0035]
However, when an error occurs in the digital transmission path, the difference between the original image and the received image is not scattered throughout the image but appears as a local block error. That is, a large deterioration occurs only in a certain part on the reception screen, and there is almost no deterioration in other parts.
In such a case, a sufficient diffusion effect cannot be obtained only by the PN sequence multiplication (by randomly changing the sign of each pixel constituting the image). That is, the orthogonal transform coefficients are not independent from each other, and a certain degree of correlation remains. As a result, the estimation accuracy of the signal-to-noise ratio is lowered.
[0036]
The fifth embodiment of the present invention has been made on the basis of the above consideration. For example, a transmission image quality monitoring apparatus that does not reduce the estimation accuracy of the signal-to-noise ratio when an error occurs in a digital transmission path. It is to provide.
This embodiment will be described with reference to FIG. In the figure, the same reference numerals as those in FIGS. 1 and 2 denote the same or equivalent components.
[0037]
The input image on the transmission side 2 is first input to the block dividing unit 11 and divided into blocks of an arbitrary size. The image divided into blocks is input to the orthogonal transformation unit 12 and orthogonally transformed.
The transform coefficient obtained by the orthogonal transform is input to the PN sequence multiplication unit 31, and is multiplied by the PN sequence in this transform region, and is spread spectrum. By this processing, each conversion coefficient is randomly changed. Details of this processing are the same as those in the second and third embodiments. In the second and third embodiments, each pixel is multiplied by the PN sequence before the orthogonal transformation, but this embodiment is characterized in that the PN sequence is multiplied after the orthogonal transformation.
[0038]
Next, each transform coefficient multiplied by the PN sequence is sent to the inverse orthogonal transform unit 32 and subjected to inverse orthogonal transform processing. When the orthogonal transformation processing is FET, inverse FET is performed, and when it is WHT, inverse WHT is performed. In the case of WHT, the inverse transformation matrix is the same as the forward transformation matrix, and the circuit creation scale can be reduced. An arbitrary number of pixels obtained by the inverse orthogonal transform are extracted for each transform block by the pixel extraction unit 33 and sent to the central monitoring room 8.
[0039]
On the other hand, the received image on the receiving side 3 is also subjected to the same processing as the input side image by the block dividing unit 11a, the orthogonal transformation unit 12a, the PN sequence multiplication unit 31a, and the inverse orthogonal transformation unit 32a. Then, the pixel extraction unit 33 a extracts the same number of pixels at the same position extracted by the pixel extraction unit 33, and these pixels are input to the central monitoring room 8.
The central monitoring room 8 performs the process shown in FIG. 3 based on the extracted pixels from the transmission and reception sides, and estimates the MSE.
[0040]
It can be said that the degradation having an abnormal value locally on the image area has an abnormal value over a wide range on the orthogonal transform spectrum as in the case where an error occurs in the digital transmission path. Therefore, if the code is randomly changed on the orthogonal transform spectrum, effective diffusion (spatial diffusion) becomes possible. As a result, the MSE can be effectively estimated by selective extraction of the inverse transform pixels.
[0041]
Next, a modification of the fifth embodiment will be described with reference to FIG. This modification is characterized in that band limiting filters 34 and 34a and sub-sampling units 35 and 35a are provided in front of the block dividing units 11 and 11a, respectively. As the band limiting filters 34 and 34a, for example, a spatio-temporal filter simulating human visual characteristics is used, and when the above-described orthogonal transformation, PN sequence multiplication, and the like are performed on the signal after application of the filter, visual characteristics are obtained. The weighted image quality degradation can be estimated, and automatic remote monitoring of the transmission image quality based on the image quality degradation close to human subjective image quality evaluation can be realized. Further, if the processing of the sub-sampling units 35 and 35a is performed, the calculation load can be reduced. As an example of the filters 34 and 34a, a two-dimensional digital filter disclosed in Japanese Patent Application No. 6-121748 “Image Motion Estimation Device” by the present applicant can be used.
[0042]
Next, a sixth embodiment of the present invention will be described with reference to FIG. This embodiment is characterized in that PN sequence multipliers 41 and 41a are added to the previous stage of the orthogonal transform units 12 and 12a in FIG. 7, respectively, and the other configurations are the same as or equivalent to those in FIG.
In this embodiment, the image of the block divided | segmented by the block division part 11 enters into the PN series multiplication part 41 similarly to 2nd, 3rd embodiment, and is spread spectrum. Next, the signal is input to the orthogonal transform unit 12 and subjected to orthogonal transform. Next, the orthogonally transformed code enters the PN sequence multiplier 31 and is spread spectrum, as in the fifth embodiment. Subsequent operations are the same as those in the fifth embodiment, and a description thereof will be omitted.
According to this embodiment, since the PN sequence is multiplied before and after the orthogonal transformation, it is possible to prevent wide-ranging degradation such as coding noise and local degradation due to transmission path errors and the like. However, it is possible to effectively spread and to accurately estimate the degradation of the signal-to-noise ratio of the received video.
[0043]
As a modification of the present embodiment, band limiting filters 34 and 34a and sub-sampling units 35 and 35a may be inserted before the block dividing units 11 and 11a as shown in FIG. If these are included, it becomes possible to estimate the image quality degradation weighted by the visual characteristics, and to reduce the calculation load.
[0044]
【The invention's effect】
As is clear from the above description, according to the present invention, even when the transmission path between each part of the chain and the central monitoring room is narrow, the MSE can be estimated effectively, and remote monitoring of accurate transmission image quality is possible. It becomes.
[0045]
In the present invention, since the orthogonal transformation is performed after the PN sequence multiplication, it is possible to accurately monitor the transmission image quality accurately against a wide range of deterioration such as coding noise.
In the present invention, in particular, by using WHT (Walsh Hadamard transform) as the orthogonal transform, the amount of calculation can be reduced without degrading the estimation accuracy of the transmission image quality. That is, remote monitoring with accurate transmission image quality can be easily realized. In the present invention, it is not necessary to transmit the real part and the imaginary part, and the MSE can be estimated effectively with a small extraction amount.
[0046]
In the present invention, since the PN sequence multiplication is performed after the orthogonal transformation, it is possible to accurately monitor the transmission image quality accurately against local degradation caused by a transmission path error or the like.
In the present invention, since the PN sequence multiplication is performed before and after the orthogonal transformation, it is possible to deal with a wide range of degradation such as coding noise and local degradation caused by a transmission path error. Remote monitoring with accurate transmission image quality becomes possible.
[0047]
Further, in the present invention, the processing is performed after the filter imitating the visual characteristic is applied to the video, so that the MSE with the visual characteristic weight can be estimated. That is, remote monitoring of transmission image quality based on human subjective evaluation can be realized.
[Brief description of the drawings]
FIG. 1 is a block diagram showing a configuration of an embodiment of the present invention.
FIG. 2 is a block diagram of a specific example of a feature amount extraction unit in FIG. 1;
FIG. 3 is a block diagram of a specific example of the MSE estimation unit in FIG. 1;
FIG. 4 is a block diagram of a feature amount extraction unit according to a second embodiment of the present invention.
FIG. 5 is a block diagram of a feature amount extraction unit according to a third embodiment of the present invention.
FIG. 6 is a block diagram showing a configuration of a fourth embodiment of the present invention.
FIG. 7 is a block diagram showing a configuration of a fifth exemplary embodiment of the present invention.
FIG. 8 is a block diagram showing a modification of the fifth embodiment.
FIG. 9 is a block diagram showing a configuration of a sixth embodiment of the present invention.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 1 ... Transmission path, 4, 5 ... Feature-value extraction part, 6, 7 ... Low speed line, 8 ... Central monitoring room, 8a ... MSE estimation part, 11 ... Block division part , 12, 12a ... orthogonal transform unit, 13 ... coefficient extraction unit, 14 ... PN sequence multiplication unit, 15 ... square sum operation unit, 21 ... subtraction unit, 22 ... square operation , 23... Addition unit, 24... Table reference unit, 31, 31 a, 41, 41 a... PN sequence multiplication unit, 32, 32 a ... inverse orthogonal transform unit, 33, 33 a. Extraction unit.

Claims (8)

A transmission image quality monitoring device for monitoring the transmission image quality on a video transmission path composed of cascade connection of a plurality of transmission devices,
Feature quantity extracting means for extracting feature quantities of video image quality at a plurality of points on the video transmission path;
MSE estimation means for estimating MSE (mean square error) from the feature quantity extraction means,
The feature quantity extracting means is a block dividing means for dividing the input video into blocks of an arbitrary size, an orthogonal transform means for orthogonally transforming the video in the block, and a frequency component value for extracting an arbitrary frequency component value of the orthogonal transform Consisting of extraction means,
The MSE estimating means selects any two points from the frequency component values sent from the plurality of points, and calculates the square value of the difference between the corresponding frequency component values, and the square value is arbitrarily set. A transmission image quality monitoring apparatus comprising: addition means for adding the number of images; and image quality deterioration estimation means for estimating image quality deterioration from an addition value obtained by the addition means. In the transmission image quality monitoring device according to claim 1,
The feature amount extraction unit includes a PN sequence multiplication unit that multiplies a signal in a block by a random sequence of +1 and −1 before the orthogonal transform unit,
A transmission image quality monitoring apparatus wherein orthogonal transformation is performed by the orthogonal transformation means after the PN sequence multiplication. A transmission image quality monitoring device for monitoring the transmission image quality on a video transmission path composed of cascade connection of a plurality of transmission devices,
Feature quantity extracting means for extracting feature quantities of video image quality at a plurality of points on the video transmission path;
MSE estimation means for estimating MSE (mean square error) from the feature quantity extraction means,
The feature amount extracting means includes a block dividing means for dividing the input video into blocks of an arbitrary size, an orthogonal transform means for orthogonally transforming signals in the block, and a sequence in which +1 and -1 are random in the coefficient of the orthogonal transform PN multiplication means for multiplying, an inverse orthogonal transformation means for inverse orthogonal transformation of the coefficient multiplied by PN, and a pixel extraction means for extracting an arbitrary pixel value from the pixels obtained by the inverse orthogonal transformation,
The MSE estimating means selects a pixel value from any two points from the pixel values sent from the plurality of points and obtains a square value of the difference between the corresponding pixel values; A transmission image quality monitoring apparatus comprising: addition means for adding only image quality; and image quality deterioration estimation means for estimating image quality deterioration from an addition value obtained by the addition means. The transmission image quality monitoring device according to claim 3,
The transmission amount monitoring apparatus according to claim 1, wherein the feature amount extraction unit further includes a PN sequence multiplication unit that multiplies a signal in the block by a random sequence of +1 and -1 before the orthogonal transform unit. The transmission image quality monitoring device according to any one of claims 1 to 4,
The transmission image quality monitoring apparatus, wherein the orthogonal transform unit uses Walsh Hadamard transform (WHT) as orthogonal transform. The transmission image quality monitoring device according to any one of claims 1 to 4,
The transmission image quality monitoring apparatus, wherein the orthogonal transform unit uses Fourier transform as orthogonal transform. The transmission image quality monitoring device according to claim 6,
The transmission image quality monitoring apparatus, wherein the frequency component value extracting means extracts only an amplitude value instead of a real part and an imaginary part when extracting an arbitrary frequency component value. The transmission image quality monitoring device according to any one of claims 1 to 6,
A transmission image quality monitoring apparatus further comprising filter means for filtering input video before the feature quantity extraction means.

Images