JP6145069B2 - Subjective image quality estimation device and subjective image quality estimation program - Google Patents

Description

  The present invention relates to a subjective image quality estimation apparatus and a subjective image quality estimation program for estimating the subjective image quality of a moving image.

  With the spread of mobile lines such as 3G (3rd Generation) and LTE (Long Term Evolution) and mobile terminals such as smartphones and tablets, it is important to be able to deliver moving images to multiple different terminals even under unstable networks. It has become.

  There is HAS (Http Adaptive Streaming) as a method for distributing moving images in accordance with the difference in dot density (dots per inch: dpi) for each terminal and the difference in available bandwidth depending on the network. In recent years, HAS has been standardized as Moving Picture Experts Group-Dynamic adaptive streaming over HTTP (MPEG-DASH) (see, for example, Non-Patent Document 1).

  In MPEG-DASH, distribution is performed by dynamically switching the bit rate according to the network status of the viewer. In order to improve the distribution quality when the number of pixels and the frame rate change at the assumed distribution bit rate, the image quality needs to be evaluated quantitatively. Here, the number of pixels is the total number of pixels in a moving picture (frame), and is a value of the number of vertical pixels × the number of horizontal pixels. Specifically, when distributing a moving image at 2000 [kbps], an image quality evaluation for determining whether the number of pixels is 2073600 (= 1920 × 1080) or 921600 (= 1280 × 720); There is image quality evaluation for determining whether the frame rate is 30 [fps] or 15 [fps].

  In the conventional image quality evaluation, ITU-R recommendation BT. 500-11, ITU-T recommendation P.500. Subjective evaluation experiments have been performed using the methods defined in 910 and the like (see, for example, Non-Patent Documents 2 and 3). In the conventional image quality evaluation, a moving image is presented to about 20 subjects, and a rating is given to the moving image according to the subjectivity of the subjects. The average of the scores by the test subject is called a mean opinion score (MOS), and is the quality of the evaluated moving image.

  In these subjective evaluation experiments, the evaluation environment must be prepared according to regulations and the subjects must be gathered. Furthermore, in these subjective evaluation experiments, it is necessary to have the subject watch the moving image to be evaluated many times and receive a score. For this reason, there is a problem that it is expensive to perform subjective evaluation on individual moving images.

In view of this, objective image quality evaluation for calculating a feature of a moving image (an image feature amount, sometimes an image feature vector) and deriving the quality of the moving image from the calculated feature is being studied. ITU-T J.I. 143 (see, for example, Non-Patent Document 4) defines a framework for objective image quality evaluation methods. The objective image quality evaluation method framework is classified into the following three categories.
(I) Full Reference (FR) type This is an evaluation method using an original image and a decoded image before compression (before encoding), or a transmission image and a reception image.
(Ii) No Reference (NR) type This is an evaluation method using only a decoded image or a received image.
(Iii) Reduced Reference (RR) type This is an evaluation method using image characteristics of an original image or transmission image with a limited amount of information and a decoded image or a received image.

  Many subjective image quality estimation methods belong to one of the above three classifications (see, for example, Non-Patent Documents 5 to 7). Here, the operation procedure up to the construction of the FR type subjective image quality estimation formula will be described. FIG. 12 is a flowchart showing an operation procedure up to construction of a subjective image quality estimation formula according to the prior art.

First, pre-coding data (original image) I _i (0 <i ≦ N, N is an integer equal to or greater than 1) and test image D _i are prepared and subjective evaluation experiments are performed to estimate subjective image quality. The expression acquires the subjective evaluation value s _i (step S81).

Next, features f ₁ ,..., F _M are extracted from the pre-coding data (original image) I _i and test image D _{i of} the moving image, and the subjective image quality estimation formula s (f ₁ ,..., F _M | D _i , I _i ) is constructed for N combinations (step S82).

Then, a subjective evaluation value _{S i,} the subjective quality estimation formula _{s (f 1, ..., f} M | D i, I i) by a combination of the subjective quality estimation formula _{s (f 1, ..., f} M | D The parameters of _i , I _i ) are optimized to obtain a subjective image quality estimation formula s (f ₁ ,..., f _M ) for an arbitrary moving image (step S83).

Then, the features f ₁ ,..., F _M extracted from the pre-encoding data (original image) I and the decoded image D of any moving image and the subjective image quality estimation formula s (f ₁ ,..., F _M ) Based on this, the subjective image quality is estimated (step S84). Here, the features f ₁ ,..., F _M include cases where pixel signals are featured.

In the RR type, pixel signals are not included. In the NR type, not based on the subjective image quality estimation formula s (f ₁ ,..., F _M | D _i , I _i ), but based on the subjective image quality estimation formula s (f ₁ ,..., F _M | D _i ) The parameters are optimized.

However, the conventional subjective image quality estimation method using these frameworks assumes a specific display environment and is not a method corresponding to changes in the number of pixels and the frame rate. As a subjective image quality estimation method corresponding to changes in the number of pixels and the frame rate, there is a subjective image quality estimation method in scalable video coding (see, for example, Non-Patent Documents 8 and 9). In Non-Patent Document 8, subjective image quality is estimated based on equation (1).

  Here, SV is “Spatial Activity” defined in the MPEG-7 standard. TV is “Temporal Activity” defined in the MPEG-7 standard. Further, FR shown in the equation (1) is a frame rate. Further, x indicates the number of horizontal pixels of the moving image. The subjective image quality estimation formula (1) expresses the influence of changes in the number of pixels and the frame rate by SV and TV in addition to the conventional objective image quality evaluation value such as PSNR (peak signal to noise ratio). Therefore, this subjective image quality estimation formula can compare the subjective image quality with respect to different pixel numbers and frame rates that cannot be compared only with PSNR.

ISO / IEC 23009-1: 2012 TECHNICAL CORRIGENDUM 1, "Information technology-Dynamic adaptive streaming over HTTP (DASH)-Part 1: Media presentation description and segment formats", 2013-06-01. Recommendation ITU-R BT.500-13, “Methodology for the subjective assessment of the quality of television pictures”, January 2012. ITU-T Recommendation P.910, “Subjective video quality assessment methods for multimedia applications” ITU-T Recommendation J.143, “User requirements for objective perceptual video quality measurements in digital cable television”, May 2000. ITU-T Recommendation J.144, “Objective perceptual video quality measurement techniques for digital cable television in the presence of a full reference”, March 2004. ITU-T Recommendation J.247, “Objective perceptual multimedia video quality measurement in the presence of a full reference”, August 2008. ITU-T Recommendation J.246, “Perceptual audiovisual quality measurement techniques for multimedia services over digital cable television networks in the presence of a reduced bandwidth reference”, August 2008. CSKim SHJin, DJSeo and YMRo., "Measuring Video Quality on Full Scalability of H.264 / AVC Scalable Video Coding," IEICE Transaction on Communication, vol.E91-B, no. 5, May 2008 pp.1269 -78. H.Sohn, H.Yoo, WDNeve, CSKim, and YMRo., "Full-Reference Video Quality Metric for Fully Scalable and Mobile SVC Content," IEEE Transactions on Broadcasting, vol.56, no.3, Sept 2010 , pp.269-280.

  In a situation where moving images are encoded for a plurality of bit rates, if it is desired to determine the number of pixels and the frame rate so that the subjective image quality is maximized for each moving image, the subjective image quality for each moving image needs to be estimated. is there. Considering that the framework of the conventional objective image quality evaluation method is applied to the subjective image quality estimation by HAS, in the above three frameworks (FR type, RR type, NR type), the subjective image quality is a moving image. Cannot be evaluated before encoding, and the moving image is evaluated after being encoded many times.

  FIG. 13 is a flowchart showing an operation procedure for estimating the subjective image quality and determining the number of pixels and the frame rate using the conventional framework. In HAS, first, a moving image is encoded based on the number of candidate pixels, frame rate, bit rate, or quantization parameter (step S91). Subsequently, the obtained encoded data is decoded, and a subjective image quality estimated value is calculated (step S92). Then, it is determined whether or not subjective image quality has been estimated for all candidates for the number of pixels, the frame rate, and the bit rate (step S93).

  If the number of pixels, frame rate, and bit rate candidates for which subjective image quality has not been estimated remain, encoding is performed for the number of pixels, frame rate, and bit rate candidates that have not yet been encoded, and the process returns. On the other hand, if the subjective image quality is estimated for all candidates for the number of pixels, the frame rate, and the bit rate, the process proceeds. Finally, the number of pixels and the frame rate at which the subjective image quality estimation value is maximized are determined from the candidate combinations and output (step S94).

  However, in the above method for estimating the subjective image quality by constructing the subjective image quality estimation formula, the encoding and decoding of the moving image must be executed many times, and the calculation cost becomes high. That is, the conventional subjective image quality estimation apparatus estimates the subjective image quality of a moving image with high accuracy unless the encoding is repeated when estimating the subjective image quality of the moving image with respect to a desired bit rate, number of pixels, and frame rate. There is a problem that can not be.

  The present invention has been made in view of such circumstances. When estimating the subjective image quality of a moving image with respect to a desired bit rate, the number of pixels, and the frame rate, the subjective image quality of the moving image is reduced while reducing the calculation cost. It is an object of the present invention to provide a subjective image quality estimation device and a subjective image quality estimation program capable of estimating with high accuracy.

  The present invention provides a moving image conversion unit that converts an input moving image into a moving image having a predetermined number of pixels and a frame rate, and a first image feature that represents the complexity of encoding the input moving image. An image feature calculation unit that calculates a second image feature that is an image feature of the moving image after conversion, and a prediction that predicts a quantization parameter from the first image feature using an average code amount per pixel An image parameter deriving unit that generates an image parameter used to describe an expression; and the second image feature, the number of pixels, the frame rate, and the average code amount per pixel as an input, and the image parameter is used. And a subjective image quality estimation unit that estimates a subjective image quality estimation value using a prediction formula.

  In the present invention, the image parameter derivation unit further includes an image parameter database storing a set of a plurality of the image parameters and the first image features, and the first image features stored in the image parameter database. And an image parameter corresponding to the first image feature stored in the image parameter database selected using the error is calculated by calculating an error between the image feature calculation unit and the first image feature calculated by the image feature calculation unit. A weighted sum that is derived from a parameter database and is a weighted average of the image parameters derived using the error as a weight is generated as the image parameter.

  In the present invention, the image parameter derivation unit further includes an image parameter database storing a set of a plurality of the image parameters and the first image features, and the first image features stored in the image parameter database. And the first image feature calculated by the image feature calculation unit, and the image parameter corresponding to the first image feature stored in the image parameter database selected using the error is calculated. It is derived from the image parameter database, and a median value of the derived image parameters is generated as the image parameter.

  In the present invention, the image parameter derivation unit further includes an image parameter database storing a set of a plurality of the image parameters and the first image features, and the first image features stored in the image parameter database. And the first image feature calculated by the image feature calculation unit, and the image parameter corresponding to the first image feature stored in the image parameter database selected using the error is calculated. It is derived from the image parameter database, and an average value of the derived image parameters is generated as an image parameter.

  The present invention is a subjective image quality estimation program for causing a computer to function as the subjective image quality estimation device.

  According to the present invention, it is possible to accurately estimate the subjective image quality of a moving image while reducing the calculation cost.

It is a block diagram which shows the structure of one Embodiment of this invention. It is a flowchart which shows the processing operation which the subjective image quality estimation apparatus shown in FIG. 1 calculates a subjective image quality estimated value. It is a flowchart which shows the processing operation of the moving image converter 2 shown in FIG. It is a flowchart which shows operation | movement of the 1st function in the image feature calculation part 3 shown in FIG. It is a flowchart which shows operation | movement of the 2nd function in the image feature calculation part 3 shown in FIG. It is a flowchart which shows the processing operation of the image parameter derivation | leading-out part 4 shown in FIG. It is explanatory drawing which shows the table structure of an image parameter database. It is a flowchart which shows the processing operation of the subjective image quality estimation part 5 shown in FIG. It is a figure which shows the QP-MOS graph of "1280 * 720" and 30 [fps] in every embodiment of this invention for every moving image. It is a figure which shows the 960-540, 30 [fps] QP-MOS graph in embodiment of this invention for every moving image. It is a figure which shows the 960-540 and QP-MOS graph of 15 [fps] for every moving image in embodiment of this invention. It is a flowchart which shows the operation | movement procedure until construction of the subjective image quality estimation formula by a prior art. It is a flowchart which shows the operation | movement procedure which estimates a subjective image quality with the conventional framework, and determines a pixel number and a frame rate.

  Hereinafter, a subjective image quality estimation apparatus according to an embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing the configuration of the embodiment. The subjective image quality estimation device 1 includes a moving image conversion unit 2, an image feature calculation unit 3, an image parameter derivation unit 4, and a subjective image quality estimation unit 5.

The moving image conversion unit 2 inputs a moving image (original image) I before encoding, the number of pixels p for which subjective image quality is to be estimated, and a frame rate f from the outside. The moving image conversion unit 2 outputs moving images I _p and _f obtained by converting the moving image before encoding of the moving image into the number of pixels p and the frame rate f.

The image feature calculation unit 3 has two functions. One function calculates and outputs an image feature Ac for expressing the complexity (details will be described later) of moving image encoding from the moving image I before encoding. Another function receives the moving images I _p , _f having the number of pixels p and the frame rate f, and calculates the image feature Bv in the converted moving images I _p , _f .

  The image parameter deriving unit 4 calculates image parameters ε and δ (details will be described later) necessary for estimating the subjective image quality. The image parameter deriving unit stores image parameters for each of a plurality of moving images in advance in a database (image parameter database), and the image parameters are associated with image features of a plurality of pre-encoded moving images calculated in advance. It has been. The image parameter deriving unit 4 calculates an error between the image feature A of the moving image before encoding for estimating the subjective image quality and the image feature held in the image parameter database, and based on the error, a plurality of image parameters having a small error. Weighted and summed and output as image parameters ε and δ of the moving image for estimating the subjective image quality.

  The subjective image quality estimation unit 5 receives the number of pixels p, the frame rate f, the average code amount b per pixel, the image feature Bv, and the image parameters ε and δ, and receives them as the subjective image quality estimation formula s (p, By substituting f, b, v, ε, δ) as parameters, the subjective image quality estimation value s is output as an estimation result.

Next, a processing operation in which the subjective image quality estimation apparatus shown in FIG. 1 calculates a subjective image quality estimation value will be described with reference to FIG. FIG. 2 is a flowchart showing a processing operation in which the subjective image quality estimation apparatus shown in FIG. 1 calculates a subjective image quality estimated value. Here, assuming that the moving image I before encoding is encoded into a combination of a plurality of pixels, a frame rate, and an average code amount per pixel (M combinations), the subjective image quality estimation value is obtained. . The number of pixels in each combination is p _j , the frame rate is f _j , and the average code amount per pixel is b _j (0 <j ≦ M).

First, the image feature calculation unit 3 inputs the moving image data I before encoding (step S1), and sends the image feature Ac to the image parameter derivation unit 4. The image parameter deriving unit 4 calculates image features Ac and image features and errors (details will be described later) of the image parameter database. The top K image parameters ε _k , δ _k (0 <k ≦ K) with low errors are weighted based on the errors, and are set as ε, δ (step S2). Then, the image parameter deriving unit 4 outputs the image parameters ε and δ to the subjective image quality estimating unit 5.

Next, the moving image conversion unit 2 inputs the moving image data I, and outputs the moving images I _pj and _fj converted into the combinations (p _j , f _j ) of candidates to be encoded to the image feature calculation unit 3. The image feature calculation unit 3 receives the converted moving image as an input, obtains an image feature Bv _j (step S3), and outputs the image feature Bv _j to the subjective image quality estimation unit 5.

Next, the subjective image quality estimation unit 5 receives as input the image parameters ε, δ, the image feature Bv _j , the average code amount b _j , the number of pixels p _j , and the frame rate f _j. s (p, f, b, v, ε, δ) is obtained (step S4), the subjective image quality estimation value is calculated by the subjective image quality estimation formula s (p, f, b, v, ε, δ), and output. (Step S5).

ここで、ｂ_ｊはビットレート、画素数、フレームレートにより算出するので、固定ビットレートの符号化であれば、符号化前にｂ_ｊを取得できる。その場合、ｂ_ｊは不図示の外部機能により算出されるとする。上記の処理を全ての組み合わせ（ｐ_ｊ，ｆ_ｊ，ｂ_ｊ）に対して主観画質推定値を出力するまで繰り返す（ステップＳ６）。 Note that b _j is calculated according to equation (2).

Here, since b _j is calculated from the bit rate, the number of pixels, and the frame rate, b _j can be acquired before encoding if encoding is performed at a fixed bit rate. In this case, b _j is calculated by an external function (not shown). The above processing is repeated until the subjective image quality estimation values are output for all the combinations (p _j , f _j , b _j ) (step S6).

  Next, details of each part shown in FIG. 1 will be described with reference to the drawings. First, the moving image conversion unit 2 shown in FIG. 1 will be described. FIG. 3 is a flowchart showing the processing operation of the moving image conversion unit 2 shown in FIG. The moving image conversion unit 2 first inputs the original image I (step S11), and up-samples or down-converts the input unencoded moving image (original image) I to a desired number of pixels p and a frame rate f. Sampling.

The downsampling filter and the upsampling filter may be properly used depending on the situation. For example, thinning may be used for downsampling the number of pixels, and a bilateral filter, a DCT interpolation filter, or the like may be used for upsampling. The frame rate downsampling may be thinned out, and the upsampling may be performed by frame copy or the like. Then, the moving image conversion unit 2 outputs the converted moving images I _p and _f (step S12).

  Next, the image feature calculation unit 3 shown in FIG. 1 will be described. The image feature calculation unit 3 has two functions, each of which will be described. FIG. 4 is a flowchart showing the operation of the first function in the image feature calculation unit 3 shown in FIG. First, the image feature calculation unit 3 inputs a moving image I before encoding (step S21), and calculates an image feature Ac that expresses the complexity of encoding (step S22).

  Here, the complexity of encoding is determined according to, for example, a difference in texture within each frame of a moving image and a difference in texture between frames. H. H.264 / AVC, H.H. In H.265 / HEVC, intra-frame prediction and inter-frame prediction are used to remove the correlation in the moving image. If the difference in texture in each frame and the difference in texture between frames are large, the code amount Has the property of becoming larger. That is, encoding is complicated means a moving image with low encoding efficiency. The complexity of encoding is determined by the difference in texture within each frame of the moving image and the magnitude of the difference in texture between frames.

（３）式の各要素は空間特徴量と時間特徴量に基づいて決定される。ここで、空間特徴量とはＩＴＵ−Ｔ勧告Ｐ．９１０のＳＩ算出時に各フレームで算出されるｓｔｄ_{ｓｐａｃｅ}［Ｓｏｂｅｌ（Ｆ_ｎ）］である。ＩＴＵ−Ｔ勧告Ｐ．９１０では時間特徴量も定義されており、ｓｔｄ_{ｓｐａｃｅ}［Ｍ_ｎ（ｉ，ｊ）］となる。 In view of the above properties, the image feature Ac is ITU-T recommendation P.264. A feature quantity such as a product of Spatial Information (SI) and Temporal Information (TI) defined in 910 (for example, Non-Patent Document 3), and a feature vector having a feature quantity as in Equation (3) are used. An example is given.

Each element of the equation (3) is determined based on the spatial feature value and the temporal feature value. Here, the spatial feature value is ITU-T recommendation P.I. Std _space [Sobel (F _n )] calculated in each frame when calculating 910 SI. ITU-T recommendation P.I. In 910, a temporal feature amount is also defined, which is std _space [M _n (i, j)].

ITU-T recommendation P.I. In SI defined in 910, a spatial feature amount is calculated for each frame of a moving image, and its maximum value is SI. SI _min is a case where the minimum value is selected. The same applies to TI _min . SI _ave is a value obtained by averaging the spatial feature amounts calculated in each frame. TI _ave is the same as SI _ave, and is an average of time feature values at each time. SI _var is a variance of spatial feature values calculated in each frame. TI _var is the variance of the time feature value at each time. Intensity _ave is an average luminance value of all pixels of the moving image. Assuming that the number of frames of a moving image whose subjective image quality is to be estimated is N and the frame number n at a certain point in time (0 <n <= N), these can be expressed by equations (4) to (12), respectively.

Ｉ_ｎは動画像のフレーム番号ｎでの平均輝度である。Ｓｏｂｅｌ（Ｆ_ｎ），Ｍ_ｎ（ｉ，ｊ）の定義はＩＴＵ−Ｔ勧告Ｐ．９１０と同等である。

I _n is the average luminance of the frame number n of the moving image. The definitions of Sobel (F _n ) and M _n (i, j) are defined in ITU-T Recommendation P.30. It is equivalent to 910.

FIG. 5 is a flowchart showing the operation of the second function in the image feature calculation unit 3 shown in FIG. The moving image I _p , _{f in} which the number of pixels and the frame rate are converted by the moving image conversion unit 2 is received as an input (step S31), and an image feature Bv corresponding to the moving image I _p , _f is calculated (step S32). . Here, v is an image feature that includes features in the spatial direction and the temporal direction of a moving image. As an example, the product of SI and TI is used.

Next, the image parameter deriving unit 4 shown in FIG. 1 will be described. FIG. 6 is a flowchart showing the processing operation of the image parameter deriving unit 4 shown in FIG. First, the image parameter deriving unit 4 receives the image feature Ac calculated by the image feature calculating unit 3 as an input (step S41). The image parameter deriving unit 4 includes an image parameter database shown in FIG. The image parameter database holds image parameters ε _i and δ _i and corresponding image features c _l (0 <l ≦ L).

ｃ（ｉ）、Ｃ_ｉ（ｉ）は特徴ベクトルのｉ番目の値である。 The image feature c _i has the same configuration as the image feature Ac described above, and is a feature amount or a feature vector that represents the complexity of encoding a moving image. All epsilon _i in the case of absolute difference, the feature vector of c and c _i is the case of the feature that is present the sum of absolute value differences of each dimension of c and c _i to the image parameter database _and tied to [delta] _i calculated by the image feature, the absolute value difference or the sum of absolute difference (error) is calculated D _i (step S42). D _i if c is a C-dimensional vector (13) Arawaseru in formula.

c (i) and C _i (i) are i-th values of feature vectors.

There is also a method of outputting the image parameters ε _i and δ _{i associated} with c _i when the error is minimized to the subjective image quality estimation unit 5, but this fails to associate the image features with ε and δ. When a certain video is selected, ε and δ that are greatly different are selected, and an inappropriate image parameter is output to the subjective image quality estimation unit 5. Therefore, in order to prevent an inappropriate image parameter from being selected, the following processing is performed.

First, the top K image features C ₁ ,..., C _K and errors D ₁ ,..., D _K with few errors from the image feature Ac are selected from the image parameter database (step S43). Here, the order of the subscripts coincides with the order of the errors, and the error of the subscript 1 is minimized. Two threshold processes are performed for all errors D ₁ ,..., D _K.

Next, it is determined whether i = K (step S44). If i = K and the error D _i (0 <i ≦ K) falls below the threshold value Th _low (step S45), the index i is given a priority weight. The sum is stored in the sum list List1 (step S46). D _i is above the threshold _{Th low,} If you have less than the threshold _{Th high} (step S47), to store and to the index i to the weighted sum list List2 (step S48). This is performed for all i (until i becomes K). If no index is stored in the priority weighted sum list List1 and the weighted sum list List2, the average of all image parameters in the image parameter database is output.

If an index is stored in the priority weighted sum list List1 (step S49) and the number of stored data is 1 (step S50), ε _i and δ _i corresponding to the stored index _i are set. Output (step S51). If the stored number is 3 or more (step S52), outlier processing described later is performed, and List1 is updated (step S53). When the number of storage is two, List1 is not updated.

Next, the weight corresponding to the error amount is calculated using the updated or not updated List 1 (step S54). Then, the weight _w 1, ..., a _{w k c 1, ..., ε} 1 was tied to _{c k, ..., ε k,} δ 1, a sum weighted by multiplying each of the ... δ _k, ε, δ Is calculated (step S55).

On the other hand, when the index is not stored in the priority weighted sum list List1 (step S49), the index of the weighted sum list List2 is stored (step S56), and the stored number is one. (Step S57) outputs ε _i and δ _i corresponding to the stored index i (Step S58). Then, the average of ε and δ held in the image parameter database is output (step S59).

If the stored number is 3 or more (step S60), outlier processing described later is performed, and List2 is updated (step S61). List2 is not updated when the number of storage is two. Next, a weight corresponding to the error amount is calculated using List2 that has been updated or not updated (Step S62). Then, the weight _w 1, ..., a _{w k c 1, ..., ε} 1 was tied to _{c k, ..., ε k,} δ 1, a sum weighted by multiplying each of the ... δ _k, ε, δ Is calculated (step S63).

  It should be noted that any outlier processing may be executed by employing K, which is generally an integer of 3 or more.

Next, the weighted sum method will be described. A weight w _k based on the error is calculated for ε _k and δ _k (0 <k <= K) associated with the top K image features having a low error. Using the weights, ε and δ _obtained by calculating a weighted sum for each ε _k and δ _k are output. As a result, even when the feature amount and the feature vector do not correspond to ε and δ, it is possible to prevent ε and δ that are greatly deviated from being output. W _k , ε, and δ can be expressed by equations (14) to (16).

Next, outlier processing will be described. Here, it is assumed that indexes of 1,... K are stored in the priority weighted sum list or the weighted sum list. Error _D 1, ... within the threshold _{Th diff} is the difference between the error of up to _{D K,} ie _| D ₁ -D k _| in the case of _{<Th diff, ε k,} of [delta] _k, the image parameters are largely deviated exclude. Specifically, these are the image parameters shown in equations (17) and (18).

ε _ave and δ _ave are calculated as in equations (19) and (20).

With these processes, it is possible to exclude moving images having greatly different image parameters even if errors are equal. In outlier processing, this processing is performed once for all indexes. Instead of performing outlier processing, a method may be used in which the priority weighted sum list or the average value or median value of the image parameters in the weighted sum list is output as ε _i and δ _i .

ｑは量子化パラメータ（ＱＰ）であり、ｂは１画素あたりの平均符号量である。（２１）式は１画素あたりの符号量の対数とＱＰが線形関係にあることを現しており、（２１）式のパラメータはエンコーダ、動画像の特徴に依存する。ε、δを求めるためには、少なくとも２回、異なる（画素数、フレームレート、ビットレート）の組み合わせで符号化を行い、実際にＱＰｑと１画素あたりの符号量ｂを２点以上取得することで求める。しかし、符号化を行うと計算量が増大するため、前述した画像パラメータ導出部により、主観画質を推定したい動画像に対しては符号化を行わずにε、δを算出する。 The image parameter deriving unit 4 outputs image parameters ε and δ necessary for subjective image quality estimation. The ε and δ will be described. Here, ε and δ are parameters of the equation (21).

q is a quantization parameter (QP), and b is an average code amount per pixel. Equation (21) shows that the logarithm of the code amount per pixel and QP are in a linear relationship, and the parameters of equation (21) depend on the characteristics of the encoder and moving image. In order to obtain ε and δ, encoding is performed at least two times with different combinations (number of pixels, frame rate, bit rate), and two or more QPq and code amount b per pixel are actually obtained. Ask for. However, since the amount of calculation increases when encoding is performed, the image parameter deriving unit described above calculates ε and δ without encoding the moving image for which the subjective image quality is to be estimated.

  Next, the subjective image quality estimation formula of the subjective image quality estimation unit 5 shown in FIG. 1 will be described. FIG. 8 is a flowchart showing the processing operation of the subjective image quality estimation unit 5 shown in FIG. First, the subjective image quality estimation unit 5 receives the number of pixels p, the frame rate f, and the average code amount b per pixel from the outside. The image feature Bv is received from the image feature calculation unit 3. Further, ε and δ are received from the image parameter deriving unit 4 (step S71). Based on these inputs, the subjective image quality estimation unit 5 outputs a subjective image quality evaluation value s by S (p, f, b, v, ε, δ). For example, S (p, f, b, v, ε, δ) includes formulas (22) to (25).

パラメータｄ_１，ｄ_２，ｄ_３，ｅ_１，ｅ_２，ｅ_３，ｇ_１，ｇ_２，ｇ_３，ｈ_１，ｈ_２，ｈ_３は、推定した主観画質の推定値（推定ＭＯＳ）と主観評価実験で得たＭＯＳとの誤差が最小になるように最適化される。例えば、ガウスーニュートン法などの最適化アルゴリズムにより最適化される。

The parameters d ₁ , d ₂ , d ₃ , e ₁ , e ₂ , e ₃ , g ₁ , g ₂ , g ₃ , h ₁ , h ₂ , h ₃ are the estimated subjective image quality estimates (estimated MOS) and It is optimized so that the error with the MOS obtained in the subjective evaluation experiment is minimized. For example, it is optimized by an optimization algorithm such as Gauss-Newton method.

  Expression (22), which is a subjective image quality estimation formula of the subjective image quality estimation unit 5, is based on the relationship between QP and MOS. 9 to 11 are diagrams in which the horizontal axis indicates QP and the vertical axis indicates MOS. As shown in these figures, QP and MOS have a certain relationship for each number of pixels and frame rate. However, since coding at a fixed bit rate cannot know QP in advance, there is a problem that QP cannot be known without coding. Therefore, the QP is predicted from the average code amount per pixel by the equation (21). The parameters of the relational expression (21) that expresses the relationship between the average code amount per pixel and the QP are ε and δ are different for each encoder and moving image. To obtain this, at least two encodings are required. Necessary. However, since the amount of calculation increases when encoding is performed, the subjective image quality can be estimated before encoding by obtaining ε and δ by the image parameter deriving unit 4.

  Further, the relationship between moving images does not change greatly in the number of pixels and the frame rate. In other words, in the QP-MOS graphs shown in FIGS. 9 to 11, the relative positional relationship of the graphs for each moving image does not change. For example, when compared with “children”, the moving image “river” obtains a high MOS even when the QP is high, and this is the same in the number of pixels and the frame rate. Depending on the image feature, the correction parameter is the image feature Bv, which makes it possible to estimate the subjective image quality with high accuracy.

  As described above, the subjective image quality estimation apparatus 1 shown in FIG. 1 includes a moving image conversion unit 2 for converting a moving image into a desired number of pixels and a frame rate, and encoding complexity from the moving image before encoding. An image feature calculation unit 3 that calculates an image feature B, which is a parameter of a subjective image quality estimation formula, from the image feature A and the converted moving image, and an image parameter necessary for the subjective image quality estimation formula is derived based on the image feature A Based on the image parameter deriving unit 4, the image feature B, the image parameter, the number of pixels, the frame rate, the average code amount per pixel, and a predetermined subjective image quality estimation formula A subjective image quality estimation unit 5 for calculating the estimated value of

  The image feature calculation unit calculates an image feature A representing the complexity of encoding based on a moving image (original image) before encoding. The image parameter deriving unit derives an image parameter based on the image feature A. The moving image conversion unit converts the moving image into a desired number of pixels and frame rate. Further, the image feature calculation unit calculates the image feature B of the moving image necessary for the predetermined subjective image quality estimation formula from the moving image converted by the moving image conversion unit. With this configuration, the subjective image quality estimation device and the subjective image quality estimation program use the image feature B, the number of pixels, the frame rate, the average code amount per pixel, and the image parameters to obtain a desired bit rate, pixel When a moving image is encoded so that the number and the frame rate are obtained, the subjective image quality of the moving image can be accurately estimated before encoding.

  You may make it implement | achieve the subjective image quality estimation apparatus 1 in embodiment mentioned above with a computer. In that case, a program for realizing this function may be recorded on a computer-readable recording medium, and the program recorded on this recording medium may be read into a computer system and executed. Here, the “computer system” includes an OS and hardware such as peripheral devices. The “computer-readable recording medium” refers to a storage device such as a flexible medium, a magneto-optical disk, a portable medium such as a ROM and a CD-ROM, and a hard disk incorporated in a computer system. Furthermore, the “computer-readable recording medium” dynamically holds a program for a short time like a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line. In this case, a volatile memory inside a computer system serving as a server or a client in that case may be included and a program held for a certain period of time. Further, the program may be for realizing a part of the functions described above, and may be a program capable of realizing the functions described above in combination with a program already recorded in the computer system. It may be realized using hardware such as PLD (Programmable Logic Device) or FPGA (Field Programmable Gate Array).

  As mentioned above, although embodiment of this invention has been described with reference to drawings, the said embodiment is only the illustration of this invention, and it is clear that this invention is not limited to the said embodiment. is there. Therefore, additions, omissions, substitutions, and other modifications of the components may be made without departing from the technical idea and scope of the present invention.

  When estimating the subjective image quality of a moving image with respect to a desired bit rate, the number of pixels, and the frame rate, it can be applied to an application in which it is essential to accurately estimate the subjective image quality of a moving image without performing encoding.

  DESCRIPTION OF SYMBOLS 1 ... Subjective image quality estimation apparatus, 2 ... Moving image conversion part, 3 ... Image feature calculation part, 4 ... Image parameter derivation part, 5 ... Subjective image quality estimation part

Claims (5)

A moving image conversion unit that converts the input moving image into a moving image having a predetermined number of pixels and a frame rate;
A first image feature representing the complexity of encoding of the input moving image, and an image feature calculating unit that calculates a second image feature that is an image feature of the converted moving image;
An image parameter deriving unit that generates an image parameter used to describe a prediction expression for predicting a quantization parameter using an average code amount per pixel from the first image feature;
A subjective image quality estimation unit configured to estimate a subjective image quality estimation value using the prediction formula using the image parameter by inputting the second image feature, the number of pixels, the frame rate, and an average code amount per pixel; A subjective image quality estimation apparatus comprising: The image parameter derivation unit includes:
An image parameter database storing a plurality of sets of the image parameters and the first image features;
An error between the first image feature stored in the image parameter database and the first image feature calculated by the image feature calculation unit is calculated, and stored in the image parameter database selected using the error. In addition, an image parameter corresponding to the first image feature is derived from an image parameter database, and a weighted sum obtained by taking a weighted average of the image parameters derived using the error as a weight is generated as the image parameter. The subjective image quality estimation apparatus according to claim 1. The image parameter derivation unit includes:
An image parameter database storing a plurality of sets of the image parameters and the first image features;
An error between the first image feature stored in the image parameter database and the first image feature calculated by the image feature calculation unit is calculated, and stored in the image parameter database selected using the error. 2. The subjective image quality according to claim 1, wherein the image parameter corresponding to the first image feature is derived from the image parameter database, and a median value of the derived image parameter is generated as the image parameter. Estimating device. The image parameter derivation unit includes:
An image parameter database storing a plurality of sets of the image parameters and the first image features;
An error between the first image feature stored in the image parameter database and the first image feature calculated by the image feature calculation unit is calculated, and stored in the image parameter database selected using the error. 2. The subjective image quality estimation according to claim 1, wherein the image parameter corresponding to the first image feature is derived from the image parameter database, and an average value of the derived image parameter is generated as an image parameter. apparatus.   A subjective image quality estimation program for causing a computer to function as the subjective image quality estimation device according to any one of claims 1 to 4.

Images