JP2007019802A - Image quality estimation device, and image quality estimation method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To estimate image quality by a simple constitution with higher precision, and in more efficient manner. <P>SOLUTION: A network/terminal quality information analysis unit 71 and an image quality information measuring unit 72 are installed in a user terminal UT. The network/terminal quality information analysis unit 71 decides an intermediate parameter (for example, a number of deteriorated frames) which is necessary for quality deterioration estimation from received image packet information and transmission quality information by a packet for control. The image quality information measuring unit 72 extracts a media feature amount from a decoded image signal, and discriminates a deterioration kind of the image signal. A subjective quality estimation unit 73 presumes subjective quality of the decoded image in the user terminal UT using a quality estimation model established in advance, based on the intermediate parameter from the network/terminal quality information analysis unit 71 and the deterioration kind from the image quality information measuring unit 72. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a quality management technique for video services via a communication network, and more particularly to a video quality estimation apparatus and a video quality estimation method for estimating the quality of a video signal converted into a packet sequence and sent.

  In recent years, video communication services (real-time video applications) such as video distribution services and video conferencing are becoming widespread as communication networks such as the Internet have become broadband. In these video services, video signals are encoded with high efficiency, converted into packet sequences, and sent to user terminals (hereinafter referred to as user terminals).

  For example, an I (Intra Picture) frame that performs encoding processing within a frame without using inter-frame prediction, a P (Predictive Picture) frame that performs encoding processing by predicting the present from the past using motion compensation prediction, and bi-directional prediction The video signal is highly efficient compression encoded by three types of frames (B (Bidirectionally Predictive Picture) frames) that perform forward and backward prediction and encoding processing, and the high efficiency compression encoded video signal is converted into a packet sequence. The data is sent to the user terminal via the network.

  In this case, for example, when packet loss occurs in the network, the frame information before and after is used for decoding at the user terminal. Therefore, the deterioration of the video quality due to the loss of one packet is not limited to one video frame. There may be. In such a case, although there is no packet loss in the next video frame, quality degradation occurs in the decoded video, and the occurrence of packet loss (network quality) on the network and the video application It does not necessarily match the occurrence of quality degradation (application quality).

  Therefore, in Patent Document 1, in order to improve the accuracy of quality estimation, in the decoding of a specific part of a moving picture code, when the decoded value of another part is referred to, the decoding value of the specific part of the moving picture code is referred to. Then, it is detected that an arbitrary abnormality of the decoded value of the other part is spread, and the area of quality degradation is calculated for each video frame.

JP 2000-341688 (see paragraphs [0049], [0083] to [0089], [0111], etc.) ITU-T Recommendation G.107, “The E-Model, a computational model for use in transmission planning”, May 2000. A. Clark, “Modeling the Effects of Burst Packet Loss and Recensy on Subjective Voice Quality”, IP Telephony Workshop 2001, Apr. 2001. R. Caseres and A. Clark, "RTP Control Protocol Extended Reports (RTCP XR)", IETF RFC3611, May 2003.

  However, in the technique disclosed in Patent Document 1 described above, it is necessary to closely follow the status of the moving image code of the packet in the decoder, and the configuration becomes complicated.

  The present applicant has previously proposed a technique for estimating the quality of video decoded by a user terminal based on the video degradation area and degradation duration as Japanese Patent Application No. 2004-213138 (reference document 1). According to this technique, it is possible to accurately estimate video quality with a simple configuration. However, there are other quality degradation factors that are not taken into account in addition to the degradation area and degradation duration of video, and it is desired to estimate video quality with higher accuracy and efficiency.

  Other techniques such as Non-Patent Documents 1, 2, and 3 are shown as techniques for estimating and managing the user experience quality from the network quality. There are quality degradation factors, and it cannot be said that the accuracy is high.

  The present invention has been made to solve the above-described problems, and an object of the present invention is to provide a video that can estimate video quality more accurately and more efficiently with a simple configuration. The object is to provide a quality estimation device and a video quality estimation method.

  In order to achieve such an object, the present invention provides a decoding unit that receives and decodes a video signal transmitted after being converted into a packet sequence, and at least a required amount based on loss information of the packet before decoding the video signal. Intermediate parameter determination means for determining a parameter as an intermediate parameter, deterioration type determination means for analyzing the decoded video signal and determining the deterioration type of the video signal, intermediate parameter and deterioration type determination means determined by the intermediate parameter determination means And quality estimation means for estimating the video quality after decoding of the video signal based on the degradation type determined by.

  In the present invention, a required parameter is determined as an intermediate parameter based on packet loss information or the like in a network or a terminal, the decoded video signal is analyzed to determine a degradation type of the video signal, and the determined intermediate parameter is determined. The quality of the decoded video is estimated on the basis of the degradation type determined to be. For example, by setting the relationship between the intermediate parameter corresponding to the degradation type and the video quality in advance, the video quality corresponding to the determined intermediate parameter from the relationship between the intermediate parameter corresponding to the degradation type and the video quality (user The video quality experienced by In the present invention, the degradation type of the video signal is determined from, for example, the feature quantity of quality degradation extracted from the decoded video signal as the media feature quantity.

  When information about video is lost (for example, when media information, frame configuration information, etc. are lost due to packet loss on the network), a phenomenon such as skipping frames without playing all the frames with lost information, Depending on the application, various deterioration phenomena occur in the reproduced video, such as block distortion in which information lost in a frame is complemented or deleted for reproduction. These deterioration phenomena have a great influence on the quality experienced by the user, and recognizing these phenomena (degradation types) plays an important role in monitoring the quality experienced by the user. By estimating the quality experienced by the user in consideration of this degradation type, it is possible to estimate video quality with higher accuracy and efficiency.

  In the present invention, examples of the intermediate parameter include the number of deteriorated frames in the same group (for example, the same GOP). In this case, it is checked whether or not there are duplicate frames determined to be degraded in the same group. If there are duplicate frames judged to be degraded, the number of degraded frames is weighted according to the degradation type. It is conceivable to obtain the video quality (video quality experienced by the user) according to the weighted number of deteriorated frames from the relationship between the preset number of deteriorated frames and the video quality.

  In addition, it is checked whether there are duplicate frames that are judged as degraded in the same group. If there are duplicate frames that are judged as degraded, it is judged as degraded in the same group. The number of deteriorated frames is given a pre-set weight obtained from the measured value of the amount of degradation in video quality when there is a new frame, and this weighted number of deteriorated frames is multiplied by a weighting factor corresponding to the deterioration type. It is conceivable to obtain the video quality (video quality experienced by the user) corresponding to the number of deteriorated frames multiplied by the weighting factor from the relationship between the preset number of deteriorated frames and the video quality.

  Also, it is confirmed whether there is a frame degradation interval that falls within a predetermined range within the same group. If there is a frame degradation interval that falls within a predetermined range, the number of degraded frames is weighted according to the degradation type. It is conceivable to obtain the video quality (video quality experienced by the user) corresponding to the weighted number of deteriorated frames from the relationship between the preset number of deteriorated frames and the video quality.

  Also, it is confirmed whether there is a frame degradation interval that falls within a predetermined range within the same group. When there is a frame degradation interval that falls within the predetermined range, there is a frame degradation interval within the same group. If the number of deteriorated frames is given a pre-set weight obtained from the measured value of the amount of degradation in video quality, the weighted coefficient corresponding to the type of deterioration is multiplied by this weighted number of deteriorated frames. Considering the relationship between the number of deteriorated frames and the image quality, it is conceivable to obtain the image quality (image quality experienced by the user) corresponding to the number of deteriorated frames multiplied by this weighting factor.

  In the present invention, the video quality is obtained from the “relationship between the intermediate parameter and the video quality” according to the degradation type. However, the intermediate parameter is substantially weighted by weighting the intermediate parameter according to the degradation type of the video signal. The video quality is changed, the quality estimation model weighted according to the video signal degradation type is used, or the quality estimation model individually determined according to the video signal degradation type is used Various methods, such as doing, can be considered.

  Further, in the present invention, without determining the degradation type of the video signal, the intermediate parameter determination means determines the relationship between the preset intermediate parameter and the characteristic amount (media feature amount) of the quality degradation of the video signal. The feature quantity of quality degradation corresponding to the determined intermediate parameter is obtained (estimated feature quantity of quality degradation), and the quality degradation feature quantity (media feature quantity) and video quality (video The video quality (video quality experienced by the user) corresponding to the obtained characteristic amount of quality degradation may be obtained from the relationship with the video quality experienced.

  In addition, the determination of the degradation type of the video signal is not performed, and the “first relation model indicating the relationship between the intermediate parameter and the feature amount of the quality degradation of the video signal (media feature amount) and the video signal is set in advance. Quality estimation model indicating the relationship between the intermediate parameters determined from the second relationship model indicating the relationship between the quality degradation feature amount (media feature amount) and the video quality (video quality experienced by the user) From the above, the video quality corresponding to the intermediate parameter determined by the intermediate parameter determination means (video quality experienced by the user) may be obtained.

In the present invention, the relationship between the intermediate parameter and the video quality is not only modeled as a mathematical formula, but the relationship between the intermediate parameter and the video quality is organized and recorded as a table, which is used as a quality estimation model, etc. It is good.
Examples of the intermediate parameter include a TS packet loss rate, a TS packet loss rate considering the loss interval of TS packets, a deteriorated frame number, a deteriorated frame number considering the deterioration interval, and the like.

  Further, the present invention can be realized not only as a video quality estimation apparatus but also as a video quality estimation method. In the present invention, the video quality estimation may be performed not on the user terminal side but on the management terminal side connected to the network. In this case, the functions are distributed to the user terminal and the management terminal, and the apparatus that combines the user terminal and the management terminal is the video quality estimation apparatus according to the present invention.

  According to the present invention, necessary parameters such as the number of deteriorated frames are determined as intermediate parameters based on packet loss information in a network or a terminal, and the video signal deterioration type (frame skipping, block distortion, etc.) and quality deterioration are determined. The quality of the video decoded at the terminal is estimated in consideration of the feature quantity (media feature quantity), and the video quality can be estimated more accurately and more efficiently with a simple configuration. It becomes like this.

Hereinafter, the present invention will be described in detail with reference to the drawings.
[Embodiment 1]
FIG. 1 is a block diagram showing an example (Embodiment 1) of a video distribution system including a video quality estimation apparatus according to the present invention.

  In the figure, 1 is a video input device such as a camera or a playback device for recorded video, 2 is a video encoding device that performs high-efficiency compression encoding of a video signal from the video input device 1 and converts it into a packet sequence, and 3 is video encoding. A video distribution server 5 that distributes a video signal converted into a packet sequence sent from the device 2 to the user terminal UT via a network (Internet network) 4 is a video quality management device that manages video quality.

  In this system, the video distribution server 3 is provided in the distribution terminal DT, the video quality management device 5 is provided in the management terminal CT, and the distribution terminal DT, the user terminal UT, and the management terminal CT are connected to each other via the network 4. Has been. The user terminal UT includes a decoding device 6, a video quality estimation device 7, and a quality information communication device 8.

[Video coding device]
FIG. 2 shows a schematic configuration of the video encoding device 2. The video encoding device 2 includes a high-efficiency compression encoding unit 21, a packet conversion unit 22, and an information embedding unit 23. The high-efficiency compression encoding unit 21 performs high-efficiency compression encoding on the video signal from the video input device 1 using I frames, P frames, and B frames. These three types of frames are generated with regularity and encoded. The packet converting unit 22 converts the video signal subjected to the high efficiency compression encoding by the high efficiency compression encoding unit 21 into a packet sequence (packets P1, P2, P3,...) And sends the packet to the information embedding unit 23.

  The information embedding unit 23 includes a frame type and frame generation at the time of high-efficiency compression encoding to which the video packet P belongs in the TOS bit or payload in each header of the packet (video packet) P from the packet conversion unit 22 Embed rules. In this embodiment, I frame, P frame, and B frame types (I, P, and B) are used as frame types at the time of high-efficiency compression encoding, and I frame, P frame, and B frame generation rules are used as frame generation rules. GOP (Group of pictures) parameters are embedded. Specifically, the repetition period N of I picture and the repetition period M of P or I are embedded in the video packet P.

  It goes without saying that the generation number of the video packet P is written in the header of the video packet P as information for specifying the video packet P. In this embodiment, the information embedding unit 23 is provided at the subsequent stage of the packet converting unit 22, but the information embedding unit 23 may be provided at the preceding stage of the packet converting unit 22. That is, the frame type and the frame generation rule may be embedded in each video packet by attaching the frame type and the frame generation rule to the encoded data of each frame at the time of high-efficiency compression encoding.

[Video distribution server]
The video packet P in which the frame type and the frame generation rule at the time of high-efficiency compression encoding are embedded from the information embedding unit 23 is sent to the video distribution server 3. The video distribution server 3 sends the video packet P from the video encoding device 2 to the user terminal UT via the network 4. In addition, the video distribution server 3 acquires information (for example, the number of transmission packets, a sequence number described in a packet header, etc.) relating to a packet of a video signal to be transmitted to the user terminal UT as transmission quality information. Is sent to the user terminal UT by a control packet.

[Decoding device]
The user terminal UT receives the video packet P from the video distribution server 3 at the decoding device 6. The decoding device 6 receives the video packets P from the video distribution server 3 one after another, decodes the data of the video packets P, and also receives the received video packet information and the decoded video signal from the video quality estimation device 7. Send to.

[Video quality estimation device]
FIG. 3 shows an outline of the internal configuration of the video quality estimation apparatus 7. The video quality estimation apparatus 7 includes a network / terminal quality information analysis unit 71, a video quality information measurement unit 72, a subjective quality estimation unit 73, and a quality degradation factor / quality estimated value storage unit 74.

  The network / terminal quality information analysis unit 71 is provided with the information of the video packet received from the decoding device 6 and the transmission quality information by the control packet from the video distribution server 3 via the network 4. The video quality information measuring unit 72 is provided with the decoded video signal from the decoding device 6.

  The network / terminal quality information analysis unit 71 includes a packet information acquisition unit 711, a video frame identification unit 712, and a loss pattern determination unit 713.

  The packet information acquisition unit 711 acquires transmission quality information (quality information on the transmission side) from the control packet from the video distribution server 3. Further, quality information on the receiving side is acquired using information of the received video packet from the decoding device 6 and a size of a fluctuation absorbing buffer (not shown) in the decoding device 6.

  The video frame identification means 712 analyzes the information of the received video packet, and identifies the frame number and the frame type (I, P, B) for the arrived packet and the packet (lost packet) lost on the network / terminal. .

  The loss pattern determination unit 713 analyzes the loss pattern from the information obtained by the packet information acquisition unit 711 and the video frame identification unit 712, and determines a parameter necessary for video quality estimation as an intermediate parameter based on the loss pattern. .

  In this embodiment, the loss pattern determination means 713 recognizes the frame type of a lost packet that has been lost on the network / terminal, and how much the loss of the frame affects other frames. (For example, when the I frame is lost, the subsequent three frames are deteriorated, and it is understood how many frames each frame type affects depending on the GOP configuration).

  And when determining whether there are multiple losses in the frame and converting at the frame level, also recognize loss pattern information such as whether the frame is continuously lost, based on this information, Intermediate parameters required for video quality estimation (for example, number of degraded frames, number of lost packets, number of transmitted packets, number of received frames, received frame type, buffer leaked packet, buffering time, video / audio CODEC type within the same GOP Video bit rate, etc.). The intermediate parameters determined by the loss pattern determination unit 713 are sent to the subjective quality estimation unit 73.

The video quality information measurement unit 72 includes a feature amount extraction unit 721 and a deterioration type determination unit 722.
The feature amount extraction means 721 analyzes the decoded video signal from the decoding device 6 and extracts a quality deterioration feature amount (media feature amount). In the present embodiment, the quality deterioration information (for example, the deterioration area of the macroblock) of the macroblock of the output video is extracted as the media feature amount.

  Note that a macro block is a structural unit for displaying a video, and a single video can be displayed by arranging a plurality of these blocks. For example, in MPEG2, the macroblock unit is 16 × 16 pixels.

  The degradation type determination unit 722 determines the degradation type (phenomenon such as block distortion and frame skipping) of the video signal based on the media feature amount extracted by the feature amount extraction unit 721. For example, by checking the degradation area of frames arranged in time series, if the area changes in time series, it is recognized as block distortion and the area does not change in time series. If there is no change in the macro block information for a certain period of time, it is recognized as a skipped frame. The deterioration type determined by the deterioration type determination unit 722 is sent to the subjective quality estimation unit 73.

  In addition, when the deterioration type is determined in advance for the target application, the information is used. For example, when the degradation type is sent from the video distribution server 3 as quality information to the user terminal UT by the control packet, and the video degradation occurs in the user terminal UT, the degradation type from the video distribution server 3 is set as the subjective quality estimation unit. Used in 73.

  In the subjective quality estimation unit 73, a quality estimation model indicating the relationship between the intermediate parameter and the video quality (MOS) experienced by the user is determined and set in advance. The subjective quality estimation unit 73 uses the quality estimation model set in advance from the intermediate parameter from the network / terminal quality information analysis unit 71 and the degradation type of the video signal from the video quality information measurement unit 72 to The quality (subjective quality) of the video decoded by the terminal UT is estimated. The subjective quality (subjective quality estimated value) estimated by the subjective quality estimating unit 73 is sent to the quality deterioration factor / quality estimated value storage unit 74.

  The quality deterioration factor / quality estimated value storage unit 74, when the subjective quality estimation value from the subjective quality estimation unit 73 and the quality deteriorate, the quality deterioration factor (for example, frame loss, loss pattern, intermediate parameter, etc.) Save. Also, the information is totaled and stored every certain time. For example, the estimated subjective quality estimation values are averaged or stored in time series.

[Quality information communication device]
The quality information communication device 8 uses the control packet for the subjective quality estimate and the quality degradation factor stored in the quality degradation factor / quality estimate storage unit 74 at a certain time interval (quality management cycle). And transmitted to the video quality management device 5 in the management terminal CT.

  In this embodiment, RTCP (RTP Control Protocol) is used for a control packet between the distribution terminal DT and the user terminal UT, a control packet between the user terminal UT and the management terminal CT, and the like. These control packets are transmitted and received based on RTP (Real-time Transport Protocol).

[Video quality control device]
As shown in FIG. 4, the video quality management apparatus 5 is provided with a video quality management target value table 51 and a video quality determination unit 52. When the control quality packet in which the subjective quality estimation value and the quality degradation factor are embedded is sent from the user terminal UT, the video quality determination unit 52 extracts the subjective quality estimation value and the quality degradation factor from the control packet. To do. Then, a video quality management target value corresponding to the extracted subjective quality estimated value is extracted from the video quality management target value table 51, and the video quality management target value table 51 is compared to determine the deterioration of the video quality. Take action.

  In this embodiment, for each video packet to be sent to the user terminal UT, the frame type and the frame generation rule at the time of high-efficiency compression encoding to which the video packet belongs are embedded, but for each video packet to be sent to the user terminal UT. A table (packet) in which the generation number of the video packet, the frame type (I, P, B) and the frame generation rule (GOP parameters (M, N)) to which the video packet P belongs are recorded. The information table may be embedded in the control packet and sent to the user terminal UT via the network 4. In this case, the user terminal UT refers to a table embedded in a control packet sent via the network 4 so that a high-efficiency compression coding to which the arrived packet and the packet missing on the network / terminal belong. It is possible to know the frame type and frame generation rule at the time.

[Quality estimation model in the subjective quality estimation section]
In the present embodiment, the quality estimation model in the subjective quality estimation unit 73 is created by modeling the relationship between intermediate parameters and MOS (video quality experienced by the user) using a regression curve. In Reference 1 previously proposed by the present applicant, it is difficult to estimate the quality based on the relationship between the packet loss rate and the MOS (see FIG. 5), and the total calculated based on the deterioration duration and the deterioration amount is used. The relationship between the number of deteriorated frames and the MOS (see FIG. 6) was used as a quality estimation model.

  In the present embodiment, in order to improve the quality estimation accuracy, an intermediate parameter necessary for video quality estimation is determined according to the situation, and the intermediate parameter is weighted according to the degradation type of the video signal. The quality estimation model weighted according to the degradation type of the video signal is used, or the quality estimation model according to the degradation type of the video signal is used.

  Hereinafter, an example of subjective quality estimation using some intermediate parameters will be described in detail. Note that this example is not related to each other, and is an independent estimation method, which is used separately in consideration of operation and estimation accuracy, or used together.

[Estimation Example 1: An example in which the number of deteriorated frames including information on the presence / absence of a frame that has been determined to be redundantly deteriorated is an intermediate parameter]
For example, it is assumed that the loss pattern determination unit 713 determines “the number of deteriorated frames including information regarding the presence / absence of redundantly determined frames (hereinafter referred to as overlapping deteriorated frames) in the same GOP” as an intermediate parameter. .

  Note that a duplicated deteriorated frame is a frame that has been judged to have deteriorated in duplicate, for example, when the frame itself is determined to be deteriorated, or when it is determined to have been deteriorated due to the influence of another deteriorated frame. Say.

  In the subjective quality estimation unit 73, a quality estimation model (see FIG. 7) indicating the relationship between the number of deteriorated frames and the MOS is determined and set in advance. This quality estimation model is defined by formulating the relationship between the number of deteriorated frames in the same GOP and the MOS by a regression equation on the condition that there are no overlapping deteriorated frames.

  In FIG. 7, the regression curve is obtained by exponential approximation. However, regarding the approximation, any probability model may be used as long as the correlation between the number of deteriorated frames and the MOS is maximized. Further, these relationships are not only modeled as mathematical formulas, but the relationship between the number of deteriorated frames and the MOS may be organized and recorded as a table and used as a quality estimation model.

  The subjective quality estimation unit 73 receives the number of deteriorated frames including information on the presence / absence of duplicate deteriorated frames in the same GOP sent from the loss pattern determination means 713, and when there is a duplicate deteriorated frame, the input deteriorated frame The number is weighted according to the deterioration type from the deterioration type determining means 722, and the video from the quality estimation model (FIG. 7) showing the relationship between the number of deteriorated frames set in advance and the MOS from the weighted number of deteriorated frames. Quality is obtained, and this video quality is set as a quality estimate value (subjective quality estimate value) of the video decoded by the user terminal UT.

  In this example, the weight for the number of deteriorated frames is determined as follows. The relationship between the number of deteriorated frames and the MOS under the condition that there is no overlapping deteriorated frame in the same GOP is modeled in advance by a regression equation or the like (see FIG. 8). The amount of change in the number of deteriorated frames with respect to the amount is calculated, and the value is used as a weight.

  For example, in the case of FIG. 8, it is assumed that MOS = 3 in the “condition that there is no overlapping deteriorated frame in the same GOP”, and in the case of “condition that there is an overlapping deteriorated frame in the same GOP”, MOS = 2.5. Assuming that there are duplicate deteriorated frames in the same GOP, the value is the same as when the number of deteriorated frames is increased by 15, and this value is used as a weight. Thus, the inverse function of the regression curve shown in FIG. 8 may be used, or the value calculated here (weight = 15) may be used as a fixed value. In this example, the value calculated here is used as a fixed value and set in the subjective quality estimation unit 73. For this “weight = 15”, a weighting coefficient (or function) corresponding to the degradation type is created in advance and set in the subjective quality estimation unit 73.

  For example, when “the number of deteriorated frames = 20” is transmitted from the loss pattern determination unit 713 as “with duplicate deteriorated frames”, the subjective quality estimation unit 73 follows the weighting condition shown below and “the number of deteriorated frames = 20”. Is added with “weight = 15”. Then, this “weight = 15” plus “degraded frame number = 20 + 15 = 35” is multiplied by a weighting factor (or function) corresponding to the degradation type to obtain a weighted degraded frame number. From this weighted degraded frame number A subjective quality estimation value is obtained from a quality estimation model (FIG. 7) showing the relationship between the number of deteriorated frames set in advance and the MOS.

Weighting condition “IF: There is a degraded frame in the same GOP, THEN: Weight of 15 frames determined from the amount of MOS reduction”

  Originally, when a frame (for example, P frame) affecting a plurality of frames is lost (for example, when a packet is lost on the network), the affected frame (for example, B frame) has already deteriorated. However, in addition to this, when the affected frame is lost (for example, when a packet is lost on the network), the loss as the video has already been determined to be a loss rate. As it was not going to change. However, here, in the case of such a phenomenon, it is not necessarily said that the loss of the affected frame does not affect the deterioration of the video, and does affect the deterioration of the video. It is not a simple addition / subtraction, and is expressed by weights.

  In this example 1, the number of deteriorated frames (intermediate parameter) is weighted according to the deterioration type. However, the weight according to the deterioration type is applied to the quality estimation model set in advance. Alternatively, a quality estimation model individually determined according to the degradation type may be created and set in advance. Although each method is different, in any case, it can be said that the video quality is obtained from “the relationship between the intermediate parameter corresponding to the degradation type of the video signal and the MOS”.

[Estimation Example 2: Example in which the number of deteriorated frames including information on the deterioration interval of frames within the same GOP is used as an intermediate parameter]
The quality experienced by the user gives different impressions depending on how the frame deterioration occurs even if the number of deteriorated frames and the ratio thereof are the same. Many of the factors that give different impressions are related to whether or not the user can perceive the deterioration phenomenon of the video. For example, when frames are continuously deteriorated at one time (in a specific example, when an image having 1 to 300 frames is evaluated, frames 20 to 25 are deteriorated (deteriorated frame duration = 6 )), The user feels that the video has deteriorated for a long time (deterioration duration), but perceives the video deterioration phenomenon only once.

  On the other hand, a deterioration phenomenon that can be perceived by the user (here, even if one frame is deteriorated, it is assumed that the deterioration of the image can be perceived) occurs in several times (in a specific example, 20 or When the frames No. 30, 50, 80, 155, and 250 are deteriorated), the user does not feel that the deterioration continuation time is long, but the deterioration phenomenon is repeated multiple times (in this example, 6 times) during one evaluation. Perceive that occurred. In such a case, the latter phenomenon has a greater impression of the deterioration phenomenon than the former. Therefore, in Example 2, weighting is performed based on the frame degradation interval.

  For example, it is assumed that the loss pattern determination unit 713 determines “the number of deteriorated frames including information related to the deterioration interval of frames within the same GOP” as an intermediate parameter. In this example, it is assumed that the frame degradation interval and the duration of the degraded frame are sent as information regarding the frame degradation interval.

  In the subjective quality estimation unit 73, a quality estimation model (see FIG. 9) indicating the relationship between the number of deteriorated frames and the MOS is determined and set in advance. This quality estimation model is defined by formulating the relationship between the number of deteriorated frames in the same GOP and the MOS by a regression equation on the condition that no frame deterioration interval exists in the same GOP.

  In FIG. 9, the regression curve is obtained by exponential approximation. However, regarding the approximation, any probability model may be used as long as the correlation between the number of deteriorated frames and the MOS is maximized. Further, these relationships are not only modeled as mathematical formulas, but the relationship between the number of deteriorated frames and the MOS may be organized and recorded as a table and used as a quality estimation model.

  The subjective quality estimation unit 73 receives the number of deteriorated frames including information on the deterioration interval of frames in the same GOP sent from the loss pattern determination unit 713, and the deterioration interval of frames that fall within a predetermined range within the same GOP. Is present, the number of input deteriorated frames is weighted according to the deterioration type from the deterioration type determining means 722, and the relationship between the number of deteriorated frames set beforehand and the MOS is determined from the weighted deteriorated frame number. The video quality is obtained from the quality estimation model (FIG. 9) shown, and this video quality is used as the quality estimation value (subjective quality estimation value) of the video decoded by the user terminal UT.

  In this example, the weight for the number of deteriorated frames is determined as follows. The relationship between the number of deteriorated frames and the MOS under the condition that no frame deterioration interval exists in the same GOP is modeled in advance by a regression equation or the like (see FIG. 8), and from this regression equation, the MOS when the deterioration interval exists The amount of change in the number of deteriorated frames with respect to the amount of decrease is calculated, and the value is used as a weight.

  For example, in the case of FIG. 8, it is assumed that MOS = 3 in “the condition that there is no frame degradation interval in the same GOP”, and in the case of “the frame degradation interval exists in the same GOP”, MOS = 2. If it is 7, since there is a frame degradation interval in the same GOP, the value is the same as when the number of degraded frames is increased by 10, so this value is used as a weight. In this way, the inverse function of the regression curve shown in FIG. 8 may be used, or the value calculated here (weight = 10) may be used as a fixed value. In this example, the value calculated here is used as a fixed value and set in the subjective quality estimation unit 73. For this “weight = 10”, a weighting coefficient (or function) corresponding to the degradation type is created in advance and set in the subjective quality estimation unit 73.

  For example, the subjective quality estimation unit 73 receives “frame deterioration interval A (β ≦ A <γ)” and “frame deterioration duration B (α ≦ B)” as information on the frame deterioration interval from the loss pattern determination unit 713. When “degraded frame number = 20” is sent, “weight = 10” is added to “degraded frame number = 20” in accordance with the following weighting condition. Then, the number of deteriorated frames = 20 + 10 = 30 to which the weight “10” is added is multiplied by a weight coefficient (or function) corresponding to the deterioration type to obtain the number of weighted deteriorated frames. A subjective quality estimation value is obtained from a quality estimation model (FIG. 9) showing the relationship between the number of deteriorated frames set to 1 and the MOS.

Weighting condition “IF: Deteriorated frame duration is α or more, and frame deterioration interval is β or more and γ or less, THEN: Weight of 10 frames determined from the amount of decrease in MOS is applied”
α or more: Degraded frame duration perceivable by the user.
β or less: The number of frames that can be perceived when the degraded frames are not continuously generated.
γ or more: The number of frames perceived as completely different from the deterioration phenomenon (the interval at which the impression of the previous deterioration phenomenon is forgotten).

  FIG. 10 shows a weight determination flowchart in the subjective quality estimation unit 73 according to this weighting condition. When the number of deteriorated frames including information (deterioration interval of frames, duration of deteriorated frames) regarding the deterioration interval of frames in the same GOP is sent from the loss pattern determination unit 713, the subjective quality estimation unit 73 receives the result shown in FIG. According to the flowchart shown, frames of α or more are continuously deteriorated (YES in step 101), there is a frame interval of β or more from the previous deteriorated frame (YES in step 102), and γ or less from the previous deteriorated frame If there is a frame interval (YES in step 103), the weight of the X frame determined from the relationship of the amount of decrease in MOS is added to the number of deteriorated frames (step 104), and the weight is deteriorated to the number of deteriorated frames. Multiply by a weighting factor corresponding to the type (step 105). If all of steps 101, 102, and 103 are NO, weight is not given and normal frame deterioration is subjectively determined from a quality estimation model (FIG. 9) showing the relationship between the number of deteriorated frames set in advance and MOS. A quality estimate is obtained (step 106).

  In Example 2, the number of deteriorated frames (intermediate parameter) is weighted according to the deterioration type, but the weight according to the deterioration type is applied to the quality estimation model set in advance. Alternatively, a quality estimation model may be individually created according to the deterioration type and set in advance. Although each method is different, in any case, it can be said that the video quality is obtained from “the relationship between the intermediate parameter corresponding to the degradation type of the video signal and the MOS”.

[Estimation Example 3: Utilization of Media Features]
In the estimation examples 1 and 2, the video quality information measurement unit 72 is provided in the video quality estimation device 7 and the degradation type determined by the video quality information measurement unit 72 is sent to the subjective quality estimation unit 73. A configuration in which the measurement unit 72 is omitted is also possible. That is, the subjective quality estimation unit 73 may be configured to estimate the subjective quality using only the intermediate parameters from the network / terminal quality information analysis unit 71. In this case, a quality estimation model is set in the subjective quality estimation unit 73 by taking the following steps, for example.

[Step 1]
The relationship between intermediate parameters and media features is modeled by a regression equation. For example, TS packet quality information (for example, packet loss rate or loss pattern (for example, weighted loss amount considering loss interval)) is applied as an intermediate parameter, and the relationship between the intermediate parameter and the media feature amount Is formulated using a regression equation. As the media feature amount, the degradation area of the macroblock (the ratio of the degradation area to the entire macroblock area may be used) is used. FIG. 11 shows the relationship between the intermediate parameter and the media feature amount. Then, the two relationships are modeled by formulating them using a regression curve. For example:
Media feature amount (deterioration area of macroblock) = g (intermediate parameter) = g (TS packet loss rate) (1)

[Step 2]
The relationship between the media feature and MOS is modeled by a regression equation. FIG. 11 shows an example in which the degradation area of the macroblock (or the ratio of the degradation area to the entire macroblock area) may be used as the media feature amount. The two relationships are modeled by formulating them using a regression curve. For example:
MOS (user experience quality) = f (media feature amount) = f (macroblock area) (2)

[Step 3]
Using the first relational model modeled by equation (1) and the second relational model modeled by equation (2), the video quality is determined from the intermediate parameters obtained by the network / terminal quality measurement. A model to be estimated (a quality estimation model indicating the relationship between intermediate parameters and MOS) is created. For example:
MOS (user experience quality) = f (g (intermediate parameter)) = f (g (TS packet loss rate)) (3)

  Note that a TS packet is a packet used when media information such as video and audio is communicated using an IP packet or the like, and the media information is divided and transmitted in units of TS packets. There are many applications that incorporate a plurality of TS packets into an IP packet and realize video / audio communication using IP.

  In FIGS. 11 and 12, the regression curve is obtained by exponential approximation. However, regarding the approximation, any probability model may be used as long as the correlation between the two parameters is maximized. Further, these relationships are not only modeled as mathematical formulas, but the relationship between two parameters may be organized and recorded as a table and used as a quality estimation model.

  A quality estimation model indicating the relationship between the intermediate parameter and the MOS is set in the subjective quality estimation unit 73 in advance. Thus, when an intermediate parameter is sent from the network / terminal quality information analysis unit 71, the subjective quality estimation unit 73 immediately obtains a subjective quality estimation value from the quality estimation model corresponding to the intermediate parameter. In this case, the media feature amount is substantially estimated from the intermediate parameters from the network / terminal quality information analysis unit 71, and a subjective quality estimation value corresponding to the estimated media feature amount is obtained. The video quality is considered in consideration.

  In this example 3, the quality estimation indicating the relationship between the intermediate parameter and the MOS using the first model modeled by the equation (1) and the second relation model modeled by the equation (2). A model is created and the subjective quality estimate is obtained from this quality estimation model. The first relation model and the second relation model are set in advance in the subjective quality estimation unit 73, and the first relation model is obtained. An intermediate parameter may be input to the model to obtain a media feature amount, and a media feature amount obtained from the first relation model may be inputted to a second relation model to obtain a subjective quality estimate.

[Embodiment 2]
FIG. 13 is a block diagram showing another example (Embodiment 2) of a video distribution system including a video quality estimation apparatus according to the present invention. In the first embodiment, the video quality estimation device 7 is provided in the user terminal UT. However, in the second embodiment, the video quality estimation device 7 ′ is provided in the management terminal CT.

  In this case, the quality information communication device 8 ′ of the user terminal UT determines the degradation type of the video signal, the degradation type of the video signal, the information of the received video packet, and the transmission quality from the video distribution server 3. Information or the like is sent to the video quality estimation apparatus 7 ′ on the management terminal CT side by a control packet.

  In this way, the processing burden on the user terminal UT can be reduced. Also, by providing the video quality estimation device 7 'in the management terminal CT, the subjective quality estimation result can be immediately used for network quality management.

[Embodiment 3]
FIG. 14 is a block diagram showing another example (Embodiment 3) of a video distribution system including a video quality estimation apparatus according to the present invention. In the first embodiment, the video quality estimation device 7 is provided in the user terminal UT and the video quality management device 5 is provided in the management terminal CT. However, in the third embodiment, the video quality estimation device 7 ′ and the video quality management device 5 are provided. It is provided in the distribution terminal DT. This configuration example is useful for individual user management in the server.

[Embodiment 4]
FIG. 15 is a block diagram showing another example (Embodiment 4) of a video distribution system including a video quality estimation apparatus according to the present invention. The fourth embodiment is a configuration example in the case of performing quality management of a bidirectional video communication service such as a video conference, and user terminals UT1 and UT2 are connected to the network 4.

  The user terminal UT1 includes a decoding device 6-1, a video quality estimation device 7-1, a quality information communication device 8-1, a video encoding device 9-1, and a transmission quality information acquisition / transmission device 10-1. have. The user terminal UT2 includes a decoding device 6-2, a video quality estimation device 7-2, a quality information communication device 8-2, a video encoding device 9-2, and a transmission quality information acquisition / transmission device 10-2. have.

  The video quality estimation apparatuses 7-1 and 7-2 have the same configuration as the video quality estimation apparatus 7 of the first embodiment shown in FIG. The quality information communication device 8-2 obtains the subjective quality estimate value in the user terminal UT2, and sends the obtained subjective quality estimate value to the counterpart user terminal UT1.

  Video encoding devices 9-1 and 9-2 correspond to video encoding device 2 in the first embodiment, convert a video signal into a packet sequence, and transmit the packet sequence to user terminals UT2 and UT1 on the other side. The transmission quality information acquisition / transmission apparatuses 10-1 and 10-2 describe information relating to packets of video signals transmitted from the video encoding apparatuses 9-1 and 9-2 (for example, described in the number of transmission packets and packet headers). Sequence number) is acquired as transmission quality information, and this transmission quality information is sent to the user terminals UT2 and UT1 by a control packet.

  In Embodiments 1 to 4 described above, the video quality management device 5 is provided in the management terminal CT and the distribution terminal DT, but may be incorporated in the user terminal UT. If the video quality management device 5 is incorporated into the user terminal UT, it is possible to quickly cope with the quality grasp of the user terminal at the time of complaint.

It is a block block diagram which shows an example (Embodiment 1) of the video delivery system containing the video quality estimation apparatus which concerns on this invention. It is a figure which shows schematic structure of the video coding apparatus in this video quality estimation method. It is a figure which shows schematic structure of the video quality estimation apparatus in this video quality estimation method. It is a figure which shows schematic structure of the video quality management apparatus in this video quality estimation method. It is a figure which shows the relationship between a packet loss rate and MOS. It is a figure which shows the relationship between the total number of degradation frames calculated based on degradation continuation length and degradation amount, and MOS. It is a figure which shows the relationship between the number of degradation frames beforehand set to a subjective quality estimation part and MOS in the example 1 of subjective quality estimation. It is a figure explaining the fall amount of MOS. It is a figure which shows the relationship between the deterioration frame number preset to a subjective quality estimation part and MOS in the example 2 of subjective quality estimation. It is a flowchart which shows the process of the weight determination in the subjective quality estimation part in the estimation example 2 of subjective quality. It is a figure which shows the relationship between an intermediate parameter and MOS at the time of using TS packet loss as an intermediate parameter in the estimation example 3 of subjective quality. It is a figure which shows the relationship between a media feature-value at the time of using the degradation area of a macroblock as a media feature-value in the example 3 of estimation of subjective quality, and MOS. It is a block block diagram which shows the other example (Embodiment 2) of the video delivery system containing the video quality estimation apparatus which concerns on this invention. It is a block block diagram which shows the other example (Embodiment 3) of the video delivery system containing the video quality estimation apparatus which concerns on this invention. It is a block block diagram which shows the other example (Embodiment 4) of the video delivery system containing the video quality estimation apparatus which concerns on this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Video input device, 2 ... Video encoding device, 21 ... High efficiency compression encoding part, 22 ... Packet conversion part, 23 ... Information embedding part, 3 ... Video distribution server, 4 ... Network (internet network), 5 ... Video quality management device, 51 ... Video quality management target value table, 52 ... Video quality determination unit, 6 ... Decoding device, 7 ... Video quality estimation device, 71 ... Network / terminal quality information analysis unit, 711 ... Packet information acquisition means 712 ... Video frame identification means, 713 ... Loss pattern judgment means, 72 ... Video quality information measurement section, 721 ... Feature quantity extraction means, 722 ... Degradation type judgment means, 73 ... Subjective quality estimation section, 74 ... Quality degradation factor Quality estimated value storage unit, UT ... user terminal, CT ... management terminal, DT ... distribution device, P (P1, P2, P3) ... video packet, UT1, UT2 ... user terminal, 6-1, -2 ... Decoding device, 7-1, 7-2 ... Video quality estimation device, 8-1, 8-2 ... Quality information communication device, 9-1, 9-2 ... Video encoding device, 10-1, 10 -2 ... Transmission quality information acquisition / transmission device.

Claims (8)

Decoding means for receiving and decoding a video signal transmitted after being converted into a packet sequence;
Intermediate parameter determining means for determining a required parameter as an intermediate parameter based on at least the packet loss information before decoding the video signal;
A degradation type determination means for analyzing the decoded video signal and determining a degradation type of the video signal;
Quality estimation means for estimating video quality after decoding of the video signal based on the intermediate parameter determined by the intermediate parameter determination means and the deterioration type determined by the deterioration type determination means Video quality estimation device. Decoding means for receiving and decoding a video signal transmitted after being converted into a packet sequence;
Intermediate parameter determining means for determining a required parameter as an intermediate parameter based on at least the packet loss information before decoding the video signal;
A degradation type determination means for analyzing the decoded video signal and determining a degradation type of the video signal;
When the number of deteriorated frames in the same group is determined as the intermediate parameter, and there are duplicated frames determined to be deteriorated in the same group, the deterioration type determined by the deterioration type determining means in the deteriorated frame number And a quality estimation means for obtaining a video quality corresponding to the weighted number of deteriorated frames from a relationship between a preset number of deteriorated frames and the video quality. Estimating device. Decoding means for receiving and decoding a video signal transmitted after being converted into a packet sequence;
Intermediate parameter determining means for determining a required parameter as an intermediate parameter based on at least the packet loss information before decoding the video signal;
A degradation type determination means for analyzing the decoded video signal and determining a degradation type of the video signal;
When the number of deteriorated frames in the same group is determined as the intermediate parameter, and there is a deterioration interval of frames that fall within a predetermined range in the same group, the deterioration type determined by the deterioration type determining means in the deteriorated frame number And a quality estimation means for obtaining a video quality corresponding to the weighted number of deteriorated frames from a relationship between a preset number of deteriorated frames and the video quality. Estimating device. Decoding means for receiving and decoding a video signal transmitted after being converted into a packet sequence;
Intermediate parameter determining means for determining a required parameter as an intermediate parameter based on at least the packet loss information before decoding the video signal;
Feature quantity deriving means for obtaining a quality degradation feature quantity according to the intermediate parameter determined by the intermediate parameter determination means from a relationship between the intermediate parameter set in advance and the quality degradation feature quantity of the video signal;
Quality estimation means for obtaining a video quality corresponding to the feature quantity of quality degradation obtained by the feature quantity deriving means from a relationship between the quality degradation feature quantity of the video signal set in advance and the video quality. A featured video quality estimation device. Decoding means for receiving and decoding a video signal transmitted after being converted into a packet sequence;
Intermediate parameter determining means for determining a required parameter as an intermediate parameter based on at least the packet loss information before decoding the video signal;
A first relation model indicating a relationship between the intermediate parameter set in advance and a feature amount of quality degradation of the video signal, and a second relationship indicating a relationship between the feature amount of quality degradation of the video signal and video quality A quality estimation unit for obtaining video quality corresponding to the intermediate parameter determined by the intermediate parameter determination unit from a quality estimation model indicating a relationship between the intermediate parameter determined from the relational model and the video quality. Video quality estimation device. A first step of receiving and decoding a video signal converted into a packet sequence and sent;
A second step of determining a required parameter as an intermediate parameter based on at least the packet loss information before decoding the video signal;
A third step of analyzing the decoded video signal and determining a degradation type of the video signal;
A video quality estimation method comprising: a fourth step of estimating the video quality after decoding of the video signal based on the intermediate parameter determined in the second step and the degradation type determined in the third step . A first step of receiving and decoding a video signal converted into a packet sequence and sent;
A second step of determining a required parameter as an intermediate parameter based on at least the packet loss information before decoding the video signal;
A third step of analyzing the decoded video signal and determining a degradation type of the video signal;
And a fourth step of obtaining a video quality corresponding to the intermediate parameter determined in the second step from a relationship between the intermediate parameter corresponding to the deterioration type determined in the third step and the video quality. Method. A first step of receiving and decoding a video signal converted into a packet sequence;
A second step of determining a required parameter as an intermediate parameter based on at least the packet loss information before decoding the video signal;
And a third step of estimating a feature amount of quality degradation of the video signal from the intermediate parameter determined in the second step and estimating a video quality after decoding of the video signal from the estimated feature amount. A video quality estimation method.

Images