JP2018074469A - Psychological factor score calculation device, psychological factor score calculation method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To enable feeling received from a video image to be estimated.SOLUTION: A psychological factor score calculation device includes a storage unit that stores a coefficient of an equation indicating a relationship between a video image parameter and a quality design parameter, and a psychological factor score that is an evaluation value of feeling received from a video image for a psychological factor for each video image parameter and each psychological factor and a calculation unit that calculates a psychological factor score by applying, to the equation, a coefficient corresponding to an input video image parameter and an input psychological factor from among coefficients stored in the storage unit and the input video image parameter and an input quality design parameter.SELECTED DRAWING: Figure 4

Description

  The present invention relates to a psychological factor score calculating apparatus, a psychological factor score calculating method, and a program.

  In order to provide a video service with good quality to the user, it is important to design the system so that the video quality experienced by the user is good before providing the service. In order to implement this, a video quality quantification method capable of appropriately evaluating the quality of the video experienced by the user is necessary.

  Methods for quantifying video quality include a subjective quality evaluation method (for example, see Non-Patent Document 1) and a quality estimation technique (for example, see Non-Patent Documents 2 and 3). The subjective quality evaluation method requires a large number of users to evaluate video images in a facility where the evaluation environment (room illuminance, room noise, etc.) can be reproduced. Therefore, evaluation takes time. Therefore, development of a quality estimation technique for outputting a video quality value from a media signal or a physical feature amount related to service or communication is being developed (for example, see Non-Patent Documents 2, 3, and 4).

  Video quality estimation techniques are classified based on input information. Depending on the amount of input information, the amount of calculation and the accuracy of quality estimation differ, so it is necessary to use them properly depending on the application. For example, when a video service designer uses a quality estimation technique for quality design before service provision, a video quality estimation technique that receives network and terminal quality design parameters (for example, bit rate and packet loss rate) as input. Is used. Thereby, it is possible to grasp what quality design parameter value needs to be set in order to achieve the target video quality (Non-Patent Document 4).

ITU-T recommendation P.910 "Subjective video quality assessment methods for multimedia applications" ITU-T recommendation J.247 "Objective perceptual multimedia video quality measurement in the presence of a full reference" ITU-T Recommendation J.341 "Objective perceptual multimedia video quality measurement of HDTV for digital cable television in the presence of a full reference" "Quality design technology for video communication services", [online], Internet <http://www.ntt.co.jp/journal/0901/files/jn200901016.pdf> "Upcoming 8K broadcasting", [online], Internet <https://www.nhk.or.jp/digital/b_tech/pdf2015/nhk2015_02_07.pdf>

  In recent years, ultra-high definition video services such as 4K / 8K video have attracted attention. In the 4K / 8K video service, the resolution, the frame rate, and the bit depth are improved as compared with the conventional full HD video service (Non-Patent Document 5). Therefore, it is expected that natural video display becomes possible, and not only high video quality but also a richer sense (psychological factor) can be experienced.

  In order to provide a richer ultra-high-definition video service, the video service should be designed in consideration of not only video quality but also psychological factors (eg, aesthetic feeling, dynamic feeling, etc.) unique to ultra-high-definition video. is necessary. For example, if a travel program with a more beautiful video can be provided, the user can clearly have an image of the video space presented about the scenery or meal of the travel destination, and the degree of satisfaction with the user's video service (user Satisfaction) is improved. In addition, if a sports program that can feel the power and enthusiasm of the game can be provided, the user can feel the dynamism as if watching the game at the audience seat, and the user satisfaction is improved. . In order to provide travel programs with high aesthetics and sports programs with high dynamism, it is necessary to design each program so that the aesthetic sensation and dynamism are maintained to some extent.

  In order to design a video service based on psychological factors, video parameters (resolution, frame rate, bit depth), quality design parameters (encoding bit rate), and evaluation values of the user's perception of psychological factors from the video (psychological factors) It is necessary to understand the relationship with the score. Here, the case where the input video parameter is one and the case where it is two or more will be described separately.

  First, the case where one video parameter is input will be described. FIG. 1 is a diagram illustrating the relationship between the encoding bit rate and the psychological factor score for each psychological factor and each video parameter. Here, as an example, five psychological factors such as aesthetic feeling, dynamic feeling, natural feeling, open feeling, and three-dimensional feeling are shown.

  FIG. 1A shows, for each psychological factor, the relationship between the encoding bit rate and the psychological factor score when resolution is input as a video parameter. FIG. 1B shows the relationship between the coding bit rate and the psychological factor score when the frame rate is input for each psychological factor. FIG. 1C shows the relationship between the coding bit rate and the psychological factor score when the bit depth is input for each psychological factor. The relationship shown in FIG. 1 is based on the subjective quality experiment results.

  FIG. 1 shows that the relationship between the encoding bit rate and the psychological factor score is different for each psychological factor and each video parameter. However, the magnitude of the influence of the video parameter on the relationship between the encoding bit rate and the psychological factor score differs depending on the psychological factor.

  For example, when the resolution changes, the sharpness of the image changes. Since the sharpness of the image affects the aesthetic feeling and the natural feeling, as shown in FIG. 1A, the relationship between the coding bit rate and the psychological factor score depends on the resolution. It turns out that it is very different.

  Also, the smoothness of movement changes as the frame rate changes. Since smoothness of movement affects dynamic feeling and natural feeling, as shown in FIG. 1B, in dynamic feeling and natural feeling, the relationship between the coding bit rate and the psychological factor score depends on the frame rate. It can be seen that is significantly different.

  Further, when the bit depth changes, the standing condition at the edge of the subject changes. Since the degree of standing at the edge affects the sense of openness and stereoscopic effect, as shown in FIG. 1C, in the sense of openness and stereoscopic effect, the relationship between the coding bit rate and the psychological factor score depends on the bit depth. It can be seen that is significantly different.

  From the above, it can be seen that the relationship between the encoding bit rate and the psychological factor score is different for each psychological factor and each video parameter. Therefore, in order to change one video parameter and the encoding bit rate and improve a specific psychological factor score, a psychology in which the relationship between the video parameter and the encoding bit rate and the psychological factor score is formulated for each psychological factor. Factor score estimation techniques should be used.

  In FIGS. 1A to 1C, in a region where the encoding bit rate is high, if the video parameter is improved, each psychological factor score increases, but in a region where the encoding bit rate is low, this is not necessarily the case. Not. This is because, in an area where the encoding bit rate is low, when the comparison is made at the same bit rate, a video with improved video parameters has a higher compression rate and a large deterioration occurs.

  Next, a case where two or more video parameters are input will be described. FIG. 2 is a diagram showing the relationship between the coding bit rate and the psychological factor score when all three parameters are changed for each psychological factor. Here, 4K60P10-bit video and 2K30P8-bit video are illustrated. The relationship shown in FIG. 2 is based on the subjective quality experiment results.

  As shown in FIG. 2, considering the characteristics for each video parameter shown in FIG. 1, due to the synergistic effect of the video parameters, each psychological factor score is higher in the 4K60P10-bit video than in the 2K30P8-bit video in the region where the encoding bit rate is high. Becomes higher. In the region where the encoding bit rate is low, the psychological factor score is higher in the 2K30P8-bit video than in the 4K60P10-bit video. Therefore, when two or more video parameters are changed simultaneously, a technique for deriving each psychological factor score in consideration of a synergistic effect between the video parameters is necessary.

  The present invention has been made in view of the above points, and an object of the present invention is to make it possible to estimate a sense received from an image.

  Therefore, in order to solve the above problem, the psychological factor score calculating apparatus calculates the coefficient of the mathematical formula indicating the relationship between the video parameter and the quality design parameter and the psychological factor score which is an evaluation value of the sensation received from the video regarding the psychological factor. A storage unit for storing each and each psychological factor; among coefficients stored in the storage unit, input video parameters and coefficients corresponding to psychological factors; and input video parameters and quality design parameters And a calculation unit that calculates a psychological factor score.

  The sensation received from the video can be estimated.

It is a figure which illustrates the relationship between an encoding bit rate and a psychological factor score for every psychological factor and every video parameter. It is a figure which shows the relationship between an encoding bit rate when all three parameters are changed for every psychological factor, and a psychological factor score. It is a figure which shows the hardware structural example of the psychological factor score calculation apparatus 10 in embodiment of this invention. It is a figure which shows the function structural example of the psychological factor score calculation apparatus 10 in embodiment of this invention. It is a figure which shows the example of a relationship between the presence or absence of a sound, and a psychological factor score. It is a figure which shows the example of a relationship between a video parameter and coefficient (alpha) of the psychological factor score derivation function. It is a figure which shows the example of a relationship between an encoding bit rate and the coefficient (gamma) of a psychological factor score derivation function.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 3 is a diagram illustrating a hardware configuration example of the psychological factor score calculation apparatus 10 according to the embodiment of the present invention. The psychological factor score calculation device 10 in FIG. 3 includes a drive device 100, an auxiliary storage device 102, a memory device 103, a CPU 104, an interface device 105, a display device 106, an input device 107, and the like that are mutually connected by a bus B. Have.

  A program for realizing processing in the psychological factor score calculating apparatus 10 is provided by a recording medium 101 such as a CD-ROM. When the recording medium 101 storing the program is set in the drive device 100, the program is installed from the recording medium 101 to the auxiliary storage device 102 via the drive device 100. However, the program need not be installed from the recording medium 101 and may be downloaded from another computer via a network. The auxiliary storage device 102 stores the installed program and also stores necessary files and data.

  The memory device 103 reads the program from the auxiliary storage device 102 and stores it when there is an instruction to start the program. The CPU 104 realizes functions related to the psychological factor score calculation device 10 according to a program stored in the memory device 103. The interface device 105 is used as an interface for connecting to a network. The display device 106 displays a GUI (Graphical User Interface) or the like by a program. The input device 107 includes a keyboard and a mouse, and is used for inputting various operation instructions.

  FIG. 4 is a diagram illustrating a functional configuration example of the psychological factor score calculation apparatus 10 according to the embodiment of the present invention. In FIG. 4, the psychological factor score calculation apparatus 10 includes an input unit 11, a psychological factor score estimation unit 12, and an output unit 13. Each of these units is realized by processing that one or more programs installed in the psychological factor score calculating apparatus 10 cause the CPU 104 to execute. The psychological factor score calculation apparatus 10 also uses the coefficient DB 14. The coefficient DB 14 can be realized by using, for example, a storage device that can be connected to the auxiliary storage device 102 or the psychological factor score calculation device 10 via a network.

  The input unit 11 receives input of input information for deriving (calculating) the psychological factor score Sif from the user. The input information includes psychological factor i, resolution x1, frame rate x2, and bit depth x3, which are examples of video parameters, and encoding bit rate br, which is an example of quality design parameters. The video parameter refers to a parameter indicating video characteristics or attributes. The quality design parameter is a parameter that defines the target quality for video distribution. Other parameters such as a packet loss rate may be used as quality design parameters.

  Further, when it is desired to derive a psychological factor score in consideration of the presence or absence of sound in the video, the sound flag f = 1 is input. When it is desired to derive the psychological factor score without considering the presence or absence of sound, no sound flag is input.

  That is, in the present embodiment, when designing a video service, not only video but also the necessity of considering the influence of sound is taken into consideration.

  FIG. 5 is a diagram illustrating an example of the relationship between the presence or absence of sound and the psychological factor score. As shown in FIG. 5, the psychological factor score of any psychological factor is improved by adding sound to a video having the same resolution, frame rate, and bit depth. However, the magnitude of the effect of adding sound on the relationship between the coding bit rate and the psychological factor score varies depending on the psychological factor. For example, in the sense of liveliness, naturalness, and openness, adding the sound emphasizes the power and spread of the video, so the relationship between the coding bit rate and the psychological factor score is large depending on the presence or absence of sound. I can see that they are different. Therefore, it is considered that each psychological factor score can be derived more appropriately by taking into account the presence or absence of sound. The relationship shown in FIG. 5 is based on the subjective quality experiment results.

  Psychological factors refer to the feelings (for example, aesthetic feeling, dynamic feeling, etc.) that the viewer of the video receives from the video, and the psychological factors score refers to the feelings that the viewer of the video receives from the video regarding certain psychological factors. The evaluation value (estimated value).

  The psychological factor score estimation unit 12 derives (calculates) a psychological factor score Sif using a psychological factor score derivation function. Each coefficient of the psychological factor score derivation function is stored in the coefficient DB 14. That is, the psychological factor score estimation unit 12 applies the psychological factor i, the video parameters x1 to x3, the encoding bit rate br, and the acoustic flag f received by the input unit 11 to the psychological factor score derivation function. The score Sif is estimated (calculated).

  The output unit 13 outputs (for example, displays on the display device 106) the psychological factor score Sif estimated by the psychological factor score estimating unit 12.

  The psychological factor score Sif is a psychological factor score deriving function that takes into account the psychological factor score deriving function, each coefficient of the psychological factor score deriving function set based on the relationship between the video parameter and the psychological factor i, and the presence or absence of sound. Are derived on the basis of the input video parameter and the value of the encoding bit rate.

The psychological factor score derivation function is formulated based on the relationships shown in FIGS. The coefficient of the psychological factor score derivation function is determined for each video parameter, each psychological factor, and each acoustic flag.
Sif = γif (br) (αi1 (x1) × αi2 (x2) × αi3 (x3) × log ₁₀ (br) + βi) (1)
αi1 = ai1 × exp (bi1 × x1) + ci1 (2)
αi2 = ai2 × exp (bi2 × x2) + ci2 (3)
αi3 = ai3 × exp (bi3 × x3) + ci3 (4)
γif = 1 (when f = 0) (5)
γif = di × br + ei (when f = 1) (6)
In equation (1), psychological factor i = 1 is aesthetic feeling, i = 2 is dynamic feeling, i = 3 is natural feeling, i = 4 is open feeling, and i = 5 is stereoscopic feeling. The coefficient αi1 (i = 1 to 5) of the psychological factor score derivation function is a function of the resolution x1, the coefficient αi2 (i = 1 to 5) is a function of the frame rate x2, and the coefficient αi3 (i = 1 to 1). 5) is a function of bit depth x3.

  The coefficient αi1 is formulated based on the relationship shown in FIG. More specifically, since the video becomes clearer when the resolution is improved, it is formulated for each psychological factor as shown in Equation (2) in consideration that the aesthetic feeling and the natural feeling are more likely to be improved. Note that FIG. 6A can be derived based on subjective quality evaluation experiment data when only the resolution is changed under the condition where the frame rate and the bit depth are fixed.

  The coefficient αi2 is formulated based on the relationship shown in FIG. Specifically, since the movement becomes smoother when the frame rate is improved, it is formulated for each psychological factor as shown in Equation (3) in consideration of the fact that the dynamic feeling and the natural feeling are more likely to be improved. Note that FIG. 6B can be derived based on subjective quality evaluation experiment data when only the frame rate is changed under the condition where the resolution and the bit depth are fixed.

  The coefficient αi3 is formulated based on the relationship shown in FIG. Specifically, when the bit depth is improved, the edge of the subject stands out, so that the stereoscopic effect and the open feeling are more easily improved, and the psychological factors are formulated as shown in Equation (4). In addition, FIG.6 (c) can be derived | led-out based on the subjective quality evaluation experiment data at the time of changing only a bit depth in the condition which fixed the resolution and the frame rate.

  As described above, the expression (1) is an expression obtained by combining mathematical expressions indicating the relationship between the video parameter and the psychological factor score for each of the plurality of video parameters.

  The coefficients aij, bij, cij, and βi (i = 1 to 5, j = 1 to 3) of the psychological factor score derivation function of the expressions (1) to (4) are subjective qualities that do not add sound performed in advance. It may be determined based on the evaluation experiment result and stored in the coefficient DB 14.

  In the equation (1), the coefficient γif is a coefficient that takes into account the effect due to the addition of sound. When the acoustic flag f is not input, the effect due to the addition of sound is not considered, so f = 0 and γif = 1 (Equation (5)). When the acoustic flag (f = 1) is input, in order to consider the effect due to the addition of sound, for example, it is formulated as shown in Equation (6) based on the relationship shown in FIG. 7 may be specified based on a subjective quality evaluation experiment or the like.

  The coefficients di and ei (i = 1 to 5, j = 1 to 3) may be determined based on the subjective quality evaluation experiment result to which the sound performed in advance is added and stored in the coefficient DB 14.

In Equation (1), the relationship between each video parameter and the psychological factor score is formulated by a nonlinear function based on FIG. 1, but it may be expressed by a linear function. Further, in order to take into account the synergistic effect between the video parameters, the coefficient αij for each video parameter is multiplied, but it is also conceivable to add as in equation (7).
Sif = γif ((αi1 (x1) + αi2 (x2) + αi3 (x3)) × log ₁₀ (br) + βi) (7)
In addition, although the expressions (2) to (4) formulate the relationship between each image parameter and the coefficient α of the psychological factor score derivation function as a non-linear function based on FIG. 6, it may be expressed as a linear function. It is done.

  Further, in the equation (6), the relationship between the coding bit rate and the coefficient γ of the psychological factor score derivation function is formulated by a linear function based on FIG. 7, but it may be expressed by a nonlinear function.

  Since the service designer basically knows all the values of resolution x1, frame rate x2, bit depth x3, encoding bit rate br, and presence / absence of sound, all of them are (1) to ( 6) Can be used as an input for equation. Therefore, the psychological factor score estimation unit 12 can derive a psychological factor score based on the equations (1) to (6).

However, for example, if the correspondence between the resolution and the psychological factor score is obtained in service design, but the correspondence between the bit depth, the frame rate, and the psychological factor is unnecessary or the correspondence is not obtained, the psychological factor score It is conceivable to simplify the derivation function as shown in equations (8) and (9). Formula (8) is formulated based on the relationship shown in FIG. 1, Formula (9) is formulated based on the relationship shown in FIG. 6, and Formula (11) is formulated based on the relationship shown in FIG.
Sif = γijf (αij (xj) × log ₁₀ (br) + βij) (8)
αij = aij × exp (bij × xj) + cij (9)
γijf = 1 (when f = 0) (10)
γijf = dij × br + eij (when f = 1) (11)
Equation (8) can be derived by simple regression analysis based on the subjective quality evaluation experiment results. That is, the coefficients aij, bij, cij, βij, dij, eij (i = 1 to 5, j = 1 to 3) of the psychological factor score derivation function are determined based on the subjective quality evaluation experiment results performed in advance, What is necessary is just to memorize | store in coefficient DB14.

  As described above, according to the present embodiment, for each psychological factor, by using a psychological factor score derivation function that formulates the relationship between the encoding bit rate of the video parameter and the quality design parameter and the psychological factor score, A psychological factor score can be derived. That is, it is possible to estimate the feeling received from the video.

  The psychological factor score derivation function also considers the presence or absence of sound. Therefore, the psychological factor score can be estimated in consideration of the presence or absence of sound.

  As a result of the above, for example, the video service designer considers the influence of sound and changes which parameters of the video parameters such as resolution, frame rate, and bit depth and the coding bit rate of the quality design parameters to change the target. You will be able to figure out if you can achieve the psychological factor score. As a result, video services with higher user satisfaction can be provided.

  In the present embodiment, the coefficient DB 14 is an example of a storage unit. The psychological factor score estimation unit 12 is an example of a calculation unit.

  As mentioned above, although the Example of this invention was explained in full detail, this invention is not limited to such specific embodiment, In the range of the summary of this invention described in the claim, various deformation | transformation・ Change is possible.

DESCRIPTION OF SYMBOLS 10 Psychological factor score calculation apparatus 11 Input part 12 Psychological factor score estimation part 13 Output part 14 Coefficient DB
100 drive device 101 recording medium 102 auxiliary storage device 103 memory device 104 CPU
105 interface device 106 display device 107 input device B bus

Claims (6)

A storage unit that stores a coefficient of a mathematical formula indicating a relationship between a video parameter and a quality design parameter and a psychological factor score that is an evaluation value of a sensation received from the video regarding a psychological factor for each video parameter and for each psychological factor,
Of the coefficients stored in the storage unit, a calculation unit that calculates a psychological factor score by applying a coefficient corresponding to the input video parameter and psychological factor, and the input video parameter and quality design parameter to the mathematical expression When,
A psychological factor score calculating apparatus characterized by comprising: The mathematical formula indicates a relationship between a video parameter and a quality design parameter based on a subjective quality experiment result, and a psychological factor score which is an evaluation value of a sensation received from the video regarding the psychological factor.
The psychological factor score calculation apparatus according to claim 1. The mathematical formula further indicates the relationship between the presence or absence of sound in the video and the psychological factor score,
The calculation unit calculates the psychological factor score by applying the presence or absence of input sound to the mathematical formula,
The psychological factor score calculation apparatus according to claim 1. The mathematical formula is a mathematical formula obtained by combining mathematical formulas showing the relationship between the video parameter and the psychological factor score for each of a plurality of video parameters.
The psychological factor score calculation apparatus according to any one of claims 1 to 3, wherein The coefficient of the mathematical formula indicating the relationship between the video parameter and the quality design parameter and the psychological factor score that is the evaluation value of the sensation received from the video with respect to the psychological factor is stored in a storage unit that stores the video parameter and the psychological factor. Of the coefficients, a calculation procedure for calculating a psychological factor score by applying a coefficient corresponding to the input video parameter and psychological factor, and the input video parameter and quality design parameter to the mathematical formula,
A psychological factor score calculation method characterized in that the computer executes   A program that causes a computer to function as each unit according to any one of claims 1 to 4.

Images