JP2018196099A - Psychological factor score estimation value calculation device, psychological factor score estimation value calculation method, and program - Google Patents

Abstract

To provide a psychological factor score estimation value calculation device capable of estimating a psychological factor score on the basis of an image parameter, a quality design parameter and a synchronization degree of image and sound.SOLUTION: The psychological factor score estimation value calculation device comprises: input means configured to input a psychological factor, an image parameter, a quality design parameter, an image signal and a sound signal; storage means configured to hold coefficients of a formula for calculating psychological factor score estimation value for each of multiple psychological factors; image-voice synchronization degree calculation means configured to calculate a synchronization degree of image and sound from the input image signal and sound signal; calculation means configured to calculate a psychological factor score estimation value by reading the input psychological factor and a coefficient corresponding to the image parameter from the storage means and inputting into formula, to which the coefficient is applied, the input image parameter, the quality design parameter, and the calculated synchronization degree of image and sound; and output means configured to output the psychological factor score estimation value.SELECTED DRAWING: Figure 4

Description

  The present invention relates to a technique for estimating a psychological factor score in a video service such as a video distribution service and a videophone service.

  In order to provide a video service with good quality to the user, it is important to design the system so that the video quality experienced by the user is good before providing the service. In order to implement these, a video quality quantification method capable of appropriately evaluating the quality of the video experienced by the user is necessary.

  Methods for quantifying video quality include a subjective quality evaluation method (for example, see Non-Patent Document 1) and a quality estimation technique (for example, see Non-Patent Documents 2 and 3). The subjective quality evaluation method requires a large number of users to evaluate video images in a facility where the evaluation environment (room illuminance, room noise, etc.) can be reproduced. Therefore, evaluation takes time. Therefore, development of a quality estimation technique for outputting a video quality value from a media signal or a physical feature amount related to service or communication is being developed (for example, see Non-Patent Documents 2, 3, and 4).

  Video quality estimation techniques are classified based on input information. Depending on the amount of input information, the amount of calculation and the quality estimation accuracy differ, so it is necessary to use them according to the purpose of use. For example, when a video service designer uses a quality estimation technique for quality design before service provision, a video quality estimation technique that receives network and terminal quality design parameters (for example, bit rate and packet loss rate) as input. Is used. Thereby, it is possible to grasp what quality design parameter value needs to be set in order to achieve the target video quality (Non-Patent Document 4).

ITU-T recommendation P.910 "Subjective video quality assessment methods for multimedia applications" ITU-T recommendation J.247 "Objective perceptual multimedia video quality measurement in the presence of a full reference" ITU-T Recommendation J.341 "Objective perceptual multimedia video quality measurement of HDTV for digital cable television in the presence of a full reference" Quality design technology for video communication services (http://www.ntt.co.jp/journal/0901/files/jn200901016.pdf) Upcoming 8K broadcast (https://www.nhk.or.jp/digital/b_tech/pdf2015/nhk2015_02_07.pdf)

  In recent years, ultra-high definition video services such as 4K / 8K video have attracted attention. In the 4K / 8K video service, the resolution, the frame rate, and the bit depth are improved compared to the conventional full HD video service (Non-Patent Document 5). Therefore, it is expected that natural video display becomes possible, and not only high video quality but also a richer sense (psychological factor) can be experienced.

  In order to provide a richer ultra-high-definition video service, the video service should be designed in consideration of not only video quality but also psychological factors (eg, aesthetic feeling, dynamic feeling, etc.) unique to ultra-high-definition video. is necessary. For example, if a travel program with a more beautiful video can be provided, the user can clearly have an image of the presented video space such as the scenery and meal of the travel destination, and the user's satisfaction with the video service (user satisfaction) Degree). Also, if a sports program that can feel the power of the game and the enthusiasm can be provided, the user can feel a sense of dynamism as if he / she was watching in the auditorium, and the user satisfaction is improved. In order to provide travel programs with high aesthetics and sports programs with high dynamism, it is necessary to design each program so that the aesthetic sensation and dynamism are maintained to some extent.

  In order to design a video service based on psychological factors, video parameters (resolution, frame rate, bit depth), quality design parameters (encoding bit rate), and sensory values (psychological factor score) that users receive from psychological factors It is necessary to grasp the relationship. Here, a description will be given separately when the input video parameter is one and when it is two or more.

  First, a case where one video parameter is input will be described. FIG. 1 illustrates the relationship between the encoding bit rate and the psychological factor score for each psychological factor and each video parameter. Here, as an example, five psychological factors such as aesthetic feeling, dynamic feeling, natural feeling, open feeling, and three-dimensional feeling are shown. FIG. 1A shows the relationship between the encoding bit rate when the resolution is input as the video parameter and the psychological factor score, and FIG. 1B shows the encoding bit rate when the frame rate is input. FIG. 1C shows the relationship between the coding bit rate when the bit depth is input and the psychological factor score for each psychological factor. FIG. 1 shows that the relationship between the encoding bit rate and the psychological factor score is different for each psychological factor and each video parameter.

  However, the magnitude of the influence of video parameters on the relationship between the encoding bit rate and the psychological factor score differs depending on the psychological factor. For example, when the resolution changes, the sharpness of the image changes. Since the sharpness of the image affects the aesthetic feeling and the natural feeling, as shown in FIG. 1A, the relationship between the coding bit rate and the psychological factor score varies greatly depending on the resolution in the aesthetic feeling and the natural feeling. I understand that. When the frame rate changes, the smoothness of movement changes. Since smoothness of movement affects dynamic feeling and natural feeling, as shown in FIG. 1B, in relation to dynamic feeling and natural feeling, the relationship between the encoding bit rate and the psychological factor score greatly depends on the frame rate. I can see that they are different. When the bit depth changes, the standing condition changes at the edge of the subject. Since the standing condition of the edge affects the feeling of openness and stereoscopic effect, as shown in FIG. 1C, in the sense of openness and stereoscopic effect, the relationship between the coding bit rate and the psychological factor score greatly depends on the bit depth. I can see that they are different. From the above, it can be seen that the relationship between the encoding bit rate and the psychological factor score is different for each psychological factor and each video parameter. Therefore, in order to change one video parameter and the encoding bit rate and improve a specific psychological factor score, a psychological factor that formulates the relationship between the video parameter and the encoding bit rate and the psychological factor score for each psychological factor It is necessary to use a score estimation technique. As shown in FIGS. 1A to 1C, in the region where the encoding bit rate is high, the psychological factor score increases when the video parameter is improved, but in the region where the encoding bit rate is low, this is not necessarily the case. Not. This is because, in an area where the encoding bit rate is low, when the comparison is made at the same bit rate, a video with improved video parameters has a higher compression rate and a large deterioration occurs.

  Next, a case where two or more video parameters are input will be described. FIG. 2 shows the relationship between the coding bit rate and the psychological factor score when all three parameters are changed for each psychological factor. Here, 4K60P10-bit video and 2K30P8-bit video are illustrated. As shown in FIG. 2, taking into account the characteristics of each video parameter shown in FIG. 1, due to the synergistic effect of the video parameters, 4K60P10bit video is higher in each psychological factor score than 2K30P8bit video in the region where the encoding bit rate is high. Becomes higher. In the region where the encoding bit rate is low, the psychological factor score is lower in the 2K30P8-bit video than in the 4K60P10-bit video. Therefore, when two or more video parameters are changed simultaneously, a technique for deriving each psychological factor score in consideration of a synergistic effect between the video parameters is necessary.

  Furthermore, when designing a video service, it is necessary to consider not only video but also the influence of sound. There are two methods for adding sound to an image: a method for adding sound unrelated to the image and a method for adding sound simultaneously recorded with the image. Examples of adding sound recorded simultaneously with the image are cooking scenes where the ingredients are cut with a kitchen knife, where the sound of the kitchen knife that matches the movement of the kitchen knife is added, and birds flying in the forest There is an example in which a song of a bird is added in accordance with the movement of a bird in a scene, and an example in which a singing voice is added to a scene in which a choir is singing.

  As shown in FIG. 3, the psychological factor score is improved by the method of adding sound unrelated to the image and the method of adding sound simultaneously recorded with the image as compared to the case without sound. However, the degree of improvement of the psychological factor score differs between the case where the sound recorded simultaneously with the video is added and the case where the sound unrelated to the video is added. When the video and sound are recorded simultaneously, the impression of the presented video space can be received more strongly, and the degree of improvement of the psychological factor score is increased. In addition, the effect of adding sound recorded simultaneously with video on the psychological factor score varies depending on the psychological factor. For example, in the sense of liveliness, naturalness, and openness, adding the sound recorded simultaneously with the video emphasizes the power and spread of the video, so when there is no sound or the audio is unrelated to the video The psychological factor score is higher than the case. Therefore, a technique for deriving a psychological factor score is required depending on whether or not sound is added for each psychological factor and whether or not the added sound is recorded simultaneously with the video.

  However, even when video and sound are recorded simultaneously, the influence on the psychological factor score varies depending on the strength of the relationship between the characteristic object and sound existing in the video. Specifically, the user's line of sight is attracted to a characteristic object that moves in synchronization with the sound among the objects shown in the video. Therefore, the impression of the object remains strong and the psychological factor score is improved. For example, if you suddenly hear the sound of chopping food with a kitchen knife, the user's line of sight is attracted to the kitchen knife that moves in synchronization with the sound, leaving a sharp impression of the kitchen knife. In addition, if a bird's cry is suddenly heard in the middle of a quiet forest scene, the user's line of sight is attracted to the moving bird in synchronization with the sound, and the bird's impression remains clear. However, even if a characteristic sound is heard, if there are many moving objects, the user's line of sight is directed to various places, so the degree of improvement of psychological factors is limited. For example, in a scene in which a choir is singing in large numbers, the user's line of sight cannot be attracted to a specific object, so the degree of improvement in psychological factors is limited. Therefore, when video and sound are recorded at the same time, it is necessary to have a technology for deriving a psychological factor score based on the strength of the relationship between the characteristic object existing in the video and the sound (the degree of synchronization between the video and the sound). is there.

  In addition, when there is a relationship between the characteristic object and the sound existing in the video, the psychological factor score depends on the length of time that the video and the sound are related and the area area of the video space where the moving object exists. The impact is different. Specifically, the psychological factor score improves as the scene in which the video and the sound are related lasts longer in the video scene being viewed. For this reason, it is necessary to quantify how much time is spent in the content being viewed by a scene in which video and sound are related, and reflect this in the estimation of the psychological factor score. In addition, the influence on the user's impression varies depending on how much the object related to sound occupies in the video space. Therefore, it is necessary to quantify the region of the video space where an object related to sound exists and reflect it in the estimation of the psychological factor score. Based on the above, the technology for deriving the psychological factor score by deriving the degree of synchronization between the image and the sound, taking into account the length of time that the image and sound are related and the area of the image space where there is a moving object is necessary.

  The present invention has been made in view of the above points, and the video service designer uses psychological factors as service appeal points based on video parameters, quality design parameters, and the degree of synchronization between video and audio. An object of the present invention is to provide a technique that makes it possible to estimate a psychological factor score.

According to the disclosed technique, input means for inputting psychological factors, one or more video parameters, one or more quality design parameters, a video signal, and an audio signal;
Storage means for holding a coefficient of a mathematical formula for calculating a psychological factor score estimate based on the video parameter, the quality design parameter, and the degree of synchronization between the video and the sound, for each of a plurality of psychological factors;
Video / audio synchronization calculating means for calculating the degree of synchronization between video and sound from the video signal and the audio signal input by the input means;
Coefficients corresponding to psychological factors and video parameters input by the input means are read from the storage means, and the video parameters and quality design parameters input by the input means, and the video and audio are added to mathematical expressions to which the coefficients are applied. Calculating means for calculating a psychological factor score estimate by inputting the degree of synchronization between the video and the sound calculated by the synchronization degree calculating means;
An apparatus for calculating a psychological factor score estimate value, comprising: output means for outputting a psychological factor score estimate value calculated by the calculating means is provided.

  According to the disclosed technology, the video service designer considers the influence of sound, and changes the target parameter by changing which of the video parameters such as resolution, frame rate, and bit depth and the encoding bit rate of the quality design parameters. It becomes possible to grasp whether or not the psychological factor score can be achieved. As a result, a video service with higher user satisfaction can be provided.

It is a figure which shows the example of a relationship between a video parameter and a psychological factor score when the input of a video parameter is one. It is a figure which shows the example of a relationship between a video parameter and a psychological factor score when the input of a video parameter is two or more. It is a figure which shows the example of a relationship between an acoustic and a psychological factor score. It is a figure which shows the structural example of the psychological factor score estimation apparatus in embodiment of this invention. It is a figure which shows the hardware structural example of the psychological factor score estimation apparatus in embodiment of this invention. It is a figure which shows the flow of the audiovisual synchronization degree estimation part 21 in embodiment of this invention. It is a figure which shows the example of extraction of the sound area in which the sound and image | video in embodiment of this invention may be synchronized. It is a figure which shows the example of extraction of the image | video frame area | region which changes according to the sound in embodiment of this invention. It is a figure which shows the example of a relationship between the image parameter in embodiment of this invention, and coefficient (alpha) of the psychological factor score derivation function. It is a figure which shows the example of a relationship between the image | video audio | voice synchronization degree in embodiment of this invention, and coefficient (gamma) of the psychological factor score derivation | leading-out function.

  Hereinafter, an embodiment (this embodiment) of the present invention will be described with reference to the drawings. The embodiment described below is merely an example, and the embodiment to which the present invention is applied is not limited to the following embodiment. In this embodiment, five psychological factors, three video parameters, and one quality design parameter are used, but these are examples. For example, more than 5 psychological factors may be used, less than 5 one or more psychological factors may be used, more than 3 video parameters may be used, 3 Less one or more video parameters may be used, and more than one quality design parameter may be used.

(Outline of the embodiment)
In the present embodiment, there is provided a psychological factor score estimating apparatus for estimating a psychological factor score of a psychological factor as a service appeal point in a video service such as a video distribution service and a videophone service. The psychological factor score estimation device is composed of a psychological factor score estimation unit and a coefficient DB. The psychological factor score estimation unit includes a video / audio synchronization degree estimation unit. Inputs are psychological factors to be estimated, video parameters, encoding bit rate, video signal, and audio signal.

  In the video / audio synchronization estimation unit, the video / sound synchronization is estimated from the video signal and the audio signal. The psychological factor score estimation unit estimates a psychological factor score using a psychological factor score derivation function. The psychological factor score derivation function takes into account that the relationship between the encoding bit rate and the psychological factor score varies depending on the input video parameters as shown in FIG. 1, and the synergistic effect between the video parameters as shown in FIG. As shown in FIG. 3, a formula formulated in consideration of the necessity of considering the effect of synchronization between video and audio is used.

  The coefficient of the psychological factor score derivation function is stored in the coefficient DB, and is set, for example, for each video parameter, for each psychological factor, and for each degree of video audio synchronization. This is because, as shown in FIG. 1, the relationship between the encoding bit rate and the psychological factor score is different for each psychological factor and for each video parameter. Also, as shown in FIG. This is because the psychological factor score fluctuates. As described above, the psychological factor score is derived by inputting the value of the video parameter to the psychological factor score deriving function reflecting the coefficient of the inquiry result to the coefficient DB.

  Hereinafter, this embodiment will be described in more detail.

(Device configuration)
FIG. 4 shows a configuration example of the psychological factor score estimation apparatus 1 in the embodiment of the present invention. As shown in FIG. 4, the psychological factor score estimating apparatus 1 according to the embodiment of the present invention includes a psychological factor score estimating unit 2 and a coefficient DB 3. The psychological factor score estimation unit 2 includes a video / audio synchronization degree estimation unit 21. The psychological factor score estimation unit 2 inputs psychological factors i, video parameters such as resolution x1, frame rate x2, bit depth x3, quality design parameter encoding bit rate br, video signal sv, and audio signal sa. Input means. The psychological factor score estimation device 1 may be referred to as a psychological factor score estimated value calculation device.

  The video / audio synchronization estimation unit 21 derives the video / audio synchronization av using the input video signal sv and the audio signal sa.

  The psychological factor score estimation unit 2 derives a psychological factor score Si using a psychological factor score derivation function. The coefficient of the psychological factor score derivation function is stored in the coefficient DB 3. The input when the psychological factor score estimation unit 2 derives the psychological factor score using the psychological factor score derivation function is the psychological factor i, video parameters such as resolution x1, frame rate x2, bit depth x3, and quality design parameters. The encoding bit rate br and the video / audio synchronization level av that is the output of the video / audio synchronization level estimation unit 21. The output is a psychological factor score Si.

  The psychological factor score estimation apparatus 1 can be realized by causing a computer to execute a program describing the processing contents described in the present embodiment. FIG. 5 is a diagram illustrating a hardware configuration example of the psychological factor score estimation apparatus 1. The apparatus illustrated in FIG. 5 includes a drive device 100, an auxiliary storage device 102, a memory device 103, a CPU 104, an interface device 105, a display device 106, an input device 107, and the like that are connected to each other via a bus B.

  A program that realizes processing in the psychological factor score estimation apparatus 1 is provided by a recording medium 101 such as a CD-ROM or a memory card, for example. When the recording medium 101 storing the program is set in the drive device 100, the program is installed from the recording medium 101 to the auxiliary storage device 102 via the drive device 100. However, the program need not be installed from the recording medium 101 and may be downloaded from another computer via a network. The auxiliary storage device 102 stores the installed program and also stores necessary files and data.

  The memory device 103 reads the program from the auxiliary storage device 102 and stores it when there is an instruction to start the program. The CPU 104 realizes functions related to the psychological factor score estimation apparatus 1 according to a program stored in the memory device 103. The interface device 105 is used as an interface for connecting to a network. The display device 106 displays a GUI (Graphical User Interface) or the like by a program. The input device 107 includes a keyboard and mouse, buttons, a touch panel, and the like, and is used to input various operation instructions.

  Hereinafter, the operation of each functional unit in the psychological factor score estimation apparatus 1 will be described in detail.

(Operation of the video / audio synchronization estimation unit 21)
The video / audio synchronization estimation unit 21 derives the video / audio synchronization av using the input video signal sv and the audio signal sa. FIG. 6 shows a flow for deriving the video / audio synchronization degree av.

  First, the video / audio synchronization estimation unit 21 checks whether or not the audio signal sa is input, and if it is not input (in the case of “No” in Step S101), av = −1 is set (Step S101). S102).

  Next, when the audio signal sa is input (in the case of “Yes” in step S101), the audio / video synchronization estimation unit 21 extracts an audio section in which the audio and the video may be synchronized. Is performed (step S103).

  FIG. 7 shows an example of extracting an acoustic section in which sound and video may be synchronized. In the example of FIG. 7, a section including a sound with a large amplitude such as a bird call is detected as the acoustic section. Specifically, it is conceivable to set a threshold value and detect a time zone in which the amplitude exceeds the threshold value. In the example of FIG. 7, sections from time t11 to t12 and from t21 to t22 are detected. As a method for setting the threshold value, it is conceivable to derive an average value and a standard deviation of all sections of the acoustic signal and use the average value + standard deviation as a threshold value. If there is no time zone that exceeds the threshold, av = 0.

  When the sound exceeding the threshold is included, the video / audio synchronization estimation unit 21 derives a video frame region including a characteristic object having a large motion in the time zone (step S104).

  A processing example of step S104 will be described with reference to FIG. Here, first, the absolute value of the inter-frame difference value of the pixel value is derived in an acoustic section in which the sound and the video may be synchronized. In the upper part of FIG. 8, as an example of extracting a video frame region including an object with large motion, pixels including a moving object in each frame are shown in black. Further, the absolute value of the inter-frame difference value of the pixel value between time t11 and time t11 + 1, the absolute value of the inter-frame difference value of the pixel value between time t11 + 1 and time t11 + 2, and time t11 It is shown that the absolute value of the inter-frame difference value of the pixel value between +2 and time t11 + 3 (= t12) is calculated.

  In the lower part of FIG. 8, pixels that exceed the threshold in absolute value of inter-frame difference values of pixel values are shown. As the threshold value, in order to extract pixels with a large amount of change between frames, the average value of absolute values of frame differences of all pixels in the frame + 2 × standard deviation may be considered. From FIG. 8, it can be seen that a moving object is included in the region from V2 to V4 in the horizontal direction and the region from H1 to H3 in the vertical direction from t11 to t12. Similarly, a video frame region including a moving object is extracted from t21 to t22.

  Finally, the audiovisual synchronization degree estimation unit 21 derives and outputs the audiovisual synchronization degree av (steps S105 and S106 in FIG. 6). Here, it is assumed that the number of pixels in the horizontal direction of the video frame region extracted in the time period tk1 to tk2 in which the audio and the video may be synchronized is Vk, and the number of pixels in the vertical direction is Hk. Since the psychological factor improves as the time period in which the video and audio are synchronized is longer, the psychological factor is considered to be improved as the frame area occupied by the object synchronized with the audio is wider. Derived by

In Equation (1), W represents the total number of pixels in the frame, L represents the time length of the video content, and n represents the number of audio sections in which the audio and video may be synchronized. In the case of FIG. 7, n = 2. Note that if there is no moving object in the time zone extracted as the sound section in which audio and video may be synchronized, that is, if Vk = 0 and Hk = 0, the video and audio Indicates that they are not synchronized, and av = 0 from equation (1).

(Operation of psychological factor score estimation unit 2)
The psychological factor score estimator 2 uses the psychological factor score based on the input psychological factor i, video parameters x1 to x3, the encoding bit rate br, and the video / audio synchronization level av output from the video / audio synchronization level estimation unit 21. Estimate Si.

  The psychological factor score Si is a coefficient of the psychological factor score derivation function set based on the relationship between the psychological factor score derivation function and the video parameter and the psychological factor i as shown in the following equations (2) to (7). And a coefficient of a psychological factor score derivation function that takes into account the degree of synchronization between video and audio, and the values of the input video parameters and encoding bit rate. As the psychological factor score derivation function, for example, a function formulated based on the relationships shown in FIGS. The coefficient of the psychological factor score derivation function is determined for each video parameter, for each psychological factor, and for each acoustic flag.

Si = γia (av) (αi1 (x1) × αi2 (x2) × αi3 (x3) × log ₁₀ (br) + βi) (2)
αi1 = ai1 × exp (bi1 × x1) + ci1 (3)
αi2 = ai2 × exp (bi2 × x2) + ci2 (4)
αi3 = ai3 × exp (bi3 × x3) + ci3 (5)
γia = 1 (when av = -1) (6)
γia = di × av + ei (when av> = 0) (7)
Here, in equation (2), psychological factor i is aesthetic sense, i = 1 is dynamic, i = 3 is natural, i = 4 is open, and i = 5 is stereoscopic. . The coefficient αi1 (i = 1 to 5) of the psychological factor score derivation function is a function of resolution x1, the coefficient αi2 (i = 1 to 5) is a function of the frame rate x2, and the coefficient αi3 (i = 1 to 5) is the bit depth x3. Is a function of

  The coefficient α i1 is formulated based on the relationship shown in FIG. Specifically, since the video becomes clearer when the resolution is improved, it is formulated according to equation (3) for each psychological factor, considering that the aesthetic feeling and the natural feeling are more likely to be improved. The coefficient α i2 is formulated based on the relationship shown in FIG. Specifically, when the frame rate is improved, the movement becomes smoother, so that the dynamic feeling and the natural feeling are more likely to be improved. The coefficient α i3 is formulated based on the relationship shown in FIG. Specifically, when the bit depth is improved, since the edge of the subject stands out, the stereoscopic effect and the open feeling are more easily improved, and the formulation is formulated as in equation (5) for each psychological factor.

  The coefficients aij, bij, cij, βi (i = 1 to 5, j = 1 to 3) of the psychological factor score derivation function in equations (2) to (5) are the subjective quality without adding the sound performed in advance. It is determined based on the evaluation experiment result and stored in the coefficient DB 3 for each of a plurality of psychological factors.

  In equation (2), the coefficient γia is a coefficient that takes into account the effect of video and audio synchronization, and is a function of the output av of the video and audio synchronization estimation unit 21. When no sound is input (av = −1), γia = 1 is set (Equation (6)) because the effect of adding sound is not considered. When av> = 0, in order to consider the effect due to the addition of sound, it is formulated as shown in equation (7) based on the relationship shown in FIG. Specifically, the addition of sound that is synchronized with the video emphasizes the power and spread of the video, so the psychological score increases with the degree of video audio synchronization in terms of liveliness, naturalness, and openness. For each psychological factor, formula (7) is formulated. Coefficients di and ei (i = 1 to 5, j = 1 to 3) are determined based on the subjective quality evaluation experiment result obtained by adding sound in advance, and stored in coefficient DB 3 for each of a plurality of psychological factors. I will keep it.

  That is, the psychological factor score estimator 2 reads out the coefficient corresponding to the input psychological factor and the video parameter from the coefficient DB 3, and inputs the coefficient to the mathematical formula (for example, the formulas (2) to (7)) to which the coefficient is applied. The estimated psychological factor score is calculated by inputting the video parameters and quality design parameters and the degree of synchronization between the video and the sound.

  In equation (2), the relationship between each video parameter and the psychological factor score is formulated by a non-linear function based on FIG. 1, but may be expressed by a linear function. Further, in order to take into account the synergistic effect between the video parameters, the coefficient αij relating to each video parameter is multiplied, but it is also possible to add as shown in the following equation (8).

Si = γia (av) × ((αi1 (x1) + αi2 (x2) + αi3 (x3)) × log ₁₀ (br) + βi) (8)
In addition, the expressions (3) to (5) formulate the relationship between each image parameter and the coefficient α of the psychological factor score derivation function as a non-linear function based on FIG. 9, but may be expressed as a linear function. Good. Further, in the expression (7), the relationship between the audiovisual synchronization degree av and the coefficient γ of the psychological factor score derivation function is formulated as a linear function based on FIG. 10, but may be expressed as a nonlinear function. .

  Since the service designer basically knows all the values of resolution x1, frame rate x2, bit depth x3, and encoding bit rate br, all of them are expressed by equations (2) to (7). Can be used as input. Therefore, if there are equations (2) to (7), a psychological factor score can be derived.

  However, for example, when the correspondence between the bit depth, the frame rate, and the psychological factor is unnecessary or the correspondence is not obtained in the service design, that is, for example, when the correspondence between the resolution and the psychological factor score is obtained, It is good also as simplifying a psychological factor score derivation function like (9) and a formula (10). Expressions (9) and (10) are expressions when a correspondence relationship between one video parameter (resolution, bit depth, or frame rate) and a psychological factor score is obtained. For example, Formula (9) is formulated based on the relationship shown in FIG. 1, Formula (10) is formulated based on the relationship shown in FIG. 9, Formula (12) is formulated based on the relationship shown in FIG. To do.

Si = γija (av) × (αij (xj) × log ₁₀ (br) + βij) (9)
αij = aij × exp (bij × xj) + cij (10)
γija = 1 (when av = -1) (11)
γija = dij × av + eij (when av> = 0) (12)
Here, as described above, the coefficients aij, bij, cij, βij, dij, and eij (i = 1 to 5, j = 1 to 3) of the psychological factor score derivation function are the results of the subjective quality evaluation experiment performed in advance. It is determined based on this and stored in the coefficient DB 3 for each of a plurality of psychological factors.

  As described above, for each psychological factor, a psychological factor score derivation function that formulates the relationship between the encoding bit rate of video parameters and quality design parameters, the relationship between video and audio synchronization indicating the degree of video and audio synchronization, and the psychological factor score. By using it, the psychological factor score can be derived.

(Summary of embodiment)
As described above, according to the present embodiment, input means for inputting psychological factors, one or more video parameters, one or more quality design parameters, video signals, and audio signals, and video parameters Storage means for holding a coefficient of a mathematical formula for calculating a psychological factor score estimate based on the quality design parameters and the degree of synchronization between video and sound for each of a plurality of psychological factors, and the input means Video / audio synchronization degree calculation means for calculating the degree of synchronization between video and sound from the input video signal and audio signal, and the psychological factors input by the input means and coefficients corresponding to the video parameters are read from the storage means. The video parameter and the quality design parameter input by the input unit, and the video / audio synchronization degree calculating unit calculated by the formula to which the coefficient is applied. A psychology feature comprising: calculation means for calculating a psychological factor score estimate value by inputting a degree of synchronization between video and sound; and an output means for outputting the psychological factor score estimate value calculated by the calculation means. A factor score estimated value calculation apparatus is provided.

  The psychological factor score estimation device 1 described in the embodiment is an example of a psychological factor score estimated value calculation device. The psychological factor score estimation unit 2 is an example of a functional unit including an input unit, a storage unit, a video / audio synchronization degree calculation unit, a calculation unit, and an output unit. The video / audio synchronization level estimation unit 21 is an example of a video / audio synchronization level calculation unit.

  The mathematical expression may be a mathematical expression obtained by formulating the relationship between the video parameter, the quality design parameter, the synchronization degree of the video and sound, and the psychological factor score obtained based on the subjective quality evaluation experiment result.

  In addition, for each of the plurality of video parameters, the formula is calculated based on the relationship between the video parameter, the quality design parameter, the degree of synchronization between the video and the sound, and the psychological factor score, and the plurality of video parameters It is good also as a numerical formula obtained by combining the said numerical formula obtained with respect to each of each.

  The video / audio synchronization degree calculating means is based on the video signal and the audio signal input from the input means, and the time length of synchronization between the video and the audio, and the object synchronized with the audio is a video frame. It is also possible to calculate the area occupied by the image and calculate the degree of synchronization between the video and the sound, which are the inputs of the mathematical formula, from the time length and the area.

  Although the present embodiment has been described above, the present invention is not limited to the specific embodiment, and various modifications and changes can be made within the scope of the gist of the present invention described in the claims. Is possible.

1 psychological factor score estimating device 2 psychological factor score estimating unit 3 coefficient DB
21 Video / audio synchronization estimation unit
Si psychological factor score
i Psychological factor ID
j Video parameter ID
x1 resolution
x2 frame rate
x3 bit depth
sa acoustic signal
sv video signal
av Audiovisual synchronization
k Time zone IDs where audio and video may be synchronized
n Number of time periods in which audio and video may be synchronized
tk1 Start time of time zone k when audio and video may be synchronized
tk2 Start time of time zone k when audio and video may be synchronized
Vk Number of pixels in the horizontal direction that contain moving objects in time zone k when audio and video may be synchronized
Hk Number of pixels in the vertical direction that contain moving objects in time zone k when audio and video may be synchronized
Total number of pixels in W frame
L Video content time book α, β, γ, a, b, c, d, e Coefficient of psychological factor score derivation function 100 Drive device 101 Recording medium 102 Auxiliary storage device 103 Memory device 104 CPU
105 interface device 106 display device 107 input device

Claims (8)

Input means for inputting psychological factors, one or more video parameters, one or more quality design parameters, a video signal, and an audio signal;
Storage means for holding a coefficient of a mathematical formula for calculating a psychological factor score estimate based on the video parameter, the quality design parameter, and the degree of synchronization between the video and the sound, for each of a plurality of psychological factors;
Video / audio synchronization calculating means for calculating the degree of synchronization between video and sound from the video signal and the audio signal input by the input means;
Coefficients corresponding to psychological factors and video parameters input by the input means are read from the storage means, and the video parameters and quality design parameters input by the input means, and the video and audio are added to mathematical expressions to which the coefficients are applied. Calculating means for calculating a psychological factor score estimate by inputting the degree of synchronization between the video and the sound calculated by the synchronization degree calculating means;
The psychological factor score estimated value calculation apparatus comprising: output means for outputting the psychological factor score estimated value calculated by the calculating means. The mathematical formula is a mathematical formula obtained by formulating a relationship between a video parameter, a quality design parameter, a video and audio synchronization degree, and a psychological factor score obtained based on a subjective quality evaluation experiment result. The psychological factor score estimated value calculation apparatus according to claim 1. For each of the plurality of video parameters, the formula obtains a formula based on the relationship between the video parameter, the quality design parameter, the degree of synchronization between the video and the sound, and the psychological factor score, and each of the plurality of video parameters The psychological factor score estimated value calculation apparatus according to claim 1, wherein the mathematical formula is obtained by combining the mathematical formulas obtained with respect to. The video / audio synchronization degree calculating means includes:
Based on the video signal and the audio signal input by the input means, the length of time in which the video and the sound are synchronized and the area occupied by the object synchronized with the sound in the video frame are calculated, and the time The psychological factor score estimation value calculation according to any one of claims 1 to 3, wherein the degree of synchronization between the image and the sound, which is the input of the mathematical expression, is calculated from the length and the region. apparatus. Psychological factor comprising storage means for holding a coefficient of a mathematical formula for calculating a psychological factor score estimate based on the video parameter, the quality design parameter, and the degree of synchronization between the video and the sound, for each of a plurality of psychological factors A psychological factor score estimated value calculation method executed by the score estimated value calculation device,
An input step for inputting psychological factors, one or more video parameters, one or more quality design parameters, a video signal, and an audio signal;
A video / audio synchronization degree calculating step for calculating a degree of synchronization between video and sound from the video signal and the audio signal input in the input step;
Coefficients corresponding to psychological factors and video parameters input in the input step are read from the storage means, and the video parameters and quality design parameters input in the input step and the video and audio are added to mathematical formulas using the coefficients. A calculation step for calculating a psychological factor score estimate by inputting the degree of synchronization between the video and the sound calculated in the synchronization calculation step;
An output step of outputting the psychological factor score estimate calculated by the calculating step. The mathematical formula is a mathematical formula obtained by formulating a relationship between a video parameter, a quality design parameter, a video and audio synchronization degree, and a psychological factor score obtained based on a subjective quality evaluation experiment result. The psychological factor score estimated value calculation method according to claim 5. In the video / audio synchronization degree calculating step, the psychological factor score estimated value calculating device comprises:
Based on the video signal and the audio signal input in the input step, the length of time in which the video and the sound are synchronized and the area occupied by the object synchronized with the sound in the video frame are calculated, and the time The psychological factor score estimated value calculation method according to claim 5 or 6, wherein a degree of synchronization between video and sound, which is an input of the mathematical formula, is calculated from the length of and the region.   The program for functioning a computer as each means in the psychological factor score estimated value calculation apparatus of any one of Claims 1 thru | or 4.