JP7351398B2 - VR video encoding parameter calculation device, VR video encoding parameter calculation method, and program - Google Patents

Description

Application of Article 30, Paragraph 2 of the Patent Act On July 11, 2019, the Institute of Electronics, Information and Communication Engineers, IEICE Technical Report, Communication Quality Study Group (CQ) vol. 119 No. 125 CQ2019-52 pp. Presented at 79-83

The present invention relates to a technique for calculating encoding parameters in VR video distribution.

In recent years, with the development of head-mounted displays (HMDs) and encoding technology, opportunities for users to view virtual reality (VR) images that can be viewed 360° are increasing. Along with this, the number of VR video distribution services is increasing, and the opportunities for users to view VR video using HMDs, smartphones, conventional stationary displays, etc. are also increasing.

When a user views a VR video, for example, by wearing an HMD, the user can change the direction of the line of sight by shaking his head or moving his body. You can change the viewing direction of the video by operating it.

In VR video distribution, for example, using a distribution method called MPEG-DASH shown in Non-Patent Document 1, files encoded at multiple bit rates are prepared on the server side, and the optimal bit rate is selected according to the bandwidth. Deliver rate videos.

Furthermore, there are broadly two types of VR video encoding/distribution methods. Similar to conventional 2D video, there is a uniform distribution method that distributes the entire video with uniform image quality, and a high-quality distribution method that distributes the video in the user's viewing direction displayed on an HMD, etc., to other HMDs, etc. There is a method called tile-based distribution that reduces bandwidth by distributing videos that are not displayed in low quality or not distributing them at all.

Quality deterioration due to VR video encoding that is common to both uniform type and tile-based methods includes spatial distortion due to blurring and reduction in definition, temporal distortion such as deterioration of smoothness and flickering, and noise at edges. There is deterioration due to spatiotemporal distortion such as.

In addition, in tile-based VR video distribution, the user can view tiles sent in high image quality (hereinafter referred to as high-quality tiles) that are viewed when the user does not change the viewing direction, and tiles (hereinafter referred to as high-quality tiles) that are viewed temporarily when the user changes the viewing direction. Watch both videos (low quality tiles below). When a user changes the viewing direction, there is a time (hereinafter referred to as switching delay) until the tile in the new viewing direction is switched from a low-quality tile to a high-quality tile, and the video is Perceiving a deterioration in quality. The switching delay can be designed to some extent by changing the length of one chunk during encoding, the buffer time during distribution, etc., but if the switching delay is long, the time spent viewing low-quality tiles may be reduced. Since the tiles are longer, it is thought that the contribution rate to the quality experienced by the user by the low-quality tiles becomes larger.

Furthermore, since the size of a high-quality tile displayed on an HMD or the like changes depending on the resolution of the high-quality tile and the low-quality tile, it is thought that the contribution rate to the quality experienced by the user changes depending on the resolution. Furthermore, if low-quality video is not distributed in tile-based VR video distribution, the user will perceive a deterioration in quality because the video will temporarily become unwatchable when the viewing direction is changed.

As described above, in a distribution system such as MPEG-DASH, in order to achieve optimal quality control for users, it is necessary to optimize the distribution quality prepared in advance. Since distribution quality varies greatly depending on the encoding parameters given when encoding video content, it is important to develop techniques for calculating encoding parameters suitable for video content in both uniform and tile-based formats. Encoding parameters include bit rate, resolution, frame rate, and quantization parameters. Furthermore, in tile-based VR video distribution, it is important to derive encoding parameters that take into account the overall distribution quality felt from the distribution of high-quality areas and low-quality areas.

Patent Document 1 describes a method for deriving encoding parameters such as a bit rate corresponding to a target quality of input in order to optimize distribution quality for conventional 2D video distribution. Specifically, by estimating the quality of many combinations of encoding parameter values, combinations that satisfy the target quality are derived. Note that the target quality may be any value between 1 and 5, or may be a value between 0 and 100.

As a conventional 2D video quality estimation technique, in addition to the above-mentioned Patent Document 1, for example, a standardized technique is disclosed in Non-Patent Document 2.

However, these quality estimation techniques are aimed at 2D images, and quality estimation techniques for VR images have not been established. Furthermore, in tile-based VR video distribution, since the user views videos of both high-quality tiles and low-quality tiles, the image quality of each of the high-quality and low-quality tiles affects the overall quality.

I. Sodagar, "The MPEG-DASH Standard for Multimedia Streaming Over the Internet," in IEEE MultiMedia, vol. 18, pp. 62-67, 2011. Parametric bitstream-based quality assessment of progressive download and adaptive audiovisual streaming services over reliable transport, Recommendation ITU-T P.1203, 2017.

JP2015-65517A

As mentioned above, conventional 2D video quality estimation and encoding parameter calculation such as Non-Patent Document 2 and Patent Document 1 do not target uniform type and tile-based VR videos. In particular, in tile-based VR video distribution, encoding parameters cannot be calculated based on the overall quality of such high-image quality and low-image quality tiles.

The present invention has been made in view of the above points, and an object of the present invention is to provide a technology that makes it possible to calculate encoding parameters of VR video based on target quality in VR video distribution.

According to the disclosed technology, a quality estimator that calculates an overall quality estimate of a VR video for each of a plurality of encoding parameters;
A VR video encoding parameter calculation device comprising: a VR video parameter extraction unit that extracts an encoding parameter corresponding to an overall quality estimate that satisfies a given target quality from among the plurality of encoding parameters,
The quality estimator includes:
a VR video quality estimator that calculates a quality estimate of a high-quality area in the VR video and a quality estimate of a low-quality area in the VR video;
an overall quality estimation unit that calculates the overall quality estimate based on the quality estimate of the high image quality area and the quality estimate of the low image quality area,
The overall quality estimating unit is
The quality of the high image quality area and the quality of the low image quality area based on at least one of the parameters related to the time taken to switch from the low image quality area display state to the high image quality area display state and information regarding the area occupied by the low image quality area. Calculate the contribution of
The overall quality estimate is calculated based on the contribution, the quality estimate of the high image quality area, and the quality estimate of the low image quality area.
A VR video encoding parameter calculation device is provided.

According to the disclosed technology, it is possible to provide a technology that makes it possible to calculate encoding parameters of VR video based on target quality in VR video distribution.

FIG. 1 is a configuration diagram of a VR video encoding parameter calculation device according to a first embodiment of the present invention. FIG. 3 is a diagram showing an image of input information. FIG. 3 is a diagram for explaining input, calculation information, and output. FIG. 3 is a diagram for explaining input, calculation information, and output. FIG. 3 is a diagram for explaining input, calculation information, and output. FIG. 3 is a diagram for explaining input, calculation information, and output. FIG. 3 is a diagram for explaining input, calculation information, and output. FIG. 3 is a diagram for explaining input, calculation information, and output. 3 is a flowchart showing a processing procedure of the VR video encoding parameter calculation device in the first embodiment. FIG. 2 is a configuration diagram of a VR video encoding parameter calculation device in a second embodiment. It is a flowchart which shows the processing procedure of the VR video encoding parameter calculation apparatus in 2nd Embodiment. FIG. 2 is a diagram illustrating an example of a hardware configuration of a VR video encoding parameter calculation device.

Embodiments of the present invention will be described below with reference to the drawings. The embodiments described below are merely examples, and embodiments to which the present invention is applied are not limited to the following embodiments.

In the following embodiments, the user can view a 360° VR image by wearing a head-mounted display (HMD), etc., and can change the direction of the user's line of sight by shaking his head, moving his body, etc. The coding of VR images is determined based on the target value of the VR image quality experienced when viewing the image on a conventional stationary display or in a state where the viewing direction of the image can be changed by operating a mouse, etc. A VR video encoding parameter calculation device that calculates encoding parameters (bit rate, resolution, frame rate, quantization parameter, etc.) will be described.

(First embodiment)
Hereinafter, a first embodiment of the present invention will be described in detail with reference to the drawings. FIG. 1 shows the configuration of a VR video encoding parameter calculation device 10 according to a first embodiment of the present invention.

As shown in FIG. 1, the VR video encoding parameter calculation device 10 includes a VR video quality estimation section 15, a total quality estimation section 16 including a contribution degree calculation section 17, and a VR video parameter extraction section 18. Note that the VR video quality estimating section 15 and the total quality estimating section 16 may be collectively referred to as a quality estimating section. The same applies to the second embodiment.

FIG. 1 also shows an encoding condition setting section 11, a distribution setting input section 12, a target quality input section 13, and a parameter setting value candidate group input section 14. As illustrated, encoding parameters are set from the encoding condition setting unit 11 to the VR video quality estimating unit 15, the total quality estimating unit 16, and the VR video parameter extracting unit 18.

Distribution settings are input from the distribution settings input section 12 to the overall quality estimation section 16 and the VR video parameter extraction section 18 . The target quality is input from the target quality input section 13 to the VR video parameter extraction section 18 . A candidate group of encoding parameters is input from the parameter setting value candidate group input unit 14 to the VR video quality estimation unit 15 , the total quality estimation unit 16 , and the VR video parameter extraction unit 18 . The operation of each part will be explained in more detail below.

The encoding condition setting unit 11 inputs video encoding settings (video encoding settings) required for VR video distribution to the VR video encoding parameter calculation device 10. Here, the video encoding settings represent, for example, the frame rate, resolution, codec, bit rate, quantization parameter (hereinafter referred to as QP) of each of the high-quality tiles and low-quality tiles of the tile-based VR video. Note that the high image quality tile and the low image quality tile may be referred to as a high image quality area and a low image quality area, respectively.

The distribution setting input unit 12 inputs distribution settings for VR video distribution to the VR video encoding parameter calculation device 10. Here, the distribution setting refers to switching delay in VR video distribution. The target quality input unit 13 inputs the target video quality of the distributed VR video to the VR video encoding parameter calculation device 10.

The parameter setting value candidate group input unit 14 selects one or more setting value candidates for the encoding parameters for the encoding parameters that have not been input in the encoding condition setting unit 11 and the distribution setting input unit 12. It is input to the encoding parameter calculation device 10.

FIG. 2 is a diagram showing an image of input information in the VR video encoding parameter calculation device 10. Values other than the bit rate illustrated in FIG. 2A are input by the encoding condition setting section 11, the distribution setting input section 12, and the target quality input section 13.

In addition, as shown in FIG. 2(b), the parameter setting value candidate group input unit 14 inputs encoding parameters that have not been input in the encoding condition setting unit 11, the distribution setting input unit 12, and the target quality input unit 13. Bitrate is entered.

FIG. 2(b) shows a case where 5 Mbps (fixed) is input for low-quality tiles and values of 10 to 20 Mbps in 1-Mbps increments are input as parameter setting value (bit rate) candidate groups for high-quality tiles. There is.

The parameter setting value candidate group input unit 14 stores parameter setting value candidates in advance in its own database, etc., and the VR video quality estimation unit 15 inputs the encoding condition setting unit 11, the distribution setting input unit 12, and the target quality. It may also be possible to refer to missing coding parameter candidates input from the input unit 13.

The VR video quality estimation unit 15 estimates the quality of each high-quality tile and low-quality tile of the VR video based on the encoding parameters input from the encoding condition setting unit 11 and the parameter setting value candidate group input unit 14. is calculated, and a quality estimate corresponding to the number of encoding parameter sets input from the parameter setting value candidate group input unit 14 is output.

The overall quality estimating unit 16 uses the quality of the high image quality tiles and the quality of the low image quality tiles output by the VR video quality estimating unit 15, the encoding parameters input from the parameter setting value candidate group input unit 14, and the distribution settings. Then, the overall quality of the VR video is calculated and output. The contribution calculation unit 17 calculates the contribution of the quality of the high image quality tile and the quality of the low image quality tile to the overall quality.

The VR video parameter extraction unit 18 extracts coding parameters that have a quality close to the target quality from among the total quality output from the total quality estimation unit 16, thereby determining the codes of each of the high-quality tiles and the low-quality tiles. Calculate the parameters.

The VR video parameter extraction unit 18 may output only one set of encoding parameters for high-quality tiles and low-quality tiles, or the quality is close to the target quality (for example, the difference from the target quality is within 0.1). The output parameters may be a plurality of sets of encoding parameters or all sets of encoding parameters whose quality satisfies the target quality. In addition, in this specification and claims, in addition to the case where the quality is equal to or higher than the target quality, the term "target quality" is also used when the quality is close to the target quality (for example, the difference from the target quality is within 0.1). It can also be expressed as "satisfying". In addition, when outputting only encoding parameters that are close to the target quality (for example, the difference from the target quality is within 0.1), the case where the quality is close to the target quality is expressed as "meeting the target quality". Good too.

Examples of input, calculated overall quality, and output in the VR video encoding parameter calculation device 10 are shown in FIGS. 3 to 5.

In this example, as shown in FIG. 3, the resolution of high-quality tiles (3840x1920), the resolution of low-quality tiles (1920x960), The frame rate (30 fps), delay in switching from low-quality tile display state to high-quality tile display state (2 seconds), target quality (4.0), and codec (H.264) are given, and these encoding conditions and distribution A case is shown in which a bit rate close to the target quality is calculated based on the conditions.

Further, as shown in FIG. 3, for the parameter setting value candidate group, the bit rate of the low image quality tile is fixed at 5 Mbps, and the bit rate of the high image quality tile is 20 to 10 Mbps, which is input every 1 Mbps.

The VR video quality estimation section 15 and the total quality estimation section 16 estimate the video quality for each bit rate set of the parameter setting value candidate group, and the total quality is output from the total quality estimation section 16. As shown in FIG. 4, the overall quality is calculated for each of the bit rate candidates (1) to (11) given from the parameter setting value candidate group input unit 14.

The VR video parameter extraction unit 18 inputs the overall quality, distribution settings, encoding parameters, and target quality, and extracts the overall quality that is close to the target quality (for example, the absolute value of the difference from the target quality is within 0.1). The extracted encoding parameters, distribution settings, and overall quality are output as the extracted encoding parameters and estimated quality of each of the high-quality tile and the low-quality tile.

In the example shown in FIG. 5, candidate (5) (high image quality tile 15 Mbps, low image quality tile 5 Mbps) and candidate (6) (high image quality tile 14 Mbps, low image quality tile 5 Mbps) are the absolute value of the difference between the target quality and the overall quality. are extracted because they are within 0.1, and the extracted encoding parameters corresponding to these candidates are output as shown in FIG.

In the examples of FIGS. 3 to 5, the encoding parameters whose overall quality is close to the target quality are output, but all encoding parameters that satisfy the target quality may be output. In that case, encoding parameters corresponding to candidates (1) to (6) whose overall quality is 4.0 or higher are output.

In the above example, the delay, resolution, frame rate, codec, and bit rate are output as encoding parameters that are close to or meet the target quality, but instead of the bit rate, the quantization parameter (QP) may be output. Good too. Examples of such cases are shown in FIGS. 6 to 8.

In the examples shown in FIGS. 6 to 8, the inputs of the encoding condition setting section 11, the distribution setting input section 12, and the goal setting input section 13 are the same as those described in FIGS. 3 to 5. In the examples of FIGS. 6 to 8, QP is used instead of bit rate.

In the examples shown in FIGS. 6 to 8, as the parameter setting value candidate group, the QP of the low image quality tile is fixed at 40, and the QP of the high image quality tile is given from 10 to 40 in 1 increments.

Similar to the examples in FIGS. 3 to 5, the quality is calculated for each QP candidate, and candidate (14) (high-quality tile QP24, low-quality tile QP40) and candidate (15) (high-quality tile QP25, The low-quality tiles QP40) are extracted because the absolute value of the difference between the target quality and the overall quality is within 0.1, and the encoding parameters corresponding to these candidates are output as shown in FIG.

In the examples shown in FIGS. 3 to 6 and 7 to 9, encoding parameters other than the bit rate and QP are input to the VR video encoding parameter calculation device 10 from the encoding condition setting section 11 and the distribution setting input section 12. The bit rate or QP is input from the parameter setting value candidate group input unit 14, but this is just an example, and the encoding condition setting unit 11 and distribution setting input unit 12 input whether or not each encoding parameter is input. can be changed freely, and the parameter setting value candidate group input section 14 inputs missing encoding parameters. Further, the output may be omitted if it is clear from the input, and it is clear that the estimated quality after extraction satisfies or is close to the target quality, so the output may be omitted. For example, in the examples of FIGS. 3 to 6 and 7 to 9, only the extracted bit rate or QP may be output.

<Overall quality estimation method>
Hereinafter, a method for estimating the total quality calculated by the VR video quality estimating section 15 and the total quality estimating section 16 will be described in detail.

Let VQ be the overall quality value (video quality value) experienced when viewing a VR video. In order to formulate the quality of high-quality tiles and the quality of low-quality tiles and obtain VQ, the VR video quality estimation unit 15 uses parameters representing the quality of high-quality tiles, parameters representing the quality of low-quality tiles, and tile switching. The quality degradation and quality of each high-quality tile and low-quality tile are estimated using parameters representing the delay of .

For example, the VR video quality estimation unit 15 can calculate the quality estimated value VQ _H of the high-quality tile using the following formula using the resolution, frame rate, and bit rate of the high-quality tile.

VQ _H = MOS _qH
MOS _qH = q ₁ + q ₂・exp (q ₃・quant _H )
quant _H =a ₁ +a ₂・ln(a ₃ +ln(br _H )+ln(br _H・bpp _H ))

上記の式において、ｂｒ_Ｈは高画質タイルのビットレート、ｒｅｓ_Ｈは高画質タイルの解像度、ｆｒはフレームレートを表し、ｑ_１～ｑ_３、ａ_１～ａ_３は予め定められた定数である。ｑｕａｎｔ_Ｈは、ｂｒ_Ｈ、ｂｐｐ_Ｈから求める代わりに、高画質タイルの量子化パラメータＱＰ_Ｈを用いてもよい。

In the above formula, br _H is the bit rate of the high-quality tile, res _H is the resolution of the high-quality tile, fr is the frame rate, and q ₁ to q ₃ and a ₁ to a ₃ are predetermined constants. . Instead of finding quant _H from br _H and bpp _H , the quantization parameter QP _H of the high-quality tile may be used.

Note that the "predetermined constant" or "predetermined coefficient" shown in this specification may be determined by experiment, for example, or may be determined in accordance with ITU-T Recommendation P. If the value specified in 1203 is applicable, that value may be used.

The VR video quality estimation unit 15 may calculate the quality estimated value VQ _H of the high-quality tile using the above-mentioned MOS _qH as follows.

VQ _H = MOS from R (100-D _H )
D _H =max(min(D _qH +D _uH +D _t ,100),0)
D _uH = max(min(u ₁・log ₁₀ (u ₂・scaleFactor _H −1)+1), 100), 0)
D _qH = max(min(100-RfromMOS(MOS _qH ), 100), 0)

ここで、Ｄ_Ｈは高画質タイルによる品質劣化量を示している。また、ｄｉｓＲｅｓ、ｃｏｄＲｅｓ_Ｈはそれぞれディスプレイの解像度、高画質タイルの符号化解像度を表している。

Here, _DH indicates the amount of quality deterioration due to high-quality tiles. Further, disRes and codRes _H represent the resolution of the display and the encoding resolution of the high-quality tile, respectively.

Furthermore, RfromMOS and MOSfromR represent functions for converting the user experience quality MOS and the psychological value R. Furthermore, in the above formula, q ₁ to q ₃ , u ₁ to u ₃ , and t ₁ to t ₃ are predetermined coefficients.

Further, the VR video quality estimating unit 15 may calculate the quality estimated value VQ _H of the high-quality tile as follows.

ただし、ｖ_１～ｖ_７は予め定めた係数である。上記数式で用いられたＸ、Ｙと同様のＸ、Ｙを用い、ｂｒ_Ｈの代わりに高画質タイルの量子化パラメータＱＰ_Ｈを用いて、

However, v ₁ to v ₇ are predetermined coefficients. Using the same X and Y as those used in the above formula, and using the quantization parameter QP _H of the high-quality tile instead of br _H ,

として算出してもよい。

It may be calculated as

Similarly, the VR video quality estimation unit 15 can calculate the quality VQ _L and quality deterioration D _L of the low image quality tile using the parameters of the low image quality tile. Note that when calculating the quality of a low-image quality tile, the same coefficient as that for a high-image quality tile may be used, or a different value may be used.

The overall quality estimation unit 16 uses the quality of each of the high-quality tiles and low-quality tiles and the switching delay, which is the time required to switch from the display state of the low-quality tiles to the display state of the high-quality tiles, to estimate the high-quality tiles. By calculating the quality contribution ω of each of the image quality tile and the low image quality tile, the overall quality value VQ of the VR video is estimated.

For example, the contribution calculation unit 17 can calculate the quality contribution ω using the following formula.

ここで、ｄｅｌａｙは切り替え遅延、ｒｅｓ_Ｈは高画質タイルの解像度、ｒｅｓは映像全体の解像度、ｄ_１～ｄ_４は予め定めた係数である。ωは高画質タイルと低画質タイルの品質劣化を分配する割合を表す値であり、タイルの切り替え遅延及び解像度を利用して求めているが、特に、ｄ_３＝０と定めた時は、解像度の影響を考慮しない場合となる。ωは同様の変数、係数を用いて、積の形で次のように算出してもよい。なお、Ｓ_Ｌは、画像全体の中で低画質タイル（低画質領域）の占める面積に相当する。低画質タイルを表示しない場合、Ｓ_Ｌは０になる。

Here, delay is a switching delay, res _H is the resolution of a high-quality tile, res is the resolution of the entire video, and d ₁ to d ₄ are predetermined coefficients. ω is a value that represents the proportion of quality deterioration distributed between high-quality tiles and low-quality tiles, and is determined using the tile switching delay and resolution. In particular, when d ₃ = 0, the resolution This is a case where the influence of ω may be calculated in the form of a product as follows using similar variables and coefficients. Note that S _L corresponds to the area occupied by low image quality tiles (low image quality areas) in the entire image. If low-quality tiles are not displayed, S _L is 0.

総合品質推定部１６は、品質寄与度ω、高画質タイルの品質ＶＱ_Ｈ、及び低画質タイルの品質ＶＱ_Ｌを利用し、以下のように総合品質ＶＱを算出できる。

The overall quality estimating unit 16 can calculate the overall quality VQ as follows using the quality contribution ω, the quality VQ _H of the high image quality tile, and the quality VQ _L of the low image quality tile.

VQ=ω・VQ _H +(1−ω)・VQ _L
Further, the total quality estimating unit 16 may calculate VQ using the following formula.

上記では高画質タイル、低画質タイルの品質ＶＱ_Ｈ、ＶＱ_Ｌを利用し総合品質を算出したが、総合品質推定部１６は、高画質タイル及び低画質タイルの品質劣化Ｄ_Ｈ、Ｄ_Ｌを利用し、以下の数式を用いてＶＱを算出してもよい。

In the above, the overall quality was calculated using the quality VQ _H and VQ _L of the high image quality tile and the low image quality tile, but the overall quality estimation unit 16 uses the quality deterioration DH _{and DL} of the high image quality tile and the low image quality tile. However, VQ may be calculated using the following formula.

D _HL =ω・D _H +(1-ω)・D _L
VQ=MOSfromR(100-max(min(D _HL , 100), 0))
Further, the total quality estimating unit 16 may calculate the _DHL as follows.

上記の数式において、ｄ_１～ｄ_１０は予め定めた係数である。

In the above formula, d ₁ to d ₁₀ are predetermined coefficients.

In the above example, videos of both high-quality tiles and low-quality tiles are distributed as VR video, but if low-quality tiles are not distributed, the encoding parameters after extraction of the low-quality tiles of the VR video are all 0. may or may not be output. In this case, if the high-quality tiles do not cover the VR video 360°, the video may not be displayed due to eye movement.

In this case, the quality contribution degree ω may be calculated using the same formula as above, or may be calculated as follows by adding a coefficient _d5 that takes into account the influence of not delivering the tile.

また、上記の低画質タイルを配信しない場合において、高画質タイルがＶＲ映像を３６０°カバーする場合は、視線移動によって品質は変化せず、一様型のＶＲ映像配信に対応する。この場合は、配信設定による切り替え遅延は０としてもよいし、出力しなくてもよい。この場合、品質寄与度ωを１とする又は単に
ＶＱ＝ＶＱ_Ｈ
とすることで符号化パラメータを算出できる。

Furthermore, in the case where the above-mentioned low-quality tiles are not distributed, if the high-quality tiles cover the VR video 360°, the quality does not change depending on the movement of the line of sight, and uniform VR video distribution is supported. In this case, the switching delay due to distribution settings may be set to 0, or may not be output. In this case, the quality contribution degree ω is set to 1 or simply VQ=VQ _H
By doing so, the encoding parameters can be calculated.

The coefficients used in the above formula may be predetermined fixed coefficients, or may be coefficients that change depending on the codec, profile settings, etc.

Furthermore, the mathematical formulas used in this specification are merely examples, and the quality may be estimated using a mathematical formula other than the mathematical formulas used in this specification.

<Processing procedure>
The processing procedure executed by the VR video encoding parameter calculation device 10 will be described below. FIG. 9 is a flowchart of the VR video encoding parameter calculation method in the first embodiment.

In S101, the VR video quality estimating unit 15 calculates the quality of the high image quality tile and the low image quality tile based on the input encoding parameters, and applies the calculated quality of the high image quality tile and the low image quality tile to the overall quality estimating unit. Output to 16.

In S102, the contribution calculation unit 17 calculates the quality contribution based on the input distribution settings.

In S103, the total quality estimation unit 16 calculates the total quality of the VR video based on the input encoding parameters, the quality of the high-quality tiles and the low-quality tiles, and the calculated quality contribution degree. The calculated overall quality is output to the VR video parameter extraction section 18.

In S104, the VR video parameter extraction unit 18 extracts and outputs encoding parameters of high-quality tiles and low-quality tiles that are close to the target quality based on the input target quality and overall quality.

(Second embodiment)
Next, a second embodiment of the present invention will be described. In the second embodiment, instead of inputting video encoding setting information from the encoding condition setting section 11, the distributed VR video is sent to the VR video encoding parameter calculation device 10 from the VR video input section 20. I am planning to input it.

FIG. 10 is a configuration diagram of a VR video encoding parameter calculation device 10 according to the second embodiment of the present invention. Unlike the first embodiment, the distributed VR video is input to the VR video encoding parameter calculation device 10 instead of the encoding parameters of the VR video.

The input VR video is encoded in a VR video encoding unit 19 in the VR video quality estimating unit 15 at a plurality of quality levels (for example, three or more). The VR video encoding unit 19 extracts quantization parameters and bit rates for each encoding in multiple stages of encoding.

In the formula for calculating quant _H in the first embodiment, a ₁ to a ₃ are predetermined coefficients, but in the second embodiment, the VR video encoding unit 19 uses the above-mentioned multiple steps. Using the quantization parameter QP _H of the video extracted by encoding and the bit rate br _H , the least squares method is used to calculate the coefficients of the following formula so that quant _H approaches QP _H in the above encoding. Optimization of a ₁ to a ₃ is performed on the input VR video to determine a ₁ to a ₃ .

quant _H =a ₁ +a ₂・ln(a ₃ +ln(br _H )+ln(br _H・bpp _H ))
Further, the VR video encoding unit 19 may perform optimization using the following formula.

低画質タイルを配信せずに高画質タイルがＶＲ映像が３６０°カバーしない場合や、高画質タイルがＶＲ映像を３６０°カバーする一様型のＶＲ映像配信を行う場合においても、上記と同様の計算により係数ａ_１～ａ_３の最適化を行うことができる。

The same method as above applies even when high-quality tiles do not cover 360° of VR video without distributing low-quality tiles, or when performing uniform VR video distribution where high-quality tiles cover 360° of VR video. The coefficients a ₁ to a ₃ can be optimized by calculation.

Through the above calculation, it is possible to determine the coefficients of the mathematical formula for each video. Therefore, characteristics such as ease of encoding for each content can be taken into consideration in the above calculation, and the overall quality can be calculated based on the characteristics of the content.

Quality estimation in the VR video quality estimation unit 15 other than the above can be performed by calculations similar to those in the first embodiment.

Furthermore, since the overall quality estimating section 16, the contribution degree calculating section 17, and the VR video parameter extracting section 18 perform the same processing as in the first embodiment, their explanations will be omitted.

The processing procedure executed by the VR video encoding parameter calculation device 10 according to the second embodiment of the present invention will be described below. FIG. 11 is a flowchart of a VR video encoding parameter calculation method according to the second embodiment of the present invention.

In S201, the VR video quality estimating unit 15 determines coefficients as described above based on the input encoding parameters and VR video, and calculates the quality of high-quality tiles and low-quality tiles. The VR video quality estimating unit 15 outputs the calculated quality of the high image quality tile and the low image quality tile to the total quality estimating unit 16.

In S202, the contribution calculation unit 17 calculates the quality contribution based on the input distribution settings.

In S203, the total quality estimating unit 16 calculates the total quality of the VR video based on the input encoding parameters, the quality of the high-quality tiles and low-quality tiles, and the calculated quality contribution degree. The calculated overall quality is output to the VR video parameter extraction section 18.

In S204, the VR video parameter extraction unit 18 extracts encoding parameters of high-quality tiles and low-quality tiles that are close to the target quality based on the input target quality and overall quality.

(Hardware configuration example)
The VR video encoding parameter calculation device 10 may be realized by hardware using logic circuits that realize the functions of each part shown in FIGS. 1 and 10, or a general-purpose computer may be used to This may be realized by executing a program that describes the processing contents described in the embodiment. Note that this "computer" may be a virtual machine. When using a virtual machine, the "hardware" described here is virtual hardware.

When using a computer, the VR video encoding parameter calculation device 10 uses hardware resources such as a CPU and memory built into the computer to program a program corresponding to the processing performed by the VR video encoding parameter calculation device 10. This can be achieved by executing the following. The above program can be recorded on a computer-readable recording medium (such as a portable memory) and can be stored or distributed. It is also possible to provide the above program through a network such as the Internet or e-mail.

FIG. 2 is a diagram showing an example of the hardware configuration of the computer. The computer in FIG. 12 includes a drive device 1000, an auxiliary storage device 1002, a memory device 1003, a CPU 1004, an interface device 1005, a display device 1006, an input device 1007, and the like, which are interconnected by a bus B.

A program for realizing processing by the computer is provided, for example, by a recording medium 1001 such as a CD-ROM or a memory card. When the recording medium 1001 storing the program is set in the drive device 1000, the program is installed from the recording medium 1001 to the auxiliary storage device 1002 via the drive device 1000. However, the program does not necessarily need to be installed from the recording medium 1001, and may be downloaded from another computer via a network. The auxiliary storage device 1002 stores installed programs as well as necessary files, data, and the like.

The memory device 1003 reads the program from the auxiliary storage device 1002 and stores it when there is an instruction to start the program. The CPU 1004 implements functions related to the VR video encoding parameter calculation device 10 according to the program stored in the memory device 1003. The interface device 1005 is used as an interface for connecting to a network. A display device 1006 displays a GUI (Graphical User Interface) or the like based on a program. The input device 1007 is composed of a keyboard, a mouse, buttons, a touch panel, or the like, and is used to input various operation instructions.

(Effects of embodiments, etc.)
As explained above, according to the present embodiment, the overall quality of the VR video is derived by inputting the VR video or the encoding settings of the VR video, the distribution settings, and the target quality of the VR video. It is possible to calculate encoding parameters (bit rate, resolution, frame rate, switching delay, quantization parameter, etc.) for tile-based VR video that are optimal for quality.

(Summary of embodiments)
This specification describes at least a VR video encoding parameter calculation device, a VR video encoding parameter calculation method, and a program described in each section below.
(Section 1)
a quality estimator that calculates an overall quality estimate of the VR video for each of the plurality of encoding parameters;
A VR video encoding parameter calculation device, comprising: a VR video parameter extraction unit that extracts an encoding parameter corresponding to an overall quality estimate that satisfies a given target quality from among the plurality of encoding parameters.
(Section 2)
The VR video encoding parameter calculation device according to claim 1, wherein the quality estimation unit calculates the overall quality estimate based on information on video encoding settings or the VR video.
(Section 3)
The quality estimator includes:
a VR video quality estimator that calculates a quality estimate of a high-quality area in the VR video and a quality estimate of a low-quality area in the VR video;
The VR video according to claim 1 or 2, further comprising: a total quality estimator that calculates the total quality estimate based on the quality estimate of the high image quality area and the quality estimate of the low image quality area. Encoding parameter calculation device.
(Section 4)
The overall quality estimating unit is
The quality of the high image quality area and the quality of the low image quality area based on at least one of the parameters related to the time taken to switch from the low image quality area display state to the high image quality area display state and information regarding the area occupied by the low image quality area. Calculate the contribution of
The VR video encoding parameter calculation device according to claim 3, wherein the overall quality estimate is calculated based on the contribution degree, the quality estimate of the high image quality area, and the quality estimate of the low image quality area.
(Section 5)
A VR video encoding parameter calculation method executed by a VR video encoding parameter calculation device, comprising:
a quality estimation step of calculating an overall quality estimate of the VR video for each of the plurality of encoding parameters;
A VR video encoding parameter calculation method, comprising: a VR video parameter extraction step of extracting an encoding parameter corresponding to an overall quality estimate that satisfies a given target quality from among the plurality of encoding parameters.
(Section 6)
A program for causing a computer to function as each part of the VR video encoding parameter calculation device according to any one of items 1 to 4.

Although the present embodiment has been described above, the present invention is not limited to such specific embodiment, and various modifications and changes can be made within the scope of the gist of the present invention as described in the claims. It is possible.

10 VR video encoding parameter calculation device 11 Encoding condition setting section 12 Distribution setting input section 13 Target quality input section 14 Parameter setting value candidate group input section 15 VR video quality estimation section 16 Total quality estimation section 17 Contribution degree calculation section 18 VR Video parameter extraction section 19 VR video encoding section 20 VR video input section 1000 Drive device 1001 Recording medium 1002 Auxiliary storage device 1003 Memory device 1004 CPU
1005 Interface device 1006 Display device 1007 Input device

Claims (4)

a quality estimator that calculates an overall quality estimate of the VR video for each of the plurality of encoding parameters;
A VR video encoding parameter calculation device comprising: a VR video parameter extraction unit that extracts an encoding parameter corresponding to an overall quality estimate that satisfies a given target quality from among the plurality of encoding parameters,
The quality estimator includes:
a VR video quality estimator that calculates a quality estimate of a high-quality area in the VR video and a quality estimate of a low-quality area in the VR video;
an overall quality estimation unit that calculates the overall quality estimate based on the quality estimate of the high image quality area and the quality estimate of the low image quality area,
The overall quality estimating unit is
The quality of the high image quality area and the quality of the low image quality area based on at least one of the parameters related to the time taken to switch from the low image quality area display state to the high image quality area display state and information regarding the area occupied by the low image quality area. Calculate the contribution of
The overall quality estimate is calculated based on the contribution, the quality estimate of the high image quality area, and the quality estimate of the low image quality area.
VR video encoding parameter calculation device . The VR video encoding parameter calculation device according to claim 1, wherein the quality estimator calculates the overall quality estimate based on information on video encoding settings or the VR video. A VR video encoding parameter calculation method executed by a VR video encoding parameter calculation device, comprising:
a quality estimation step of calculating an overall quality estimate of the VR video for each of the plurality of encoding parameters;
a VR video parameter extraction step of extracting a coding parameter corresponding to an overall quality estimate that satisfies a given target quality from among the plurality of coding parameters;
The quality estimation step includes:
a VR video quality estimation step of calculating a quality estimated value of a high quality area in the VR video and a quality estimated value of a low quality area in the VR video;
a total quality estimation step of calculating the total quality estimate based on the quality estimate of the high image quality area and the quality estimate of the low image quality area,
In the overall quality estimation step, the VR video encoding parameter calculation device:
The quality of the high image quality area and the quality of the low image quality area based on at least one of the parameters related to the time taken to switch from the low image quality area display state to the high image quality area display state and information regarding the area occupied by the low image quality area. Calculate the contribution of
The overall quality estimate is calculated based on the contribution, the quality estimate of the high image quality area, and the quality estimate of the low image quality area.
VR video encoding parameter calculation method. A program for causing a computer to function as each part of the VR video encoding parameter calculation device according to claim 1 or 2 .

Images