JP5600649B2 - Video processing apparatus, method, and program - Google Patents

Description

  The present invention relates to a video processing apparatus, method, and program, and in particular, by reducing the resolution of video content while maintaining constant video quality for input video content (hereinafter referred to as “input video content”). The present invention relates to a video processing apparatus, method, and program for reducing the data amount of video content.

  Digitalization of terrestrial television broadcasting has created an environment in which general homes can view full-HD video content. Along with this, provision of full-HD video content has started in video distribution services and package media. High-resolution video content is appealing to users because of the high image quality, but the amount of data increases, so storage costs for storing video data and network equipment costs for transmission become a heavy burden. .

  Here, the relationship between resolution and video quality will be described. FIG. 1 shows the difference in the influence on the quality given by the resolution conversion (Full-HD 1920 × 1080 → Half-HD 960 × 1080) in each of the seven types of video content. In FIG. 1, the video contents 5 to 7 increase in the degree of video quality deterioration as the resolution decreases. That is, while the video quality is reduced due to the decrease in resolution, the difference in resolution hardly affects the video quality in the video contents 1 to 4. As shown in FIG. 1, depending on the video content, there is a threshold at which the reduction in the resolution of the video content does not affect the reduction in the video quality. Therefore, the minimum necessary amount that does not affect the reduction in the video quality for each video content. By setting the resolution to, the data amount of the video content can be reduced.

  Conventionally, Non-Patent Document 1 describes a technique in which a video producer gives priorities and intensity of motion as meta information in advance for each scene, and changes the resolution based on the information.

Yoshitoshi Murata, Akihisa Masuda, Ken Ota, Susumu Ishihara, Tadanori Mizuno, "Content Oriented Spatio-Temporal Resolution Control Method for Mobile Multimedia Streaming Service", Transactions of Information Processing Society of Japan, Vol.43, No. 12, pp. 3838-3847, 2002 (12).

  However, a technique (Non-Patent Document 1) in which a video producer assigns priorities and intensity of motion as meta information in advance for each scene and changes the video quality in consideration of the information (non-patent document 1) There is a problem that it is necessary for the producer to cope with it, and human and time costs increase.

  The present invention has been made in view of the above, and by using content feature amounts obtained from video content data, video quality (video quality perceived by humans) can be obtained without incurring human and time costs. The degree of degradation) is to satisfy the specified condition, that is, to maintain the video quality within the specified threshold, to reduce the amount of data by converting the resolution to the resolution of the video content, and to the specified video quality condition It is an object of the present invention to provide a video processing apparatus, method, and program for converting to a video content resolution according to the above.

In order to solve the above problems, the present invention reduces the input resolution of the input video data to the resolution that satisfies the specified video quality condition for the input video data (hereinafter referred to as “input video data”). A video processing device for conversion,
Optimum video parameter derivation means for calculating the optimum resolution that is the minimum resolution among the resolutions that satisfy the specified video quality conditions for the input video data;
Video optimization processing means for converting the resolution of the input video data and outputting optimized video data based on the optimal resolution calculated by the optimal video parameter derivation means;
With
The optimum video parameter derivation means includes:
Based on the input image data, an edge amount calculation means for calculating an edge amount,
For each combination of resolutions before and after resolution conversion, a model formula representing the relationship between the edge amount and the video quality at the time of resolution conversion is set, and the input resolution is set as the resolution before resolution conversion, and the resolution conversion is performed. Based on the model formula set for each combination of resolution before and after, the degree of degradation of video quality with respect to the edge amount is calculated, and among the combinations of resolutions that satisfy the specified video quality condition, An optimal resolution calculating means for calculating a resolution at which the resolution after resolution conversion is minimized as the optimal resolution;
including.

  According to the present invention, it is possible to reduce the data amount of video content by lowering the resolution of the video content while keeping the video quality of the input video content constant. Therefore, it is possible to reduce storage capacity and communication traffic without affecting the video quality by reducing the video quality beyond a specified threshold value, leading to cost reduction.

This is the degree of video quality degradation at the time of resolution conversion for each video content. Edge video data. This is the relationship between the edge amount of video content and the degree of video quality degradation during resolution conversion. It is a block diagram of the video processing apparatus in one embodiment of this invention. 6 is a model formula showing the relationship between the edge amount and the degree of video quality degradation during resolution conversion in an embodiment of the present invention. It is a block diagram of the optimal resolution calculation part in one embodiment of this invention. It is a flowchart of operation | movement of the video processing apparatus in one embodiment of this invention.

  Embodiments of the present invention will be described below with reference to the drawings.

  The present invention provides an apparatus and a method for converting video content resolution according to specified video quality conditions. Specifically, the present invention reduces the video content resolution (before conversion). Calculate the resolution that will be the minimum resolution (optimal resolution) of the resolutions after conversion so that the degree of degradation in video quality due to resolution conversion will be within the pre-specified threshold, and the actual video at that resolution. Performs processing to convert the resolution of the content.

  The present invention is based on "a model formula for the degree of degradation in video quality before and after resolution conversion", which is defined based on the experimental results on the effect of degradation in resolution on the "degree of degradation in video quality perceived by humans" A feature is that the optimum resolution is calculated.

  The solving means of the present invention will be described. In the present invention, by using the edge amount that is the feature amount of the content obtained from the pixel information of the input video content, the minimum resolution (optimum resolution) of the video content that can satisfy the specified video quality condition. ) And converting the resolution of the input video content to the optimum resolution, the data amount of the video content can be converted in a state where the specified constant video quality is maintained.

  Satisfying the designated video quality condition means keeping the “degree of degradation in video quality” within a designated threshold.

  As a cause of deterioration in video quality due to resolution conversion in video content, the edge (contour line) of the video content is lost or distorted due to resolution conversion. For this reason, it is considered that a video content including more edges has a higher degree of deterioration in video quality at the time of resolution conversion. Therefore, a technique for calculating an optimum resolution from the amount of edges is provided.

  The amount of edge is, for example, an average value of pixel values for each frame with respect to edge video data (FIG. 2) obtained by applying a known edge extraction filter such as a Sobel filter, Laplacian filter, Previt filter, and Canny edge filter. Alternatively, it is calculated as an average value or a maximum value of standard deviations of pixel values for each frame in all frames.

  FIG. 3 shows the relationship between the edge amount of video content and the degree of video quality degradation at the time of resolution conversion. As shown in FIG. 3, as the edge amount increases, there is a relationship in which the degree of deterioration in video quality during resolution conversion increases. This relationship is modeled for each combination of resolutions before and after resolution conversion (for example, 1920 × 1080 to 1440 × 1080, 1920 × 1080 to 960 × 1080, 1920 × 1080 to 1280 × 720, etc.). Using this model, for each combination of the resolution of the input video content and the resolution at the time of resolution conversion, the edge amount and the degree of video quality degradation at the time of resolution conversion are calculated, and the degree of video quality degradation is specified It is possible to calculate the minimum resolution that keeps within the set threshold and maintains a constant video quality as the optimum resolution.

  FIG. 4 shows the configuration of the video processing apparatus in one embodiment of the present invention.

  The video processing apparatus shown in FIG. 1 includes an optimal video parameter derivation unit 1 and a video optimization processing unit 2. Video content data (hereinafter referred to as “video data”) is input, and video data obtained by converting the resolution of the video content to the optimal resolution (hereinafter referred to as “optimized video data”) is output. The video data input to the video processing apparatus is input to the optimized video parameter deriving unit 1 and the video optimization processing unit 2.

  The “video quality” is the degree of degradation of video quality, and the “designated video quality condition” is to keep the degree of degradation of video quality due to resolution conversion below a specified threshold.

  The optimized video parameter derivation unit 1 includes an edge amount calculation unit 11 and an optimal resolution calculation unit 12. The optimized video parameter derivation unit 1 receives the video data, and the edge amount calculation unit 11 calculates the edge amount and the input resolution based on the input video data, and outputs it as an input to the optimal resolution calculation unit 12. The optimum resolution calculation unit 12 calculates the optimum resolution based on the edge amount and the input resolution calculated by the edge amount calculation unit 11, outputs the input resolution and the optimization resolution, and outputs the image optimization processing unit. 2 is output as an input to 2.

  In the edge amount calculation unit 11, for example, a Sobel filter, a Laplacian filter, a Previt filter, a Canny edge filter (reference document 1 ““ A Computational Approach to Edge Detection ”is applied to each frame of the luminance layer included in the input video data. , "by JF Canny, IEEE Transactions on Pattern Analysis and Machine Intelligence, 8, pp.769-798, 1986"), and so on, to generate edge video data. In this embodiment, edge video data E (w, h, f) is calculated using a Sobel filter as in the following equation.

ここでy(w,h,f)はｆ番目のフレームの画素位置（w,h）の輝度値を示す。

Here, y (w, h, f) represents the luminance value at the pixel position (w, h) of the f-th frame.

  Based on the standard deviation (or average value) of the pixel values of each frame of the generated edge video data, the average value (or maximum value) of all frames is calculated as the edge amount. In this embodiment, the edge amount e is calculated using an average value of standard deviations as in the following equation.

ここで、上記式中の関数std_space[ ]は、フレーム毎にフレーム内の画素値の標準偏差を出力する関数、Nはフレームの総数を示す。

Here, the function std _space [] in the above equation is a function for outputting the standard deviation of the pixel values in the frame for each frame, and N indicates the total number of frames.

  The edge amount calculation unit 11 inputs the calculated edge amount e to the optimum resolution calculation unit 12, and based on the metadata included in the input video data, the resolution of the input video data (hereinafter referred to as “input”). The input resolution is input to the optimum resolution calculation unit 12 with reference to “resolution”. The metadata has a data structure defined for each file format such as AVI or mp4. In the case of video data that does not include metadata, it is necessary to accept no input, to use a default input resolution, or to set manually.

  The optimum resolution calculation unit 12 is a combination of resolutions before and after resolution conversion (for example, 1920 × 1080 and 1440 × 1080, 1920 × 080 and 960 × 1080, 1920 × 1080 and 720 × 1080, 1920 × 1080 and 720 × 480). Etc.), a model formula and a coefficient of the model formula (model coefficient) representing the relationship between the edge amount and the degree of deterioration of the video quality at the time of resolution conversion as shown in FIG. 5 are provided. In the present embodiment, a unified model formula is used, but the model formula may be changed for each combination of resolutions. Model coefficients are measured quality values obtained by experiments in advance, or estimated quality values obtained from video data using a video quality objective evaluation model such as ITU-T Recommendation J.247. It is obtained for each combination of resolutions by the least square method. The model formulas and model coefficients are stored in the storage unit 122 as shown in FIG. The model formula and model coefficient may be changed after the initial setting.

  A method for calculating the optimum resolution in the optimum resolution calculator 12 will be described.

  Based on the input edge amount and input resolution, the video quality corresponding to the input edge amount is reduced for each pre-defined model formula corresponding to the combination of the input resolution and resolution after conversion. Calculate the resolution, and select the resolution that minimizes the resolution after conversion from among the combination of the input resolution and the resolution after resolution conversion where the degree of video quality degradation at the time of resolution conversion is below a specified threshold. The optimum resolution is calculated and the input resolution and the input resolution that have been input are output as an input to the video optimization processing unit 2. The threshold is accumulated in the storage unit 122 as shown in FIG. 6 and is referred to by the arithmetic processing unit. If the threshold value is set low, the deterioration in quality is reduced, but the effect of reducing the data amount is also reduced, and if the threshold value is set high, the reduction in quality is increased but the effect of reducing the data amount is increased. The setting of the threshold value is determined based on an operation policy of a service provider using an apparatus incorporating the present invention. Note that the threshold value may be changed after the initial setting. In the present embodiment, the following model formula is used.

ここでdVQは解像度の変換時の映像品質の低下の度合い、ｅはエッジ量を表す。また、ｖ₁，ｖ₂，ｖ₃は係数であり解像度変換前後の解像度の組み合わせ毎に、事前に主観品質評価実験等で得られる映像品質と、入力された映像データから得られるエッジ量を用いて、最小二乗法などによりモデル式を最適化する必要がある。本モデル式は、エッジ量が低下するとdVQが0に収束し、エッジ量が増加するとdVQがｖ₁に収束する特性を持つ。

Here, dVQ represents the degree of degradation of video quality at the time of resolution conversion, and e represents the edge amount. Further, v ₁ , v ₂ , and v ₃ are coefficients, and for each combination of resolutions before and after resolution conversion, the video quality obtained by a subjective quality evaluation experiment or the like and the edge amount obtained from input video data are used. Therefore, it is necessary to optimize the model formula by the least square method. This model formula has a characteristic that when the edge amount decreases, dVQ converges to 0, and when the edge amount increases, dVQ converges to v ₁ .

  The video optimization processing unit 2 receives the optimal resolution and input resolution output from the optimal video parameter deriving unit 1 and the video data input to the video processing apparatus, and the input resolution is larger than the input optimal resolution. Uses, for example, a known image reduction algorithm such as a nearest neighbor method, a bilinear method, a bicubic method, or an average pixel method to convert the resolution of the input video data to the optimal resolution, thereby generating optimized video data. If the input optimum resolution is smaller than the input resolution, the input video data is output as optimized video data using an image reduction algorithm.

  Next, the processing flow of this embodiment will be described using the flowchart of FIG.

  First, the edge amount calculation unit 11 calculates an edge amount and input resolution from the input video data, and inputs the calculated edge amount and input resolution to the optimum resolution calculation unit 12 (step 1).

  Next, the optimal resolution calculation unit 12 calculates the optimal resolution from the input edge amount and the input resolution, and inputs the calculated optimal resolution and the input resolution to the video optimization processing unit 2 (step 2).

  Then, the video optimization processing unit 2 generates optimized video data from the input video data, the optimal resolution, and the input resolution (step 3).

  As described above, the video quality can be maintained within the specified threshold value by converting the resolution of the video content in accordance with the specified video quality condition (the degree of deterioration of the video quality). As a result, it is possible to realize a process of changing the resolution in consideration of video quality without incurring human / time costs.

  It should be noted that each operation of the components of the video processing apparatus shown in FIG. 4 can be constructed as a program, installed in a computer used as the video processing apparatus and executed, or distributed via a network. is there.

  The present invention is not limited to the above-described embodiments, and various modifications and applications are possible within the scope of the claims.

DESCRIPTION OF SYMBOLS 1 Optimal video parameter derivation | leading-out part 2 Video | video optimization processing part 11 Edge amount calculation part 12 Optimal resolution calculation part 121 Operation processing part 122 Storage part

Claims (5)

A video processing device that converts input resolution of input video data to a resolution that satisfies a specified video quality condition for input video data (hereinafter referred to as “input video data”),
Optimum video parameter derivation means for calculating the optimum resolution that is the minimum resolution among the resolutions that satisfy the specified video quality conditions for the input video data;
Video optimization processing means for converting the resolution of the input video data and outputting optimized video data based on the optimal resolution calculated by the optimal video parameter derivation means;
With
The optimum video parameter derivation means includes:
Based on the input image data, an edge amount calculation means for calculating an edge amount,
For each combination of resolutions before and after resolution conversion, a model formula representing the relationship between the edge amount and the video quality at the time of resolution conversion is set, and the input resolution is set as the resolution before resolution conversion, and the resolution conversion is performed. Based on the model formula set for each combination of resolution before and after, the degree of degradation of video quality with respect to the edge amount is calculated, and among the combinations of resolutions that satisfy the specified video quality condition, An optimal resolution calculating means for calculating a resolution at which the resolution after resolution conversion is minimized as the optimal resolution;
A video processing apparatus comprising: The edge amount calculating means includes
Means for applying an edge extraction filter to each frame of the luminance layer of the input video data to convert it to edge video data;
Means for calculating a feature value of a pixel value for each frame of the edge video data, and calculating as an edge amount of all frames based on the calculated feature value of the pixel value for each frame;
The video processing apparatus according to claim 1, further comprising: A video processing method for converting input resolution of input video data to resolution satisfying a specified video quality condition for input video data (hereinafter referred to as “input video data”),
Optimum video parameter derivation means, for the input video data, an optimal video parameter derivation step for calculating an optimal resolution that is a minimum resolution among resolutions that satisfy a specified video quality condition;
A video optimization processing step, wherein the video optimization processing means converts the resolution of the input video data based on the optimal resolution calculated in the optimal video parameter derivation step, and outputs optimized video data;
With
In the optimum video parameter derivation step,
Based on the input image data, an edge amount calculating step of calculating an edge amount,
For each combination of resolutions before and after resolution conversion, a model formula representing the relationship between the edge amount and the video quality at the time of resolution conversion is set, and the input resolution is set as the resolution before resolution conversion, and the resolution conversion is performed. Based on the model formula set for each combination of resolution before and after, the degree of degradation of video quality with respect to the edge amount is calculated, and among the combinations of resolutions that satisfy the specified video quality condition, An image processing method comprising: an optimum resolution calculating step of calculating, as the optimum resolution, a resolution at which the resolution after resolution conversion is minimized. In the edge amount calculating step,
Applying an edge extraction filter to each frame of the luminance layer of the input video data to convert it to edge video data;
Calculating a feature value of a pixel value for each frame of the edge video data, and calculating as an edge amount of all frames based on the calculated feature value of the pixel value for each frame;
The video processing method according to claim 3, further comprising: Computer
A video processing program for functioning as each unit of the video processing device according to claim 1.

Images