JP2019092105A - Video encoding apparatus, video encoding method, and program - Google Patents

Abstract

To provide a video encoding apparatus, a video encoding method, and a program that can realize miniaturization of the apparatus itself while achieving high image quality of a video to be broadcast.SOLUTION: A video encoding apparatus 10 includes a pre-processing unit 11 that calculates a first quantization parameter of a frame on the basis of the complexity of a frame for each of a plurality of frames, divides the frame into two or more blocks, and calculates a second quantization parameter by using first quantization parameter for each of the blocks, and a video encoding unit 12 that performs quantization on each of the plurality of frames for each of the blocks making up the frame by using the second quantization parameter calculated for the block, and further encodes the frame in which the block is quantized.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a video coding apparatus, a video coding method, and a program.

  In recent years, the purpose is to realize terrestrial broadcasting of high-resolution 4K resolution (hereinafter referred to as "4K") / 8K resolution (hereinafter referred to as "8K") rather than digital terrestrial television broadcast, and a broadcast network Reorganization is taking place. In addition, in order to effectively utilize limited radio wave resources, it has been considered that the broadcast band of the present broadcast network can be suppressed, and high quality 4K / 8K terrestrial broadcast can not be realized in the remaining band. From such a background, there is a demand for the same image quality or high image quality as the current terrestrial broadcast video even in a frequency band narrower than the present. Therefore, a video encoding apparatus with a higher video compression rate is required.

  As a technique for solving such a problem, a multi-pass coding process has been proposed in which the entire video sequence is pre-analyzed and encoded based on the pre-analysis in order to increase the video compression rate. For example, Patent Documents 1 to 4 disclose video coding apparatuses that execute multi-pass coding processing.

  Specifically, the video encoding device disclosed in Patent Document 1 generates a statistical value by pre-analyzing only a portion overlapping in window units in the entire sequence of the input image as the first pass processing. . Furthermore, as the second pass processing, the video encoding apparatus disclosed in Patent Document 1 adjusts the encoding parameter of each input image as appropriate based on the statistical value generated in the first pass, and adjusts the code. The input image pre-analyzed in the first pass is encoded based on the quantization parameter.

  Further, the video encoding device disclosed in Patent Document 2 analyzes the sequence of the input image in advance as the first pass processing, generates the first analysis information, and generates the first analysis information based on the generated first analysis information. And perform preprocessing on the input image. Examples of preprocessing include frequency band limiting processing, image size conversion processing, and frame rate conversion processing.

  Furthermore, as processing for the second pass, the video encoding device disclosed in Patent Document 2 performs pre-analysis again on the input image that has been preprocessed in the first pass, to obtain second analysis information. The target code amount of the input image preprocessed in the first pass is set using the first analysis information and the generated second analysis information.

  Furthermore, as processing of the third pass, the video encoding device disclosed in Patent Document 2 encodes an input image on which preprocessing of the first pass has been performed. At that time, the video encoding apparatus disclosed in Patent Document 2 performs encoding such that the encoding amount assigned at the time of encoding reaches the target code amount set in the second pass.

  In addition, the video encoding device disclosed in Patent Document 3 analyzes the sequence of the input image in advance as the first pass processing, and based on this prior analysis, appropriately allocates code amount while performing buffer corruption avoidance processing. Set Furthermore, as processing for the second pass, the video encoding device disclosed in Patent Document 3 encodes an input image based on the allocated code amount.

In addition, the video encoding device disclosed in Patent Document 4 analyzes the sequence of the input image in advance as processing of the first pass, and encodes the coding difficulty level that reflects the conspicuousness (flatness) of the encoding degradation Is calculated, and the amount of allocated bits for encoding based on the encoding difficulty is set. Furthermore, as processing for the second pass, the video encoding apparatus disclosed in Patent Document 4 encodes the input image so that the set allocation bit amount and the encoding amount to be allocated match.

JP 2008-503919 A JP 2005-94458 A Unexamined-Japanese-Patent No. 2002-232882 Japanese Patent Application Laid-Open No. 10-302396

  As described above, in the video encoding devices disclosed in Patent Documents 1 to 4 described above, the multi-pass encoding process is performed, so the encoding process itself becomes complicated. Therefore, in these video encoding devices, the number of substrates to be mounted increases, which causes a problem that the device itself becomes large.

  An example of the object of the present invention is to provide a video encoding apparatus, a video encoding method, and a program that can realize the miniaturization of the apparatus itself while solving the above problems and achieving high image quality of a video to be broadcast. It is to do.

In order to achieve the above object, a video encoding apparatus according to an aspect of the present invention is:
An apparatus for encoding a time-series image composed of a plurality of frames, comprising:
The first quantization parameter of the frame is calculated based on the complexity of the frame for each of the plurality of frames, and the frame is further divided into two or more blocks. A pre-processing unit that calculates a second quantization parameter using a quantization parameter of 1;
Each of the plurality of frames is quantized using the second quantization parameter calculated for the block for each of the blocks that constitute the frame, and the frame is further quantized A video encoding unit for encoding;
It is characterized by having.

Further, to achieve the above object, a video encoding method according to an aspect of the present invention is:
A method for encoding a time series image composed of a plurality of frames, comprising:
(A) For each of the plurality of frames, the first quantization parameter of the frame is calculated based on the complexity of the frame, and the frame is further divided into two or more blocks, and for each of the blocks Calculating a second quantization parameter using the first quantization parameter;
(B) Each of the plurality of frames is quantized using the second quantization parameter calculated for the block for each of the blocks constituting the frame, and the blocks are further quantized Encoding the frame;
It is characterized by having.

Furthermore, in order to achieve the above object, a program according to an aspect of the present invention can be executed by a computer.
A program for encoding a time-series image composed of a plurality of frames, the program comprising
On the computer
(A) For each of the plurality of frames, the first quantization parameter of the frame is calculated based on the complexity of the frame, and the frame is further divided into two or more blocks, and for each of the blocks Calculating a second quantization parameter using the first quantization parameter;
(B) Each of the plurality of frames is quantized using the second quantization parameter calculated for the block for each of the blocks constituting the frame, and the blocks are further quantized Encoding the frame;
To execute.

  As described above, according to the present invention, it is possible to realize high image quality of a video to be broadcast and downsizing of the apparatus itself.

FIG. 1 is a block diagram showing a schematic configuration of a video encoding apparatus according to Embodiment 1 of the present invention. FIG. 2 is a diagram for explaining processing in the pre-processing unit of the video encoding device according to Embodiment 1 of the present invention. FIG. 3 is a flow chart showing an operation of the video encoding apparatus in Embodiment 1 of the present invention. FIG. 4 is a diagram for explaining the processing in the pre-processing unit of the video encoding device in the second embodiment of the present invention. FIG. 5 is a flowchart showing the operation of the video encoding apparatus in the second embodiment of the present invention. FIG. 6 is a diagram for explaining processing in the pre-processing unit of the video encoding device in Embodiment 3 of the present invention. FIG. 7 is a flow chart showing the operation of the video encoding apparatus in the third embodiment of the present invention. FIG. 8 is a flow chart showing the operation of the video encoding apparatus in the fourth embodiment of the present invention. FIG. 9 is a block diagram showing an example of a computer for realizing the video encoding apparatus in the first to fourth embodiments of the present invention.

Embodiment 1
Hereinafter, a video encoding apparatus, a video encoding method, and a program according to Embodiment 1 of the present invention will be described with reference to FIGS. 1 to 3.

[Device configuration]
First, the configuration of the video encoding apparatus according to the first embodiment will be described with reference to FIG.
FIG. 1 is a block diagram showing a schematic configuration of a video encoding apparatus according to Embodiment 1 of the present invention.

  As shown in FIG. 1, the video encoding device 10 according to the first embodiment is a device for encoding a time-series image composed of a plurality of frames. As shown in FIG. 1, the video encoding device 10 includes a pre-processing unit 11 and a video encoding unit 12.

  The pre-processing unit 11 calculates, for each of a plurality of frames, a first quantization parameter of the frame based on the complexity of the frame, and further divides the frame into two or more blocks. A second quantization parameter is calculated using a quantization parameter of one.

  The video encoding unit 12 performs quantization for each of a plurality of frames using the second quantization parameter calculated for each block that constitutes the frame, and further, the frame in which the block is quantized is Encode.

  As described above, in the first embodiment, the second quantization parameter is calculated for each block of a frame using the first quantization parameter calculated based on the complexity of the frame for each frame. . Then, quantization and coding are performed for each block using the second quantization parameter. That is, in the first embodiment, in the calculation process of the second quantization parameter, unlike the conventional multi-pass coding process, it is not necessary to use statistical information or analysis information etc., and the whole process can be simplified. An increase in the number of substrates can be suppressed. Furthermore, in the first embodiment, since the frame is encoded using the second quantization parameter calculated for each block, deterioration in image quality is also suppressed. Therefore, according to the first embodiment, it is possible to miniaturize the apparatus itself while achieving high image quality of the video to be broadcast.

  Further, in the first embodiment, the “complexity” may indicate the complexity of the change of the pixel in the image. As a method of calculating the degree of complexity, for example, each pixel constituting the frame is binarized as “1” if the luminance is equal to or greater than the threshold, or “0” if the luminance is less than the threshold, “0” to “1 Or the number of changes from “1” to “0”, and there is a method of setting the calculated number as the complexity.

  Subsequently, the configuration of the video encoding device 10 according to the first embodiment will be more specifically described using FIG. 2 in addition to FIG. FIG. 2 is a diagram for explaining processing in the pre-processing unit of the video encoding device according to Embodiment 1 of the present invention.

  In the first embodiment, the pre-processing unit 11 first obtains a time-series image composed of a plurality of frames, and one of the frames is a target frame of the encoding process (hereinafter referred to as a target frame 22). Set as Then, the pre-processing unit 11 calculates the complexity of the target frame 22 and calculates a first quantization parameter QPm of the target frame 22 based on the calculated complexity. Here, the target frame 22 is the m-th frame of a plurality of frames, the complexity of the m-th frame is Xm, the video coding rate is Y [bps], and the video frame rate is Z [fps] Let A_x (Ai, Ap, Ab) be an arbitrary coefficient according to the type x of the picture (I picture, P picture, B picture). In this case, the pre-processing unit 11 calculates the first quantization parameter QPm using Equation 1 below.

  Subsequently, as illustrated in FIG. 2, the pre-processing unit 11 divides the target frame 22 into two or more blocks 21. Further, the size of each block 21 is not particularly limited. For example, in the first embodiment, the size of the block 21 is set to 16 × 16 pixels.

  Next, as shown in FIG. 2, the pre-processing unit 11 selects a reference frame (hereinafter referred to as a reference frame 23) from among the plurality of frames. Here, the reference frame 23 to be selected may be a frame immediately before the target frame 22 or may be any frame. The reference frame 23 may be selected in advance. The pre-processing unit 11 also divides the reference frame 23 into two or more blocks 21 as described above.

  Next, as shown in FIG. 2, the pre-processing unit 11 compares the target frame 22 with the reference frame 23 to calculate a difference for each block of the target frame 22. Specifically, the pre-processing unit 11 identifies an arbitrary block 21 of the target frame 22 and a block of the reference frame 23 corresponding thereto. Then, the pre-processing unit 11 obtains the difference between the corresponding pixels in the two identified blocks, integrates the found differences between the respective pixels in the entire block, and calculates the difference between the identified two blocks. The pre-processing unit 11 performs the same process on all the blocks of the target frame 22 to calculate a difference.

  Here, the target frame 22 at time t is Org (t), the reference frame 23 at time t + 1 is Org (t + 1), the y-th block is MBy, and the i-th pixel value constituting the block is Yi. . Further, in the first embodiment, since the block size is configured of 16 × 16 pixels, the number of pixels n is 256. In this case, the pre-processing unit 11 calculates the difference between the y-th block of the target frame 22 and the y-th block of the reference frame 23 corresponding thereto (hereinafter referred to as “MBy difference”) as follows. It can be calculated using Equation 2.

  Next, the pre-processing unit 11 calculates the second quantization parameter QP_MBy of the target frame 22 for each block. Specifically, the pre-processing unit 11 uses the MBy difference calculated for each block from the target frame 22 and the reference frame 23 and the first quantization parameter QPm of the target frame 22 to perform the target for each block. The second quantization parameter QP_MBy of the frame 22 is calculated. Specifically, the pre-processing unit 11 can calculate the second quantization parameter QP_MBy using the following Equations 3 to 7. Further, the pre-processing unit 11 outputs the calculated second quantization parameter QP_MBy and the target frame 22 to the video encoding unit 12.

  In the first embodiment, the video encoding unit 12 first converts the pixel values of the pixels forming the block into frequency components for each block in the target frame 22. Moreover, conversion from each pixel value to a frequency component can be performed using an existing method, and specifically, DCT (Discrete Cosine Transform: Discrete Cosine Transform) can be used.

  Next, the video encoding unit 12 quantizes the value of each pixel constituting the block using the second quantization parameter for each block in which each pixel is converted into a frequency component in the target frame 22. Here, assuming that transformation to frequency components is performed by DCT, the value after quantization of each pixel is the value of the corresponding frequency component (DCT coefficient), the corresponding element of the quantization matrix, and For example, it is calculated by the following equation 8 using a quantization parameter of 2.

  Note that the quantization of each pixel may be quantization based on inverse quantization defined in ISO / IEC 13818-2 (MPEG-2 standard document) other than Expression 8. Also, examples of the quantization matrix used in the first embodiment include those based on ISO / IEC 13818-2 (MPEG-2 standard).

  Next, for each block in which each pixel is quantized, the video encoding unit 12 reads the data while scanning the inside of the block, and encodes the entire target frame by compressing the read data.

[Device operation]
Next, the operation of the video encoding device 10 according to the first embodiment of the present invention will be described using FIG. FIG. 3 is a flow chart showing an operation of the video encoding apparatus in Embodiment 1 of the present invention. In the following description, FIGS. 1 and 2 will be referred to as appropriate. Further, in the first embodiment, the video coding method is implemented by operating the video coding apparatus 10. Therefore, the description of the video coding method in the first embodiment is replaced with the following description of the operation of the video coding apparatus 10.

  As shown in FIG. 3, first, the pre-processing unit 11 acquires a time-series image composed of a plurality of frames (step A1).

  Next, the pre-processing unit 11 takes out one frame constituting the time-series image acquired in step A1 and sets it as a target frame 22 (step A2).

  Next, the pre-processing unit 11 calculates the complexity of the target frame 22 set in step A2 (step A3). Specifically, the pre-processing unit 11 binarizes each pixel constituting the frame as “1” if the luminance is equal to or greater than the threshold, and as “0” if the luminance is less than the threshold, “1” and “1”. The number of changes between 0 and 0 is measured, and the obtained value is taken as the complexity.

  Next, the pre-processing unit 11 calculates a first quantization parameter QPm of the target frame 22 based on the degree of complexity calculated in step A3 (step A4). Specifically, the pre-processing unit 11 calculates the first quantization parameter QPm using the above Equation 1.

  Next, the pre-processing unit 11 divides the target frame 22 into two or more blocks 21 (step A5). Specifically, the pre-processing unit 11 divides the target frame 22 so that the block size is 16 × 16.

  Next, the pre-processing unit 11 selects a reference frame 23 as a reference frame from the time-series image acquired in step A1 (step A6). In step A6, the pre-processing unit 11 divides the reference frame 23 so that the block size is 16 × 16, as in step A5. When a frame to be used as a reference frame is set in advance, step A6 is omitted.

  Next, the pre-processing unit 11 compares the target frame 22 with the reference frame 23 to calculate a difference for each block of the target frame 22 (step A7). Specifically, the pre-processing unit 11 calculates the MBy difference for each block, using the above-mentioned Equation 2.

  Next, the pre-processing unit 11 calculates, for each block, the second quantization parameter QP_MBy of the target frame 22 using the MBy difference and the first quantization parameter QPm (step A8). Specifically, the pre-processing unit 11 calculates the second quantization parameter QP_MBy for each block using the above-described Equations 3 to 7.

  Next, the video encoding unit 12 quantizes the value of each pixel making up each block in the target frame 22 using the second quantization parameter (step A9). Specifically, the video encoding unit 12 quantizes each pixel forming the block, using the above-described Equation 8 for each block.

  Next, the video encoding unit 12 encodes the target frame 22 in which each block is quantized (step A10). Specifically, for each block in which each pixel is quantized, the video encoding unit 12 reads the data while scanning the inside of the block, and encodes the entire target frame by compressing the read data.

  After that, the pre-processing unit 11 determines whether the process is completed for all the frames (step A11). As a result of the determination in step A11, when the process has not been completed for all the frames, the pre-processing unit 11 executes step A2 again. On the other hand, as a result of the determination in step A11, when the processing for all the frames is completed, the processing in the video encoding device 10 is ended.

[Effect in Embodiment 1]
As described above, according to the first embodiment, unlike the conventional multi-pass coding processing, it is not necessary to use statistical information or analysis information and the like, and the entire processing can be simplified, so that the increase of the substrate can be suppressed. In addition, since the frame is encoded using the second quantization parameter calculated for each block, deterioration in image quality is also suppressed. Furthermore, in the first embodiment, since the video encoding device 10 can flexibly allocate a bit rate to each block, encoding can be efficiently performed.

[program]
The program according to the first embodiment of the present invention may be any program that causes a computer to execute steps A1 to A11 shown in FIG. By installing this program in a computer and executing it, the video encoding device 10 and the video encoding method in the present embodiment can be realized. In this case, the processor of the computer functions as the pre-processing unit 11 and the video encoding unit 12 to perform processing.

  Also, the program in the first embodiment may be executed by a computer system constructed by a plurality of computers. In this case, for example, each computer may function as either the pre-processing unit 11 or the video encoding unit 12.

Second Embodiment
Next, a video coding apparatus, a video coding method, and a program according to the second embodiment of the present invention will be described with reference to FIGS. 4 and 5.

[Device configuration]
First, the configuration of the video encoding apparatus according to the second embodiment will be described. The video coding apparatus in the second embodiment has the same configuration as the video coding apparatus in the first embodiment shown in FIG. Therefore, in the following description, FIG. 1 will be referred to as appropriate. However, the video encoding apparatus according to the second embodiment differs from the video encoding apparatus according to the first embodiment in the function of the pre-processing unit 11. Hereinafter, differences from the first embodiment will be mainly described with reference to FIG.

  FIG. 4 is a diagram for explaining the processing in the pre-processing unit of the video encoding device in the second embodiment of the present invention. As shown in FIG. 4, in the second embodiment, the pre-processing unit 11 first calculates, for each block of the target frame 22, the variance value in the block from the pixel value of each pixel constituting the block.

  Specifically, the pre-processing unit 11 calculates an average value of all pixel values constituting the block of the target frame 22. And the pre-processing part 11 calculates | requires the difference of the pixel value of each pixel, and the average value of the calculated all pixel values for every pixel which comprises a block, and integrates the difference calculated | required for every pixel over the whole block. Furthermore, the pre-processing unit 11 divides the integrated value of the differences of the entire block by the number of pixels of the entire block to calculate the dispersion value V [y] of the block. The pre-processing unit 11 performs the same process on all the blocks of the target frame 22 to calculate the dispersion value V [MBy] for each block.

  Here, the pixel value of each pixel forming the block 21 is yi, and the average value of the pixel values of all the pixels forming the block 21 is u. Further, as in the first embodiment, also in the second embodiment, when the block 21 is configured by 16 × 16 pixels, the number of pixels n is 256. In this case, the pre-processing unit 11 can calculate the block variance value V [y] using the following Equation 9.

  Next, the pre-processing unit 11 uses the variance value calculated for each block and the first quantization parameter QPm of the target frame 22 to generate the second quantization parameter QP_MBy of the target frame 22 for each block. calculate. Here, assuming that the dispersion value of each block of the target frame 22 is V [MBy] and an arbitrary coefficient to the dispersion value of each block of the target frame 22 is f, the second quantization parameter QP_MBy is It can be calculated using Equation 11. Further, the pre-processing unit 11 outputs the calculated second quantization parameter QP_MBy and the target frame 22 to the video encoding unit 12.

  After that, as in the first embodiment, the video encoding unit 12 performs quantization for each block constituting the target frame 22 using the second quantization parameter calculated for each block, and further performs block calculation. Encodes the quantized target frame 22.

[Device operation]
Next, the operation of the video encoding device 10 according to the second embodiment of the present invention will be described using FIG. FIG. 5 is a flowchart showing the operation of the video encoding apparatus in the second embodiment of the present invention. In the following description, FIGS. 1 and 4 will be referred to as appropriate. Also, in the second embodiment, the video coding method is implemented by operating the video coding apparatus. Therefore, the description of the video coding method in the second embodiment is replaced with the following description of the operation of the video coding apparatus.

  As shown in FIG. 4, first, the pre-processing unit 11 acquires a time-series image composed of a plurality of frames (step B1). Step B1 is the same step as step A1 shown in FIG.

  Next, the pre-processing unit 11 takes out one frame constituting the time-series image acquired in step B1 and sets it as the target frame 22 (step B2). Step B2 is the same step as step A2 shown in FIG.

  Next, the pre-processing unit 11 calculates the complexity of the target frame 22 set in step B2 (step B3). Step B3 is the same step as step A3 shown in FIG.

  Next, the pre-processing unit 11 calculates a first quantization parameter QPm of the target frame 22 based on the degree of complexity calculated in step B3 (step B4). Step B4 is the same step as step A4 shown in FIG.

  Next, the pre-processing unit 11 divides the target frame 22 into two or more blocks 21 (step B5). Step B5 is the same step as step A5 shown in FIG.

  Next, the pre-processing unit 11 calculates the variance value V [y] for each block of the target frame 22 (step B6). Specifically, the pre-processing unit 11 calculates the variance value V [y] for each block using the above-described Equation 9.

  Next, the pre-processing unit 11 calculates, for each block, the second quantization parameter QP_MBy of the target frame 22 using the dispersion value V [y] and the first quantization parameter QPm (step B7). . Specifically, the pre-processing unit 11 calculates the second quantization parameter QP_MBy for each block using the above Equations 10 and 11.

Next, the video encoding unit 12 quantizes each block in the target frame 22 using the second quantization parameter calculated in step B7 (step B8). Specifically, the video encoding unit 12 quantizes each pixel forming the block, using the above-described Equation 8 for each block.

  Next, the video encoding unit 12 encodes the target frame 22 in which each block is quantized (step B9). Step B9 is similar to step A10 shown in FIG.

  Thereafter, the pre-processing unit 11 determines whether the process is completed for all the frames (step B10). If it is determined in step B10 that the process has not been completed for all the frames, the pre-processing unit 11 executes step B2 again. On the other hand, as a result of the determination in step B10, when the process is completed for all the frames, the process in the video encoding device is completed.

[Effect in Embodiment 2]
As described above, also in the second embodiment, as in the first embodiment, the entire process can be simplified, so that the increase in the number of substrates can be suppressed. Further, in the second embodiment, the video encoding apparatus obtains the variance value for each block. Therefore, the video encoding apparatus determines that the frame has a gradual change to a block having a small variance value, and allocates a small bit rate. On the other hand, the video encoding apparatus allocates a large bit rate to a block having a large variance value by determining that the frame is changing rapidly. For this reason, also when the second embodiment is used, bit rates can be flexibly allocated similarly to the first embodiment described above, and encoding can be efficiently performed.

  In addition, in the second embodiment, since the state of change in the entire frame is specified by the dispersion value obtained for each block, the image having a large amount of motion as compared with the first embodiment can be obtained. Can perform encoding more efficiently.

[program]
The program according to the second embodiment of the present invention may be a program that causes a computer to execute steps B1 to B10 shown in FIG. By installing this program in a computer and executing it, the video encoding device and the video encoding method according to the present embodiment can be realized. In this case, the processor of the computer functions as the pre-processing unit 11 and the video encoding unit 12 to perform processing.

  Further, the program in the second embodiment may be executed by a computer system constructed by a plurality of computers. In this case, for example, each computer may function as either the pre-processing unit 11 or the video encoding unit 12.

Third Embodiment
Next, a video encoding apparatus, a video encoding method, and a program according to the third embodiment of the present invention will be described with reference to FIGS. 6 and 7.

[Device configuration]
First, the video encoding apparatus according to the third embodiment will be described. The video coding apparatus in the third embodiment also has the same configuration as the video coding apparatus in the first embodiment shown in FIG. Therefore, in the following description, FIG. 1 will be referred to as appropriate. However, the video encoding apparatus according to the third embodiment differs from the video encoding apparatus according to the first embodiment in the function of the pre-processing unit 11. Hereinafter, differences from the first embodiment will be mainly described with reference to FIG.

  FIG. 6 is a diagram for explaining processing in the pre-processing unit of the video encoding device in Embodiment 3 of the present invention. As shown in FIG. 6, in the third embodiment, the pre-processing unit 11 first generates a prediction frame 30 by motion-compensated inter-frame prediction. Specifically, the prediction frame 30 is generated from the target frame 22 and the reference frame 23 by motion compensation interframe prediction. In addition, motion compensation inter-frame prediction can be performed using an existing method, and specifically, a gradient method or a block matching method can be used.

  Next, the pre-processing unit 11 compares the generated predicted frame 30 with the target frame 22 to calculate a difference for each block of the target frame 22. Specifically, the pre-processing unit 11 identifies an arbitrary block 21 of the target frame 22 and a block of the prediction frame 30 corresponding to the arbitrary block 21. Then, the pre-processing unit 11 obtains a difference between corresponding pixels in the two identified blocks, integrates the obtained differences between the respective pixels in the entire block, and calculates a difference between the identified two blocks. The pre-processing unit 11 performs the same process on all the blocks of the target frame 22 to calculate a difference.

  Here, the target frame 22 at time t is Org (t), the predicted frame 30 at time t is Pred (t), the y-th block is MBy, and the i-th pixel value constituting the block is Yi. . Further, in the third embodiment, assuming that the block size is configured by 16 × 16 pixels, the number of pixels n is 256. In this case, the pre-processing unit 11 calculates the difference between the y-th block of the target frame 22 and the y-th block of the predicted frame 30 corresponding to this (denoted as "Diff [MBy]") by the following number: It can be calculated using 12.

  Next, the pre-processing unit 11 uses the Diff [MBy] calculated for each block from the target frame 22 and the predicted frame 30 and the first quantization parameter QPm of the target frame 22 to perform the target for each block. The second quantization parameter QP_MBy of the frame 22 is calculated. Specifically, the pre-processing unit 11 can calculate the second quantization parameter QP_MBy using the following Equations 13 to 17. Further, the pre-processing unit 11 outputs the calculated second quantization parameter QP_MBy and the target frame 22 to the video encoding unit 12.

  After that, as in the first embodiment, the video encoding unit 12 performs quantization using the second quantization parameter calculated for each block, for each of the blocks that constitute the target frame 22, and further, The target frame 22 in which the block is quantized is encoded.

[Device operation]
Next, the operation of the video encoding device 10 according to the third embodiment of the present invention will be described using FIG. FIG. 7 is a flow chart showing the operation of the video encoding apparatus in the third embodiment of the present invention. In the following description, FIGS. 1 and 6 will be referred to as appropriate. Also, in the third embodiment, the video coding method is implemented by operating the video coding apparatus. Therefore, the description of the video coding method in the third embodiment is changed to the following description of the operation of the video coding apparatus.

  As shown in FIG. 7, first, the pre-processing unit 11 acquires a time-series image composed of a plurality of frames (step C1). Step C1 is the same step as step A1 shown in FIG.

  Next, the pre-processing unit 11 takes out one frame that organizes the time-series image acquired in step C1 and sets it as a target frame 22 (step C2). Step C2 is the same step as step A2 shown in FIG.

  Next, the pre-processing unit 11 calculates the complexity of the target frame 22 set in step C2 (step C3). Step C3 is similar to step A3 shown in FIG.

  Next, the pre-processing unit 11 calculates a first quantization parameter QPm of the target frame 22 based on the degree of complexity calculated in step C3 (step C4). Step C4 is the same step as step A4 shown in FIG.

  Next, the pre-processing unit 11 divides the target frame 22 into two or more blocks 21 (step C5). Step C5 is similar to step A5 shown in FIG.

  Next, the pre-processing unit 11 selects the reference frame 23 as a reference frame from the time-series image acquired in step C1 (step C6). Step C6 is the same step as step A6 shown in FIG.

  Next, the pre-processing unit 11 generates a predicted frame 30 from the target frame 22 and the reference frame 23 (step C7). Specifically, the pre-processing unit 11 generates a prediction frame 30 from the target frame 22 and the reference frame 23 by motion compensation inter-frame prediction.

  Next, the pre-processing unit 11 compares the target frame 22 with the prediction frame 30, and calculates a difference for each block of the target frame 22 (step C9). Specifically, the pre-processing unit 11 calculates Diff [MBy] for each block using the above equation 12.

Next, the pre-processing unit 11 calculates, for each block, a second quantization parameter QP_MBy of the target frame 22 using Diff [MBy] and the first quantization parameter QPm (step C8). Specifically, the pre-processing unit 11 uses the above Equations 13 to 17 to
A second quantization parameter QP_MBy is calculated for each block.

  Next, the video encoding unit 12 quantizes each block in the target frame 22 using the second quantization parameter calculated in step C9 (step C10). Specifically, the video encoding unit 12 quantizes each pixel forming the block, using the above-described Equation 8 for each block.

  Next, the video encoding unit 12 encodes the target frame 22 in which each block is quantized (step C11). Step C11 is similar to step A10 shown in FIG.

  Thereafter, the pre-processing unit 11 determines whether the process is completed for all the frames (step C12). If it is determined in step C12 that the process has not been completed for all the frames, the pre-processing unit 11 executes step C2 again. On the other hand, as a result of the determination in step C12, when the processing for all the frames is completed, the processing in the video encoding device is ended.

[Effect in Embodiment 3]
As described above, also in the third embodiment, as in the first embodiment, the whole process can be simplified, so that the increase in the number of substrates can be suppressed. Further, in the third embodiment, the video encoding apparatus obtains the second quantization parameter for each block using a prediction frame generated by motion compensation inter-frame prediction. For this reason, even when Embodiment 3 is used, bit rates can be flexibly allocated similarly to Embodiment 1 described above, and encoding can be performed efficiently.

  In addition, in the third embodiment, a frame having a low similarity between frames is specified by the prediction frame, so that the efficiency is higher for a video with a large amount of motion as compared with the first embodiment. Encoding can be performed.

[program]
The program according to the third embodiment of the present invention may be a program that causes a computer to execute steps C1 to C12 shown in FIG. By installing this program in a computer and executing it, the video encoding device and the video encoding method in the present invention can be realized. In this case, the processor of the computer functions as the pre-processing unit 11 and the video encoding unit 12 to perform processing.

  Furthermore, the program in the third embodiment may be executed by a computer system constructed by a plurality of computers. In this case, for example, each computer may function as either the pre-processing unit 11 or the video encoding unit 12.

Embodiment 4
Next, a video encoding apparatus, a video encoding method, and a program according to a fourth embodiment of the present invention will be described with reference to FIG.

[Device configuration]
First, the video encoding apparatus according to the fourth embodiment will be described. The coding apparatus according to the fourth embodiment also has the same configuration as the video coding apparatus according to the first embodiment shown in FIG. Therefore, in the following description, FIG. 1 will be referred to as appropriate. However, the video encoding apparatus in the fourth embodiment has all the functions of the pre-processing unit 11 of the video encoding apparatus in the first embodiment, the second embodiment, and the third embodiment. This will be specifically described below with reference to FIGS. 2, 4 and 6.

  In the fourth embodiment, the pre-processing unit 11 first selects a reference frame serving as a reference from among a plurality of frames, compares the target frame with the reference frame, and selects the first for each block of the target frame. Calculate the difference of Next, for each block of the target frame, the pre-processing unit 11 calculates the variance value in each block from the pixel value of each pixel constituting each block. In addition, the pre-processing unit 11 generates a prediction frame by motion compensation inter-frame prediction, compares the target frame with the prediction frame, and calculates a second difference for each block of the target frame. Then, the pre-processing unit 11 calculates a second quantization parameter for each block of the target frame using the first difference, the variance value, the second difference, and the first quantization parameter.

  Specifically, the pre-processing unit 11 first calculates the first quantization parameter QPm using FIG. 1 shown in the above-described first embodiment. Next, the pre-processing unit 11 calculates the MBy difference as the first difference using the number 2 shown in the above-described first embodiment. Further, the pre-processing unit 11 calculates the variance value V [MBy] using the equation 9 shown in the above-mentioned second embodiment. Furthermore, the pre-processing unit 11 calculates the difference Diff [MBy] as the second difference using Expression 12 shown in the above-described third embodiment.

  Subsequently, the pre-processing unit 11 applies the MBy difference, the variance value V [MBy], the difference Diff [MBy], and the first quantization parameter QPm to the following Equations 18 to 23 to obtain the target frame 22. The second quantization parameter QP_MBy is calculated for each block of In Expression 18, α indicates an arbitrary coefficient (0 ≦ α ≦ 1). Further, in the equation (18), crit is calculated from the equation (10) used in the second embodiment described above. Then, the pre-processing unit 11 outputs the calculated second quantization parameter QP_MBy and the target frame 22 to the video encoding unit 12.

  After that, as in the first embodiment, the video encoding unit 12 performs quantization using the second quantization parameter calculated for each block, for each block that configures each frame, and further performs block calculation. Encodes each frame that has been quantized.

[Device operation]
Next, the operation of the video encoding device 10 according to the fourth embodiment of the present invention will be described using FIG. FIG. 8 is a flow chart showing the operation of the video encoding apparatus in the fourth embodiment of the present invention. In the following description, FIG. 1, FIG. 2, FIG. 4, and FIG. 6 will be referred to as appropriate. Also, in the fourth embodiment, the video coding method is implemented by operating the video coding apparatus. Therefore, the description of the video coding method in the fourth embodiment is changed to the following description of the operation of the video coding apparatus.

  As shown in FIG. 8, first, the pre-processing unit 11 acquires a time-series image composed of a plurality of frames (step D1). Step D1 is the same step as step A1 shown in FIG.

  Next, the pre-processing unit 11 takes out one frame that organizes the time-series image acquired in step D1 and sets it as the target frame 22 (step D2). Step D2 is similar to step A2 shown in FIG.

  Next, the pre-processing unit 11 calculates the complexity of the target frame 22 set in step D2 (step D3). Step D3 is the same step as step A3 shown in FIG.

  Next, the pre-processing unit 11 calculates a first quantization parameter QPm of the target frame 22 based on the degree of complexity calculated in step D3 (step D4). Step D4 is the same step as step A4 shown in FIG.

  Next, the pre-processing unit 11 divides the target frame 22 into two or more blocks 21 (step D5). Step D5 is the same as step A5 shown in FIG.

  Next, the pre-processing unit 11 selects a reference frame 23 as a reference frame from the time-series image acquired in step D1 (step D6). Step D6 is the same step as step A6 shown in FIG.

  Next, the pre-processing unit 11 compares the target frame 22 with the reference frame 23 to calculate a first difference for each block of the target frame 22 (step D7). Step D7 is similar to step A7 shown in FIG.

  Next, the pre-processing unit 11 calculates a variance value for each block of the target frame 22 (step D8). Step D8 is similar to step B6 shown in FIG.

  Next, the pre-processing unit 11 generates a prediction frame 30 from the target frame 22 and the reference frame 23 (step D9). Step D9 is similar to step C7 shown in FIG.

  Next, the pre-processing unit 11 compares the target frame 22 with the prediction frame 30, and calculates a second difference for each block of the target frame 22 (step D10). Step D10 is the same as step C8 shown in FIG.

  Next, the pre-processing unit 11 uses the first difference, the variance value, the second difference, and the first quantization parameter for each block of the target frame to perform second processing for each block of the target frame. Calculate the quantization parameter. Specifically, the pre-processing unit 11 calculates the second quantization parameter QP_MBy for each block using the above Equations 18 to 23.

  Next, the video encoding unit 12 quantizes each block in the target frame 22 using the second quantization parameter calculated in step D11 (step D12). Specifically, the video encoding unit 12 quantizes each pixel forming the block, using the above-described Equation 8 for each block.

  Next, the video encoding unit 12 encodes the target frame 22 in which each block is quantized (step D13). Step D13 is the same as step A10 shown in FIG.

  Thereafter, the pre-processing unit 11 determines whether the process is completed for all the frames (step D14). If it is determined in step D14 that the process has not been completed for all the frames, the pre-processing unit 11 executes step D2 again. On the other hand, as a result of the determination in step D14, when the processing for all the frames is completed, the processing in the video encoding device is ended.

[Effect in Embodiment 4]
As described above, in the fourth embodiment, in the video encoding device, the pre-processing unit 11 also has the function of the pre-processing unit used in each of the first to third embodiments. Therefore, according to the fourth embodiment, the effects described in the first to third embodiments can be obtained.

[program]
The program according to the fourth embodiment of the present invention may be a program that causes a computer to execute steps D1 to D14 shown in FIG. By installing this program in a computer and executing it, the video encoding device and the video encoding method in the present invention can be realized. In this case, the processor of the computer functions as the pre-processing unit 11 and the video encoding unit 12 to perform processing.

  Further, the program in the fourth embodiment may be executed by a computer system constructed by a plurality of computers. In this case, for example, each computer may function as either the pre-processing unit 11 or the video encoding unit 12.

(Physical configuration)
Here, a computer for realizing the video encoding device by executing the program according to the first to fourth embodiments will be described with reference to FIG. FIG. 9 is a block diagram showing an example of a computer for realizing the video encoding apparatus in the first to fourth embodiments of the present invention.

  As shown in FIG. 9, the computer 110 includes a CPU 111, a main memory 112, a storage device 113, an input interface 114, a display controller 115, a data reader / writer 116, and a communication interface 117. These units are communicably connected to each other via a bus 121. The computer 110 may include a graphics processing unit (GPU) or a field-programmable gate array (FPGA) in addition to or instead of the CPU 111.

The CPU 111 develops the program (code) in the present embodiment stored in the storage device 113 in the main memory 112 and executes various operations by executing these in a predetermined order. The main memory 112 is typically a volatile storage device such as a dynamic random access memory (DRAM). In addition, the program in the present embodiment is provided in the state of being stored in computer readable recording medium 120. The program in the present embodiment may be distributed on the Internet connected via communication interface 117.

  Further, as a specific example of the storage device 113, besides a hard disk drive, a semiconductor storage device such as a flash memory may be mentioned. The input interface 114 mediates data transmission between the CPU 111 and an input device 118 such as a keyboard and a mouse. The display controller 115 is connected to the display device 119 and controls the display on the display device 119.

  The data reader / writer 116 mediates data transmission between the CPU 111 and the recording medium 120, and executes reading of a program from the recording medium 120 and writing of the processing result in the computer 110 to the recording medium 120. The communication interface 117 mediates data transmission between the CPU 111 and another computer.

  In addition, specific examples of the recording medium 120 include general-purpose semiconductor storage devices such as CF (Compact Flash (registered trademark) and SD (Secure Digital), magnetic recording media such as flexible disk (Flexible Disk), or CD- Optical recording media such as ROM (Compact Disk Read Only Memory) can be mentioned.

  Note that the video encoding apparatus in the present embodiment can also be realized by using hardware corresponding to each unit, not a computer on which a program is installed. Furthermore, a part of the video encoding apparatus may be realized by a program, and the remaining part may be realized by hardware.

  A part or all of the embodiment described above can be expressed by (Appendix 1) to (Appendix 15) described below, but is not limited to the following description.

(Supplementary Note 1)
An apparatus for encoding a time-series image composed of a plurality of frames, comprising:
The first quantization parameter of the frame is calculated based on the complexity of the frame for each of the plurality of frames, and the frame is further divided into two or more blocks. A pre-processing unit that calculates a second quantization parameter using a quantization parameter of 1;
Each of the plurality of frames is quantized using the second quantization parameter calculated for the block for each of the blocks that constitute the frame, and the frame is further quantized A video encoding unit for encoding;
A video coding apparatus comprising:

(Supplementary Note 2)
The pre-processing unit selects a reference frame as a reference from among the plurality of frames, compares the frame with the reference frame, calculates a difference for each block of the frame, and calculates the difference. Calculating the second quantization parameter for each block of the frame, using the first quantization parameter and the first quantization parameter;
The video encoding device according to appendix 1.

(Supplementary Note 3)
The pre-processing unit calculates, for each of the blocks of the frame, the variance of the block from the pixel values of the pixels forming the block, and the calculated variance and the first quantization parameter. Calculate the second quantization parameter for each block of the frame using
The video encoding device according to appendix 1.

(Supplementary Note 4)
The pre-processing unit generates a prediction frame by motion compensation inter-frame prediction, compares the frame with the prediction frame, calculates a difference for each block of the frame, and calculates the difference and the first difference. Calculating the second quantization parameter for each block of the frame using the quantization parameter of
The video encoding device according to appendix 1.

(Supplementary Note 5)
The pre-processing unit selects a reference frame serving as a reference from the plurality of frames, compares the frame with the reference frame, and calculates a first difference for each block of the frame.
Furthermore, for each of the blocks of the frame, the variance value of the block is calculated from the pixel values of the pixels that make up the block,
In addition, a prediction frame is generated by motion compensation inter-frame prediction, the frame is compared with the prediction frame, and a second difference is calculated for each block of the frame,
Then, using the first difference, the variance value, the second difference, and the first quantization parameter, the second quantization parameter is calculated for each block of the frame.
The video encoding device according to appendix 1.

(Supplementary Note 6)
A method of encoding a time-series image composed of a plurality of frames, comprising:
(A) For each of the plurality of frames, the first quantization parameter of the frame is calculated based on the complexity of the frame, and the frame is further divided into two or more blocks, and for each of the blocks Calculating a second quantization parameter using the first quantization parameter;
(B) Each of the plurality of frames is quantized using the second quantization parameter calculated for the block for each of the blocks constituting the frame, and the blocks are further quantized Encoding the frame;
A video coding method characterized in that:

(Appendix 7)
In the step (a),
A reference frame serving as a reference is selected from the plurality of frames, the frame is compared with the reference frame, a difference is calculated for each block of the frame, and the calculated difference and the first quantization. Calculating the second quantization parameter for each block of the frame using the parameter;
The video coding method according to appendix 6.

(Supplementary Note 8)
In the step (a),
For each of the blocks of the frame, the variance of the block is calculated from the pixel values of the pixels forming the block, and the calculated variance and the first quantization parameter are used to Calculating the second quantization parameter for each of the blocks;
The video coding method according to appendix 6.

(Appendix 9)
In the step (a),
A prediction frame is generated by motion compensation interframe prediction, the frame is compared with the prediction frame, a difference is calculated for each block of the frame, and the calculated difference and the first quantization parameter are used. Calculating the second quantization parameter for each block of the frame,
The video coding method according to appendix 6.

(Supplementary Note 10)
In the step (a),
A reference frame serving as a reference is selected from the plurality of frames, the frame is compared with the reference frame, and a first difference is calculated for each block of the frame.
Furthermore, for each of the blocks of the frame, the variance value of the block is calculated from the pixel values of the pixels that make up the block,
In addition, a prediction frame is generated by motion compensation inter-frame prediction, the frame is compared with the prediction frame, and a second difference is calculated for each block of the frame,
Then, using the first difference, the variance value, the second difference, and the first quantization parameter, the second quantization parameter is calculated for each block of the frame.
The video coding method according to appendix 6.

(Supplementary Note 11)
A program for encoding a time series image composed of a plurality of frames by a computer,
On the computer
(A) For each of the plurality of frames, the first quantization parameter of the frame is calculated based on the complexity of the frame, and the frame is further divided into two or more blocks, and for each of the blocks Calculating a second quantization parameter using the first quantization parameter;
(B) Each of the plurality of frames is quantized using the second quantization parameter calculated for the block for each of the blocks constituting the frame, and the blocks are further quantized Encoding the frame;
A program that runs

(Supplementary Note 12)
In the step (a),
A reference frame serving as a reference is selected from the plurality of frames, the frame is compared with the reference frame, a difference is calculated for each block of the frame, and the calculated difference and the first quantization. Calculating the second quantization parameter for each block of the frame using the parameter;
The program according to appendix 11.

(Supplementary Note 13)
In the step (a),
For each of the blocks of the frame, the variance of the block is calculated from the pixel values of the pixels forming the block, and the calculated variance and the first quantization parameter are used to Calculating the second quantization parameter for each of the blocks;
The program according to appendix 11.

(Supplementary Note 14)
In the step (a),
A prediction frame is generated by motion compensation interframe prediction, the frame is compared with the prediction frame, a difference is calculated for each block of the frame, and the calculated difference and the first quantization parameter are used. Calculating the second quantization parameter for each block of the frame,
The program according to appendix 11.

(Supplementary Note 15)
In the step (a),
A reference frame serving as a reference is selected from the plurality of frames, the frame is compared with the reference frame, and a first difference is calculated for each block of the frame.
Furthermore, for each of the blocks of the frame, the variance value of the block is calculated from the pixel values of the pixels that make up the block,
In addition, a prediction frame is generated by motion compensation inter-frame prediction, the frame is compared with the prediction frame, and a second difference is calculated for each block of the frame,
Then, using the first difference, the variance value, the second difference, and the first quantization parameter, the second quantization parameter is calculated for each block of the frame.
The program according to appendix 11.

  As described above, according to the present invention, it is possible to realize high image quality of a video to be broadcast and downsizing of the apparatus itself. The present invention is useful in the broadcast industry.

DESCRIPTION OF SYMBOLS 10 video coding apparatus 11 pre-processing part 12 video coding part 21 block 22 object frame 23 reference frame 30 prediction frame 110 computer 111 CPU
112 main memory 113 storage device 114 input interface 115 display controller 116 data reader / writer 117 communication interface 118 input device 119 display device 120 recording medium 121 bus

Claims (7)

An apparatus for encoding a time-series image composed of a plurality of frames, comprising:
The first quantization parameter of the frame is calculated based on the complexity of the frame for each of the plurality of frames, and the frame is further divided into two or more blocks. A pre-processing unit that calculates a second quantization parameter using a quantization parameter of 1;
Each of the plurality of frames is quantized using the second quantization parameter calculated for the block for each of the blocks that constitute the frame, and the frame is further quantized A video encoding unit for encoding;
A video coding apparatus comprising: The pre-processing unit selects a reference frame as a reference from among the plurality of frames, compares the frame with the reference frame, calculates a difference for each block of the frame, and calculates the difference. Calculating the second quantization parameter for each block of the frame, using the first quantization parameter and the first quantization parameter;
The video coding apparatus according to claim 1. The pre-processing unit calculates, for each of the blocks of the frame, the variance of the block from the pixel values of the pixels forming the block, and the calculated variance and the first quantization parameter. Calculate the second quantization parameter for each block of the frame using
The video coding apparatus according to claim 1. The pre-processing unit generates a prediction frame by motion compensation inter-frame prediction, compares the frame with the prediction frame, calculates a difference for each block of the frame, and calculates the difference and the first difference. Calculating the second quantization parameter for each block of the frame using the quantization parameter of
The video coding apparatus according to claim 1. The pre-processing unit selects a reference frame serving as a reference from the plurality of frames, compares the frame with the reference frame, and calculates a first difference for each block of the frame.
Furthermore, for each of the blocks of the frame, the variance value of the block is calculated from the pixel values of the pixels that make up the block,
In addition, a prediction frame is generated by motion compensation inter-frame prediction, the frame is compared with the prediction frame, and a second difference is calculated for each block of the frame,
Then, using the first difference, the variance value, the second difference, and the first quantization parameter, the second quantization parameter is calculated for each block of the frame.
The video coding apparatus according to claim 1. A method of encoding a time-series image composed of a plurality of frames, comprising:
(A) For each of the plurality of frames, the first quantization parameter of the frame is calculated based on the complexity of the frame, and the frame is further divided into two or more blocks, and for each of the blocks Calculating a second quantization parameter using the first quantization parameter;
(B) Each of the plurality of frames is quantized using the second quantization parameter calculated for the block for each of the blocks constituting the frame, and the blocks are further quantized Encoding the frame;
A video coding method characterized in that: A program for encoding a time series image composed of a plurality of frames by a computer,
On the computer
(A) For each of the plurality of frames, the first quantization parameter of the frame is calculated based on the complexity of the frame, and the frame is further divided into two or more blocks, and for each of the blocks Calculating a second quantization parameter using the first quantization parameter;
(B) Each of the plurality of frames is quantized using the second quantization parameter calculated for the block for each of the blocks constituting the frame, and the blocks are further quantized Encoding the frame;
A program that runs

Images