JP3712344B2 - Repetitive video signal encoding method and recording medium recording program of this method - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a highly efficient repetitive video signal encoding method and a recording medium on which a program of this method is recorded.
[0002]
[Prior art]
In a video coding method that allows a plurality of coding trials for the same image sequence to improve coding efficiency in terms of code amount versus noise amount, the evaluation of coding efficiency is based on “Lagrange” established in the 18th century. The undetermined multiplier method is applied. For the undetermined multiplier λ given from the outside, the total code amount R and the total error E at the time of sequence encoding (this measure may be the sum of squares of errors or the sum of absolute values of errors depending on the situation) To evaluation criteria L = R + λ * E
And an optimization search (parameter adjustment) is performed in a direction that minimizes L. Note that L is called a Lagrangian cost function.
[0003]
Conventionally, a quantization parameter or a Lagrangian undetermined multiplier is set for each frame, and optimization is performed by repeating a cycle of parameter adjustment, coding, code amount versus noise amount evaluation.
[0004]
[Problems to be solved by the invention]
In high-efficiency video coding technology such as MPEG-2 (ISO / IEC 13818-2), which is widely used nowadays, there is a means to divide a frame into small blocks and perform inter-frame motion compensation and coding in this unit. Adopted. If parameters such as the quantization step are set independently for each small block by such means, the best theoretical optimization is possible, but the processing becomes too large (for example, in a CCIR 601 image within one frame) 1350 small blocks), not suitable for practical use. When processing is performed within a practical time, only one parameter to be adjusted in an optimization cycle has been proposed for each frame.
[0005]
For example, Mizuno et al. Derives the optimal code amount allocated for each frame (Picture coding simposium '99, “A study on bit allocation method based on rate-distortion properties for different coded picture types”). . Also, K.Ramchandran et al. Use the quantization parameter for each frame as an optimization adjustment parameter (IEEE transactions on image processing, Vo1.3, No. 5, September 1994, “Bit allocation for dependent quantization with applications to multiresolution and MPEG video” coders ”).
[0006]
In any of the methods, the frame is encoded with a uniform parameter, and consideration of more detailed encoding units such as small blocks is omitted. As a result, there is essentially a room for optimization. was there.
[0007]
The present invention has been made in view of the above circumstances, and it is an iterative video signal code that determines a unit of parameter adjustment that can perform better optimization than conventional methods in terms of code amount versus noise while maintaining a practical processing amount. It is an object to provide a recording method and a recording medium on which a program for the method is recorded.
[0008]
[Means for Solving the Problems]
In order to solve the above problems, in the first invention, in a video signal encoding method for dividing an image into small blocks, performing inter-frame motion compensation, and further encoding the same image multiple times,
A procedure for converting the number of references from other frames to be encoded using motion vector information into numerical values (hereinafter referred to as “referenced degree”);
This is a procedure for determining the code amount allocation of a small block to be encoded using the reference degree ,
A procedure for using the motion vector information to make a reference to some of the references from other frames of the small block to be encoded,
For each small block that is a coding unit, a process of voting a predetermined amount according to a predicted macroblock of the small block to an area referred to by the small block in the reference image frame;
A process of adding the values voted for each area and setting the result as a reference degree of the area;
A process of repeating this process in the order of image frames referenced from image frames that are not referenced any more;
It is a repetition video signal encoding method characterized by having.
[0010]
In the second invention , when determining the code amount allocation of the small block to be encoded using the reference degree,
A procedure for converting a referenced degree into an undetermined multiplier using a table in which the referenced degree and Lagrange's undetermined multiplier correspond one-to-one;
A procedure for obtaining a Lagrangian cost function L of a small block to be encoded from a given undetermined multiplier λ _B from the following equation :
A procedure for obtaining a quantization parameter that minimizes the obtained Lagrangian cost function L ;
It is a repetition video signal encoding method characterized by having.
L = R _B + λ _B E _B
(Where R is the number of bits of the small block, E is the decoding error of the small block)
[0011]
A third invention is, when determining the bits allocation of the encoding target small block using the referenced degree,
A procedure for converting a referenced degree into an undetermined multiplier using a table in which the referenced degree and Lagrange's undetermined multiplier correspond one-to-one;
For each small block, according to the Lagrange's undetermined multiplier corresponding to the reference degree of the small block, a procedure for obtaining a quantization parameter that minimizes the Lagrangian cost function;
A procedure for obtaining a Lagrangian cost function L of the entire image sequence from the following equation from the code amount and the total noise of the entire image sequence :
A procedure for evaluating the obtained Lagrangian function L and determining whether the amount of noise with respect to the code amount is minimum;
It is a repetition video signal encoding method characterized by having.
L = R + λ ₀ E
(Where R is the number of bits of the small block, E is the decoding error of the small block, and λ ₀ is an undetermined multiplier for the entire image)
[0012]
The fourth invention is a procedure for converting the number of references from other frames to be encoded using motion vector information into numerical values (hereinafter referred to as “referenced degree”);
A procedure for determining a code amount allocation of a small block to be encoded using the dereferenced degree, and a procedure for converting the reference from other frames to the small block to be encoded using the motion vector information. But,
For each small block that is a coding unit, a process of voting a predetermined amount according to a predicted macroblock of the small block to an area referred to by the small block in the reference image frame;
A process of adding the values voted for each area and setting the result as a reference degree of the area;
A process of repeating this process in the order of image frames that are referenced from image frames that are not referenced any more.
A recording medium recording a program for a repetitive video signal encoding method, wherein the program is recorded on a recording medium readable by the computer as a program to be executed by a computer.
[0014]
In the fifth aspect of the present invention, when determining the code amount allocation of the encoding target small block using the reference degree, the reference degree is not determined by using a table in which the reference degree and the Lagrange's undetermined multiplier are associated one-to-one. Steps to convert to a multiplier,
A procedure for obtaining a Lagrangian cost function L of a small block to be encoded from a given undetermined multiplier λ _B from the following equation:
A procedure for obtaining a quantization parameter that minimizes the obtained Lagrangian cost function L;
A recording medium recording a program for a repetitive video signal encoding method, wherein the program is recorded on a recording medium readable by the computer as a program to be executed by a computer.
L = R _B + λ _B E _B
(Where R is the number of bits of the small block, E is the decoding error of the small block)
[0015]
According to a sixth aspect of the present invention, when determining the code amount allocation of a small block to be encoded using the degree of reference, the degree of reference is determined as an undetermined multiplier using a table in which the degree of reference and the Lagrange's undetermined multiplier are associated with each other. To convert to
For each small block, according to the Lagrange's undetermined multiplier corresponding to the reference degree of the small block, a procedure for obtaining a quantization parameter that minimizes the Lagrangian cost function;
A procedure for obtaining a Lagrangian cost function L of the entire image sequence from the following equation from the code amount and the total noise of the entire image sequence:
A procedure for evaluating the obtained Lagrangian function L and determining whether the amount of noise with respect to the amount of codes is minimum;
A recording medium recording a program for a repetitive video signal encoding method, wherein the program is recorded on a recording medium readable by the computer as a program to be executed by a computer.
L = R + λ ₀ E
(Where R is the number of bits of the small block, E is the decoding error of the small block, and λ ₀ is an undetermined multiplier for the entire image)
[0016]
As described above, in the present invention, an inter-frame reference relationship is obtained from image motion vector information, and the encoding importance for each image small block is determined based on this.
[0017]
Prior to processing, an image is once encoded to determine motion compensation information and encoding mode information. For each small block which is a coding unit, the motion vector information is used to vote an amount appropriately determined according to the attribute of the small block to the area referred to by the small block in the reference image frame.
[0018]
For example, in the case of a bidirectional prediction macroblock, “V = 0.5” is voted for each reference area in the frame being referenced (two frames before and after). If it is a forward prediction macroblock, “V (= V ₀ +1)” is voted for a reference area in a frame (one frame) that is referenced in time. Here, V ₀ is a vote value already stored in the adder of the small block. If it is an intra (intra-frame coding) macroblock, no image signal is referred to, so no voting is performed.
[0019]
For example, in MPEG-2, there may be a small block, which is skipped macro b1ock, in which encoding is omitted, but this is voted on the assumption that there is no motion (motion vector is (0, 0)). In addition, a motion vector called concealment motion vector may be added to an intra macroblock, but this motion vector is ignored and voting is not performed.
[0020]
The voting performed here is a two-dimensional one using a two-dimensional array adder prepared for each frame. For example, since the reference degree of all blocks in the MPEG-2 B frame (bidirectional prediction frame) is always “0”, the adder can be omitted. Also, since all blocks in the I frame (intra-frame encoded frame) are intra-frame encoded, voting operations can be omitted for that frame. All the adders are initialized to “0” before the voting operation.
[0021]
In the case of a method using a translation model such as MPEG-2, the voting value is added within the same shape and size range as the small block. In the case of a method using an affine / perspective transformation model such as MPEG-4 Version 2 (ISO / IEC14496-2AMD1), a vote value is added to an area obtained by inversely transforming a small block shape. In the latter case, the area S _{v of the} voting region is generally different from the area S _{b of the} small block, so when the voting value is V, for example, V ′ = V * S _b / S _v
The vote value V ′ modified as follows is used.
[0022]
As the two-dimensional adder array used here, for example, a configuration in which one adder corresponds to one pixel, a configuration in which one adder corresponds to one small block, or the like can be considered. In the former case, the vote value is added to a full adder in the voting area, for example, at the time of voting. In the latter case, at the time of voting, the ratio of the overlapping area is added to the full adder in which the voting area and the small block overlap. That is, if the vote value is V, the area of the vote area is S _v , and the area overlapping the adder area is S ₀ , the add value is V * S ₀ / S _v ,
It becomes. This is shown in FIG. In the example shown in FIG. 1, the target small block of the prediction frame spans four small blocks of the reference frame, and when the small block area is “1”, the overlapping areas are in order from left to right and from top to bottom. Since these values are 0.07V, 0.26V, 0.15V, and 0.52V, these values are added to the corresponding adder (in FIG. 1, the adder is indicated by a symbol “•”). This voting operation is repeated from the frame that is not referred to any more, from the bottom up in the order of the low degree of reference. This is shown in FIG.
[0023]
By this processing, the voting results regarding the degree of reference are accumulated for the adders in all the small blocks in the encoding target image signal. When one adder corresponds to one pixel, for example, the sum of the voting results, which is the output of the full adder in the small block, is set as the reference degree of the small block. When one adder corresponds to one small block, the output value itself of the adder is used as the reference level. The voting value is always “0” for B frames that are not referenced by others, so the adder may be omitted.
[0024]
In each small block, a parameter that embodies a larger encoding weight is given as the degree of reference is higher. The correspondence between the referenced level and the parameter is arbitrary, but the parameter is such that less coding noise or a larger amount of code is generated for a higher referenced level. For example, a smaller quantization parameter, a larger Lagrangian undetermined multiplier λ, or an assigned code amount itself may be increased. For a small block with a degree of reference of “0”, for example, a Lagrange undetermined multiplier itself (assumed to be λ ₀ ) given to the entire sequence is used to obtain a quantization parameter.
[0025]
That is, when the code amount of the small block is R ₀ and the error is E ₀ (both are normalized so as to be proportional to the amount per frame or sequence), L ₀ = R ₀ + λ ₀ * E ₀
The quantization parameter that minimizes the Lagrangian cost function L ₀ defined in (1) is used for encoding the small block. This parameter is adjusted sequentially to perform optimization so as to minimize the evaluation amount L of the entire sequence.
[0026]
In order to minimize the Lagrangian cost function L described above, optimization is performed by the following double loop.
[0027]
Outer loop: The correspondence between the degree of reference and λ (a line graph control point shown in FIG. 4) is adjusted. Inner loop: For each small block, a quantization parameter that minimizes L according to λ corresponding to the degree of reference of the small block is obtained brute force.
[0028]
When a code amount, a quantization parameter, or a Lagrange undetermined multiplier is given for each frame as in the past, there is a small area that is not referenced at all in an I or P frame (forward prediction frame), or within a certain P frame. Even if there is a small area that is referenced very frequently, the code amount allocation cannot be considered, whereas the above-described method allows a larger amount of code to be allocated to a smaller block with a higher degree of reference. As a result, the coding efficiency of the entire sequence is improved.
[0029]
Also, parameter adjustment / optimization is performed according to the reference degree of the small block, so that different code amount allocation is realized for each small block, but the optimization search is performed by associating the parameter with all the small blocks of the image sequence. Compared with, the amount of calculation is significantly reduced.
[0030]
DETAILED DESCRIPTION OF THE INVENTION
Embodiments of the present invention will be described below with reference to the drawings. FIG. 3 is a flowchart of the processing procedure of the encoding method describing the embodiment of the present invention. In FIG. 3, first, the motion detector 01 extracts the encoding mode and motion vector information 04 of the input image signal 02. No encoding is performed at this stage. This information is stored in the memory 05 and input to the voting device 06. The voting device 06 obtains the reference degree 07 from the encoding mode and the motion vector information 04. The obtained reference degree 07 is stored in the memory 08.
[0031]
The parameter adjustment unit 03 determines the degree of code amount allocation priority corresponding to the reference degree 07. For example, as shown in FIG. 4, one Lagrange multiplier λ is associated with one reference degree by a line graph determined at a plurality of control points. When the reference degree is “0”, λ ₀ equal to the undetermined multiplier of the entire image is set. In order to correspond a higher code amount allocation priority to a small block having a higher reference level, for example, a constraint condition in which the line graph shown in FIG.
[0032]
In order to convert a certain degree of reference to an undetermined multiplier, a table in which the degree of reference and the Lagrange's undetermined multiplier are associated with each other may be used. In this case, when the reference degree 07 has digits after the decimal point, the size of the table can be suppressed by rounding off to an appropriate digit. Using this correspondence information, the priority information determiner 09 determines the code amount allocation priority (Lagrange's undetermined multiplier) for all small blocks.
[0033]
Based on this information and the stored motion vector information from the memory 05, the in-loop encoder 10 encodes the input image signal 02 again. That is, if the Lagrange's undetermined multiplier given from the priority information 09 is λ _B for a small block, the Lagrange cost function L for that block
L = R _B + λ _B E _B
A quantization parameter that minimizes (where R is the number of bits of the small block and E is the decoding error of the small block) is obtained, for example, brute force. The bit stream output 11 is overwritten and stored on the storage medium 12.
[0034]
Further, based on the code amount R13 and the total noise amount E14 for the entire image sequence output from the encoder 10, the cost evaluation unit 15 sets the Lagrange cost function L to L = R + λ ₀ E
Asking. When this value is minimized, the amount of noise with respect to the code amount is minimized. Thereafter, the convergence determination unit 16 performs convergence determination. For example, if it is determined that the convergence condition such that no change is seen in the Lagrangian cost function L is satisfied, the process is terminated.
[0035]
Otherwise, the parameter adjustment unit 03 again adjusts the correspondence between the reference degree 07 and the code amount allocation priority (actually, the control point in FIG. 4 is moved, or the reference degree and Lagrange's undetermined multiplier are paired. 1), the encoding / evaluation / determination loop is repeated. The bit stream stored in the storage medium 12 at the end of the process becomes a desired optimum bit stream.
[0036]
In this embodiment, the encoder 10 in the loop does not perform motion search, so that the encoding is performed relatively fast. In this embodiment, the image encoder in the loop performs encoding with fixed selection of a motion vector and a small block encoding mode (bidirectional prediction / forward prediction / skip, etc.). A configuration in which the mode selection part is placed in a loop is also possible. In this case, if the convergence determination in the convergence determination unit 16 is “No”, the control shifts to immediately before instead of immediately after the motion detector 01.
[0037]
It goes without saying that each processing procedure shown in FIG. 3 can be executed by a computer, and a program for causing the computer to execute the processing procedure is recorded on a recording medium that can be read by the computer, such as an FD (floppy disk). ), An MO, a ROM, a memory card, a CD, a DVD, a removable disk, etc., provided, and distributed.
[0038]
【The invention's effect】
As described above, according to the present invention, it is possible to perform more detailed and optimum code amount allocation than setting parameters for each frame type. Also, parameter adjustment / optimization is performed according to the degree of reference of small blocks, so that different code amount allocation is realized for each small block, but optimization search is performed by associating parameters with all small blocks in the image sequence. Compared with, the amount of calculation is significantly reduced.
[Brief description of the drawings]
FIG. 1 is an explanatory diagram showing a state of voting to an adder when one adder corresponds to a small block.
FIG. 2 is an explanatory diagram illustrating a flow of bottom-up voting operation and addition processing.
FIG. 3 is a processing procedure diagram of an encoding method showing an embodiment of the present invention.
FIG. 4 is a correspondence graph between a referenced level and an encoding priority according to the embodiment of the present invention.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 01 ... Motion detector 02 ... Input image signal 03 ... Parameter adjustment part 04 ... Coding mode and motion vector information 05 ... Memory 06 ... Voter 07 ... Referenced degree 08 ... Memory 09 ... Priority information determiner 10 ... Encoding Device 12 ... Storage medium 15 ... Cost evaluation unit 16 ... Convergence determination unit

Claims (6)

In a video signal encoding method in which an image is divided into small blocks, inter-frame motion compensation is performed, and the same image is encoded multiple times.
A procedure for converting the number of references from other frames to be encoded using motion vector information into numerical values (hereinafter referred to as “referenced degree”);
This is a procedure for determining the code amount allocation of a small block to be encoded using the reference degree,
A procedure for using the motion vector information to make a reference to some of the references from other frames of the small block to be encoded,
For each small block that is a coding unit, a process of voting a predetermined amount according to a predicted macroblock of the small block to an area referred to by the small block in the reference image frame;
A process of adding the values voted for each area and setting the result as a reference degree of the area;
A process of repeating this process in the order of image frames referenced from image frames that are not referenced any more;
A repetitive video signal encoding method characterized by comprising: The repetitive video signal encoding method according to claim 1,
A procedure for converting a referenced degree into an undetermined multiplier using a table that associates the referenced degree and the Lagrange's undetermined multiplier in a one-to-one correspondence when determining the code amount allocation of the encoding target small block using the referenced degree; ,
A procedure for obtaining a Lagrangian cost function L of a small block to be encoded from a given undetermined multiplier λ _B from the following equation:
A procedure for obtaining a quantization parameter that minimizes the obtained Lagrangian cost function L;
A repetitive video signal encoding method characterized by comprising:
L = R _B + λ _B E _B
(Where R is the number of bits of the small block, E is the decoding error of the small block) The repetitive video signal encoding method according to claim 1,
A procedure for converting a referenced degree into an undetermined multiplier using a table that associates the referenced degree and the Lagrange's undetermined multiplier in a one-to-one correspondence when determining the code amount allocation of the encoding target small block using the referenced degree; ,
For each small block, according to the Lagrange's undetermined multiplier corresponding to the reference degree of the small block, a procedure for obtaining a quantization parameter that minimizes the Lagrangian cost function;
A procedure for obtaining a Lagrangian cost function L of the entire image sequence from the following equation from the code amount and the total noise of the entire image sequence:
A procedure for evaluating the obtained Lagrangian function L and determining whether the amount of noise with respect to the code amount is minimum;
A repetitive video signal encoding method characterized by comprising:
L = R + λ ₀ E
(Where R is the number of bits of the small block, E is the decoding error of the small block, and λ ₀ is an undetermined multiplier for the entire image) A recording medium recording a program for executing the iterative video signal encoding method according to claim 1 on a computer,
A procedure for converting the number of references from other frames to be encoded using motion vector information into numerical values (hereinafter referred to as “referenced degree”);
A procedure for determining a code amount allocation of a small block to be encoded using the dereferenced degree, and a procedure for converting the reference from other frames to the small block to be encoded using the motion vector information. But,
For each small block that is a coding unit, a process of voting a predetermined amount according to a predicted macroblock of the small block to an area referred to by the small block in the reference image frame;
A process of adding the values voted for each area and setting the result as a reference degree of the area;
A process of repeating this process in the order of image frames that are referenced from image frames that are not referenced any more.
A recording medium recording a program for a repetitive video signal encoding method, wherein the program is recorded on a recording medium readable by the computer as a program to be executed by a computer. A recording medium recording a program for executing the iterative video signal encoding method according to claim 2 on a computer,
A procedure for converting a referenced degree into an undetermined multiplier using a table that associates the referenced degree and the Lagrange's undetermined multiplier in a one-to-one correspondence when determining the code amount allocation of the encoding target small block using the referenced degree; ,
A procedure for obtaining a Lagrangian cost function L of a small block to be encoded from a given undetermined multiplier λ _B from the following equation:
A procedure for obtaining a quantization parameter that minimizes the obtained Lagrangian cost function L;
A recording medium recording a program for a repetitive video signal encoding method, wherein the program is recorded on a recording medium readable by the computer as a program to be executed by a computer.
L = R _B + λ _B E _B
(Where R is the number of bits of the small block, E is the decoding error of the small block) A recording medium recording a program for executing the iterative video signal encoding method according to claim 3 on a computer,
A procedure for converting a referenced degree into an undetermined multiplier using a table that associates the referenced degree and Lagrange's undetermined multiplier in a one-to-one correspondence when determining the code amount allocation of a small block to be encoded using the referenced degree;
For each small block, according to the Lagrange's undetermined multiplier corresponding to the reference degree of the small block, a procedure for obtaining a quantization parameter that minimizes the Lagrangian cost function;
A procedure for obtaining a Lagrangian cost function L of the entire image sequence from the following equation from the code amount and the total noise of the entire image sequence:
A procedure for evaluating the obtained Lagrangian function L and determining whether the amount of noise with respect to the amount of codes is minimum;
A recording medium recording a program for a repetitive video signal encoding method, wherein the program is recorded on a recording medium readable by the computer as a program to be executed by a computer.
L = R + λ ₀ E
(Where R is the number of bits of the small block, E is the decoding error of the small block, and λ ₀ is an undetermined multiplier for the entire image)

Images