KR20100010125A - Video coding method using multiple reference frame, computer readable medium recording the method and appraratus therefor - Google Patents

Abstract

PURPOSE: A video encoding method using small number of reference frames, a computer readable recording medium in which the method is recorded, and a device thereof is provided to minimize complexity by efficiently reducing the number of reference frames searched when predicting a movement. CONSTITUTION: A P16×16 mode searcher(10) determines an optimal reference frame about P16×16 mode of current macroblock for encoding. A candidate determination unit(20) primarily determines a candidate reference frame for searching a lower mode of the current macroblock according to information of the optimal reference frame. A final selection unit(30) selects a final reference frame in the determined candidate reference frame based on distribution of the optimal reference frame. A lower mode search unit(40) searches the low mode of the final reference frame.

Description

VIDEO CODING METHOD USING MULTIPLE REFERENCE FRAME, COMPUTER READABLE MEDIUM RECORDING THE METHOD AND APPRARATUS THEREFOR}

The present invention relates to a video encoding method using multiple reference frames, a computer-readable recording medium on which the method is recorded, and an apparatus thereof.

Recently, the compression transmission and encoding technology of digital video is rapidly progressing with the development of communication technology such as broadcasting and internet. In particular, various specifications have been proposed in the field of compression or encoding of digital video, and various efforts have been made to improve performance.

New video coding jointly established by the Joint Video Team (JVT), recently formed by the ITU-T Video Coding Experts Group (VCEG) and the ISO / IEC Motion Picture Expert Group (MPEG). As a standard, H.264 / AVC has been proposed.

H.264 applies a variety of new coding techniques to achieve high coding performance compared to previous standards such as MPEG-2, MPEG-3, and H.263. For example, methods such as quarter-pel accuracy motion prediction, use of multiple reference frames, coding using a variable block size, optimization of rate distortion, and the like have great advantages in inter-screen coding. These coding schemes show an excellent improvement in terms of rate-distortion performance compared to previous coding standards, but also have a disadvantage in that complexity increases excessively.

In particular, the H.264 / AVC standard uses several frames as reference images, as shown in FIG. This shows excellent performance when the movement occurs repeatedly or when an area is hidden by other objects, and the compression efficiency can be maximized.

In addition, unlike conventional video coding standards for motion prediction only in 16 × 16 or 8 × 8 size, as shown in FIG. 2, the H.264 / AVC standard makes the size of the block more varied to 16 × 16. The motion prediction is performed in units of 16 × 8, 8 × 16, 8 × 8, 8 × 4, 4 × 8, and 4 × 4 units. In particular, motion prediction performed with submacroblocks of 8x8 block size or less enables higher coding gain in complex images.

However, these seven variable block size motion prediction methods of H.264 / AVC have a problem of increasing complexity compared to the previous standard. In addition, the calculation amount is further increased by using multiple reference frames for motion prediction. However, despite the increase in the amount of calculation, the decrease in the prediction error is rather related to the characteristics of the image. Therefore, in some cases, the prediction gain that can be obtained by using more reference frames may be large, but otherwise, only a computation amount is wasted without obtaining a satisfactory gain.

Accordingly, an object of the present invention is to provide a video encoding method using multiple reference frames that minimizes complexity by efficiently reducing the number of reference frames to be searched during motion prediction, and a computer-readable recording medium on which the method is recorded and an apparatus thereof. It is.

According to an aspect of the present invention, there is provided a video encoding method using multiple reference frames, the method comprising: determining an optimal reference frame for a P16 × 16 mode of a current macroblock to be encoded; And adaptively determining the number of reference frames to be searched for the sub-mode of the current macroblock according to the information of the optimal reference frame determined in the P16x16 mode. It can be achieved by the video encoding method used.

In the determining of the number of reference frames, a reference frame to be searched by the sub-mode may be determined from a previous frame to the optimal reference frame for the P16 × 16 mode.

And finally selecting a reference frame from among the reference frames determined for the sub-mode based on a distribution of optimal reference frames of adjacent macroblocks of the current macroblock; And searching for the submode with respect to the finally selected reference frame.

The step of finally selecting the reference frame may include dividing the neighboring macroblocks into 8 × 8 sizes and dividing the neighboring macroblocks into 16 adjacent blocks; Counting the number of adjacent blocks belonging to each of the reference frames based on the optimal reference frame information of the sixteen adjacent blocks; And comparing the number of adjacent blocks belonging to each of the reference frames to finally select a reference frame for the sub-mode.

In addition, the finally selected reference frame includes an optimal reference frame of the P16 × 16 mode; The comparing of the number of adjacent blocks may include: multiplying the number of adjacent blocks having the optimal reference frame of the P16 × 16 mode as the optimal reference frame by a specific constant and the number of the adjacent blocks having the previous frame as the optimal reference frame. Comparing and determining whether to finally include the previous frame in the reference frame; And performing a comparison from the previous frame to the frame immediately before the optimal reference frame in the P16 × 16 mode according to the following equation to determine whether to finally include the reference frame in the reference frame:

Ref _{i +1} ∈CRF, if λH (Ref _i ) <H (Ref _{i +1} ), (i = 0 to N-2)

Here, Ref means a reference frame, Ref _N means the optimal reference frame of the P16 × 16 mode, λ is the specific constant value, H (x) frequency, CRF is the final reference frame set box)

Meanwhile, according to the present invention, there is provided a video encoding method using multiple reference frames, comprising: determining an optimal reference frame for a P16 × 16 mode of a current macroblock to be encoded; Selecting at least one of a frame from the previous frame to the optimal reference frame determined in the P16 × 16 mode as a reference frame for the sub-mode, based on a distribution of optimal reference frames of adjacent macroblocks of the current macroblock; And performing a search for the sub-mode with respect to the selected reference frame. The method may also be achieved by a video encoding method using multiple reference frames.

Meanwhile, according to the present invention, the above object can be achieved by a computer-readable recording medium storing a video encoding method using multiple reference frames according to any one of the above.

Meanwhile, according to the present invention, there is provided a video encoding apparatus using multiple reference frames, comprising: a P16 × 16 mode search unit for determining an optimal reference frame for a P16 × 16 mode of a current macroblock to be encoded; A candidate determiner for first determining a candidate reference frame to be searched for the sub-mode of the current macroblock adaptively according to the information of the optimal reference frame determined in the P16x16 mode; A final selection unit which finally selects a candidate reference frame among the determined candidate reference frames based on a distribution of optimal reference frames of adjacent macroblocks of the current macroblock; And a sub-mode search unit for searching for the sub-mode with respect to the selected final reference frame.

Further, the candidate determiner determines the candidate reference frame for the lower mode from the previous frame to the optimal reference frame for the P16 × 16 mode, and the final selector divides the adjacent macroblocks into 8 × 8 sizes. By dividing the data into 16 neighboring blocks, counting the number of neighboring blocks belonging to each of the candidate reference frames based on the optimal reference frame information of the 16 neighboring blocks, and comparing the number of neighboring blocks belonging to each of the candidate reference frames. By doing so, the candidate reference frame for the lower mode can be finally selected.

Wherein the finally selected reference frame includes an optimal reference frame of the P16 × 16 mode; The last selector compares the number of adjacent blocks having the previous frame as the best reference frame by multiplying the number of adjacent blocks having the best reference frame of the P16 × 16 mode as the best reference frame and the number of adjacent blocks having the previous reference as the best reference frame. Finally, it is determined whether to include the candidate reference frame, and a comparison is performed from the previous frame to the frame immediately before the optimal reference frame in the P16 × 16 mode according to the following equation to finally attach the candidate reference frame to the candidate reference frame. You can decide whether to include:

Ref _{i +1} ∈CRF, if λH (Ref _i ) <H (Ref _{i +1} ), (i = 0 to N-2)

Here, Ref means a reference frame, Ref _N means the optimal reference frame of the P16 × 16 mode, λ is the specific constant value, H (x) frequency, CRF is the final candidate reference frame set Meaning).

As described above, according to the present invention, a video encoding method using multiple reference frames that minimizes complexity by efficiently reducing the number of reference frames to be searched during motion prediction, and a computer-readable recording medium on which the method is recorded An apparatus is provided.

Hereinafter, specific embodiments of the present invention will be described with reference to the drawings.

The motion prediction process of the H.264 / AVC standard separates 16 × 16 macroblocks into 16 × 16, 16 × 8, 8 × 16, 8 × 8, 8 × 4, 4 × 8, and 4 × 4 shapes. All of them are performed over one block to determine an optimal mode and an optimal reference frame. In addition, motion prediction is performed in order from large blocks to small blocks such as 16 × 16, 16 × 8, 8 × 16, 8 × 8, 8 × 4, 4 × 8, and 4 × 4. Although the motion prediction process is performed for each block, even if it is a separate block, since the block is eventually part of a 16 × 16 macroblock, the P16 × 16 mode and each sub-mode are more likely to select the same optimal reference frame. It can be seen as high. Therefore, the finally determined optimal reference frame will also show close correlation with the optimal reference frame in P16x16 mode.

In order to verify the characteristics of this optimal reference frame, simulations were performed with H.264 reference software (JM 11.0) using three CIF resolution test images (Coastguard, Mobile, Flower). The number of reference frames is set to 5, the quantization parameter is set to 28, the search range is set to 32, the global search method is used for motion prediction, and a total of 100 frames are encoded. The experimental results are shown in Tables 1 to 3 below for each test image. Table 1 shows the optimal reference frame association between the modes of the Coastguard picture, Table 2 shows the optimal reference frame association between the modes of the Mobile picture, and Table 3 shows the optimum reference frame association between the modes of the Flower picture.

As can be seen from the above tables, the optimal reference frame determined in the P16 × 16 mode has a close relationship with the optimal reference frame finally determined for the block, which shows a consistent result in all images. That is, as a result of the experiment, when the reference frame determined in the P16 × 16 mode is Ref0 (previous frame), the most likely reference frame determined for the block is Ref0.

These experimental results indicate that most of the many reference frames used in the motion prediction process are not searched for the optimal reference frame but are searched for uselessly, resulting in a waste of computation. Accordingly, the present invention proposes a method of adaptively reducing the number of unnecessary reference frames searched using the optimal reference frame information determined in the P16x16 mode based on such statistical information.

For example, when a search for the P16x16 mode of the current macroblock is performed, and Ref2 is determined to be an optimal reference frame, the submodes P16x8, P8x16, P8x8, P8x4, In the P4 × 8 and P4 × 4 modes, the number of reference frames to be searched may be reduced to Ref0 to Ref2, that is, three. Detailed description thereof will be continued in the following embodiments.

On the other hand, the video signal is closely related to neighboring blocks due to its characteristics. Therefore, it may be effective to use information of neighboring blocks in encoding a specific block. FIG. 3A illustrates an optimal mode determined as a result of motion prediction of the 88 th frame of the cost guard test image QCIF, and FIG. 3B illustrates an optimal reference frame thereof. The encoder was set to 32 quantization parameters, 16 search ranges, and 5 reference frames. As shown in FIG. 3B, the optimal reference frame determined in the motion prediction tends to appear spatially concentrated when the entire reference image is examined. This tendency indicates that reference frames are closely related to each other. The present invention proposes a method for effectively reducing the number of reference frames of a current block by using reference frame information of neighboring blocks of a block to be encoded based on the relationship between adjacent blocks.

Hereinafter, specific embodiments of the present invention will be described with reference to the drawings. 4 is a schematic control block diagram of a video encoding apparatus using multiple reference frames according to an embodiment of the present invention.

As shown in FIG. 4, the video encoding apparatus using the multiple reference frames according to the present embodiment includes a P16 × 16 mode search unit 10, a candidate determiner 20, a final selector 30, and a lower mode search. A portion 40 is included.

The P16x16 mode search unit 10 searches all the available reference frames in the same manner as the conventional motion prediction method to determine an optimal reference frame of the P16x16 mode.

The candidate determination unit 20 may select a lower mode (for example, P16 × 8, P8 × 16, P8 × 8, or P8 × 4 based on an optimal reference frame of the P16 × 16 mode determined by the P16 × 16 mode search unit 10. , Reference frames (hereinafter also referred to as candidate reference frames) to be searched for P4x8 and P4x4 modes are determined. For example, if the immediately preceding frame Ref0 is determined to be an optimal reference frame in the P16x16 mode, the candidate determiner 20 determines only one reference frame Ref0 as a candidate reference frame in the lower mode. Also, in the P16 × 16 mode, if the previous frame Ref1 is determined to be an optimal reference frame, the lower mode has two reference frames Ref0 and Ref1 as candidate reference frames.

As described above, this is based on the statistics that the optimal reference frame determined in the P16 × 16 mode is likely to be the optimal reference frame of the current macroblock, and is adaptively lowered according to the optimal reference frame information determined in the P16 × 16 mode. The complexity of the motion prediction process can be greatly reduced by reducing the number of reference frames for the mode.

The final selector 30 further reduces the number of reference frames determined by the candidate determiner 20 by using reference frame information of neighboring macroblocks of the current macroblock. In other words, the final selector 30 finally selects the candidate reference frame from the candidate reference frames determined by the candidate determiner 20 by using the reference frame information of adjacent macroblocks of the current macroblock.

5 illustrates adjacent blocks of the current macroblock used in the final selector 30. The hatched regions in FIG. 5 represent previously encoded neighboring macroblocks that are available in the current macroblock. In the present embodiment, since the adjacent macroblocks may vary in size between modes, they are divided into 8 × 8 sizes, which are the minimum units of the optimal reference frame determination, and are expressed as 16 blocks.

The final selector 30 generates a histogram by applying optimal reference frame information of 16 adjacent blocks of FIG. 5 to the following equation.

Here, i is the index (0, ... N) of the candidate reference frames determined by the candidate determining unit 20, j is the index (0, .. when the adjacent macroblock is divided into 8 × 8 size in FIG. (15), N each represent an index of the optimal reference frame determined in the P16x16 mode.

Through the above equation, the optimal reference frame information of the neighboring block that does not belong to the candidate reference frame is discarded, and only the one belonging to the candidate reference frame is counted to generate a histogram.

6 illustrates an example of a histogram according to an optimal reference frame distribution of adjacent blocks generated by the final selector 30. In FIG. 6, the number of candidate reference frames is 4 (ie, the optimal reference frame determined in the P16 × 16 mode is Ref3).

The final selector 30 selects a final candidate reference frame among candidate reference frames by using the histogram. In this way, the number of reference frames used in the remaining modes (that is, the lower mode) except for the P16x16 mode of the current macroblock can be further reduced.

First, we need to reorder the histogram. Since the optimal reference frame determined in the P16x16 mode of the current macroblock is most likely to appear in the neighboring macroblock, the corresponding histogram item H (Ref ₃ ) is moved to the left in FIG. 6. The histogram rearranged accordingly is shown in FIG. 7.

 As can be seen in FIG. 7, the order of each item is arranged to be generally smaller and smaller in order from left to right through the reordering of the histogram.

First, the final selector 30 selects Ref3, which is the optimal reference frame determined in the P16x16 mode, as the final candidate reference frame.

Next, the final selector 30 sequentially compares the frequency of each item in the order arranged in the histogram to select a frame to be included in the final candidate reference frame set among the remaining candidate reference frames. Specifically, the value obtained by multiplying the histogram frequency H (Ref ₃ ) corresponding to the optimal reference frame determined in the P16 × 16 mode by a specific constant λ less than 1 is compared with the frequency H of the right item, H (Ref ₀ ). .

If the former is smaller, the reference frame (Ref0 in this example) corresponding to the right item is included in the final candidate reference frame (CRF).

The above comparison is expressed as a formula.

if λ · H (Ref _N ) <H (Ref ₀ ), Ref ₀ ∈ CRF

If the former is smaller, the same process is performed in order of H (Ref ₀ ), H (Ref ₁ ), H (Ref ₁ ), and H (Ref ₂ ).

The above comparison is expressed as a formula.

if λ · H (Ref _i ) <H (Ref _{i +1} ), Ref _{i +1} ∈ CRF, i = 0 ~ N-2

However, if the former is larger than the latter, the comparison process is terminated, and the frame corresponding to the remaining item on the right side is removed from the candidate reference frame set. That is, only the frame corresponding to the item on the left side is selected as the final candidate reference frame.

Here, the lambda value can be appropriately adjusted in consideration of the image quality performance, and was determined to be 0.35 through experiments.

When the candidate reference frame is finally determined through this process, the lower mode search unit 40 performs motion estimation on the lower mode using the reference frame.

8A and 8B are flowcharts illustrating a video encoding method by a video encoding apparatus using multiple reference frames according to an embodiment of the present invention described above.

Referring to FIG. 8A, first, motion prediction is performed in the entire reference frame for the P16 × 16 mode of the current macroblock (S10). For example, when the number of all reference frames is set to five, a search for the P16 × 16 mode is performed using the five reference frames.

As a result, an optimal reference frame N for the P16x16 mode is determined (S11). For example, if the optimal reference frame N for the P16x16 mode is determined to be 3, the number of candidate reference frames is four (Ref ₀ to Ref ₃ ).

As described above, when the primary candidate reference frame is determined, the candidate reference frame is finally determined by using optimal reference frame information of neighboring blocks of the current macroblock.

As shown in FIG. 5, four adjacent macroblocks are divided into 8 × 8 sizes, and H (Ref _i ) is calculated for each candidate reference frame according to Equation 1 above to generate a histogram (S12).

First, RefN is selected as a final candidate reference frame (S13).

Then, it is determined whether to include the Ref0 frame in the final candidate reference frame according to Equation 2 (S14). If the equation is satisfied, the frame is determined as the final candidate reference frame (S15).

However, if the equation is not satisfied, the final candidate reference frame is determined only by RefN. At this time, the motion prediction for the lower mode is performed only for the selected RefN (S21), and accordingly, the optimum mode and the optimal frame for the current macroblock are determined (S22).

On the other hand, when the Ref0 frame is selected as the final candidate reference frame, the final candidate reference frame is selected by comparing the frequencies according to Equation 3 from i = 0 to N-2.

First, i = 0 is set (S16), and comparison is performed according to equation (3) (S17). If Equation 3 is satisfied, the frame corresponding to Ref _{i + 1} (that is, Ref1 in this case) is selected as the final candidate reference frame (S18). This process continues from i = 0 to N-2, and if i = N-2 is satisfied, the repetitive comparison process ends (S19, S20).

That is, if Equation 2 or Equation 3 is not satisfied or i = N-2 is satisfied, the candidate reference frame corresponding to the item located on the right side is removed, and motion prediction for the lower mode is performed only for the candidate reference frame already selected. In operation S21, an optimal mode and an optimal frame for the current macroblock are determined according to the motion prediction result (S22).

The pseudo code of the algorithm according to the encoding method described above may be expressed as follows.

As described above, the present invention adaptively reduces the number of reference frames for the lower mode by using optimal reference frame information for the P16 × 16 mode of the current macroblock, and uses the optimal reference frame information of neighboring blocks. The reference frame is finally determined again among the candidate reference frames.

Hereinafter, a result of comparing the encoding method of the present invention and the conventional encoding method through experiments will be described. In this experiment, five test images (News, Foreman, Table, Coastguard, Stefan) with CIF (352 × 288) resolution were encoded using a baseline profile up to 100 frames. The main experimental conditions are as follows. The frame rate was set to 30. Each picture was encoded with only the first frame as an I picture, and the remaining frames were encoded with a P picture. As a motion estimation method, a global search method was used.

In addition, the search range was set to 32, the number of reference frames was set to 5, and the inter screen mode was used. The quantization parameter used four values of 24, 28, 32, 36, and applied rate-distortion optimization. This experiment was performed on a Pentium IV 2.8GHz, 1GB system.

Table 5 below compares the objective picture quality (PSNR) of the conventional JM 11.0 and the coding method according to the present invention.

As can be seen from the above table, it can be seen from the perspective of PSNR that the encoding method according to the H.264 standard and the present invention has negligible image quality difference. The encoding method according to the present invention shows an image quality degradation of about 0.033 dB on average compared to JM 11.0.

Table 6 below shows a comparison result in terms of bit rate between the encoding method and the H.264 standard according to the present invention.

As can be seen from the above table, similar to the PSNR result, the bit rate difference of the encoding method according to the present invention is also almost negligible. The encoding method according to the present invention shows an average bit rate increase of 1.40% compared to JM 11.0.

Table 7 below is a result of comparing the motion prediction time of JM 11.0 and the encoding method according to the present invention.

As can be seen from the above table, it can be seen that by using the encoding method according to the present invention, considerable motion prediction time can be saved compared to JM 11.0. An average of 57.31% of motion estimation time can be obtained, and up to 62% of motion prediction time can be saved.

9 shows the result of comparing the reference software in a table test image and the motion prediction time of the encoding method according to the present invention from frame 1 to frame 100 (QP = 24). This result also explicitly shows that the encoding method according to the present invention significantly reduces the motion estimation time compared with the conventional JM 11.0.

As described above, the encoding method according to the present invention shows an amazing result of reducing the encoding time by an average of 57.31% while having almost similar PSNR and bit rate as compared with the conventional H.264 standard. Therefore, the coding method according to the present invention is particularly applicable to a real-time system that requires low complexity.

The encoding apparatus according to the embodiment of the present invention described in the above embodiments may be implemented by an apparatus including a processor in which each component is implemented as a software algorithm and a memory in which the corresponding algorithm is executed. It is also applicable to all displays which process and display an image.

In the above-described embodiment, when the optimal reference frame determined in the P16x16 mode is determined as the optimal reference frame, for example, two frames before the frame Ref1 is determined as the optimal reference frame, the ranges of candidate reference frames for the lower mode are Ref0 and Ref1. It may be determined up to Ref2 in which one frame is added to the optimum reference frame determined in the P16 × 16 mode, and other examples are possible to those skilled in the art.

Although some embodiments of the invention have been shown and described, it will be apparent to those skilled in the art that modifications may be made to the embodiment without departing from the spirit or spirit of the invention. . It is intended that the scope of the invention be defined by the claims appended hereto and their equivalents.

1 shows an example of using multiple reference frames in the H.264 standard,

2 schematically illustrates an encoding mode for inter prediction of an H.264 standard.

FIG. 3A illustrates an optimal split mode of the cost guard image, and FIG. 3B illustrates an optimal reference frame determined in the encoding process of the cost guard image.

4 is a schematic control block diagram of an encoding apparatus using multiple reference frames according to an embodiment of the present invention;

5 is an example of an adjacent macroblock used in the present invention;

6 is an example of a histogram according to a distribution of reference frames of adjacent macroblocks.

FIG. 7 is an example of a histogram in which FIG. 6 is rearranged.

8A and 8B are a flowchart of an encoding method according to an embodiment of the present invention;

9 is a graph comparing motion estimation time for each frame between the encoding method and the conventional method according to the present invention.

Claims (10)

In the video encoding method using multiple reference frames, Determining an optimal reference frame for the P16 × 16 mode of the current macroblock to be encoded; And And determining the number of reference frames to be searched for the sub-mode of the current macroblock adaptively according to the information of the optimal reference frame determined in the P16x16 mode. Encoding method. The method of claim 1, The determining of the number of reference frames may include determining from the previous frame to the optimal reference frame for the P16 × 16 mode as a reference frame to be searched by the sub-mode. The method according to claim 1 or 2, Finally selecting a reference frame among the reference frames determined for the sub-mode, based on a distribution of optimal reference frames of adjacent macroblocks of the current macroblock; And And performing a search for the sub-mode with respect to the finally selected reference frame. The method of claim 3, The selecting of the reference frame may include dividing the adjacent macroblocks into 8 × 8 size into 16 adjacent blocks; Counting the number of adjacent blocks belonging to each of the reference frames based on the optimal reference frame information of the sixteen adjacent blocks; And And comparing the number of adjacent blocks belonging to each of the reference frames, and finally selecting a reference frame for the sub-mode. The method of claim 4, wherein The last selected reference frame includes an optimal reference frame in the P16 × 16 mode; The comparing of the number of adjacent blocks may include: multiplying the number of adjacent blocks having the optimal reference frame of the P16 × 16 mode as the optimal reference frame by a specific constant and the number of the adjacent blocks having the previous frame as the optimal reference frame. Comparing and determining whether to finally include the previous frame in the reference frame; And Performing a comparison from the previous frame to the frame immediately before the optimal reference frame in the P16 × 16 mode according to the following equation to determine whether to finally include the reference frame in the reference frame. Video encoding method using reference frame: Ref _{i +1} ∈CRF, if λH (Ref _i ) <H (Ref _{i +1} ), (i = 0 to N-2) Here, Ref means a reference frame, Ref _N means the optimal reference frame of the P16 × 16 mode, λ is the specific constant value, H (x) frequency, CRF is the final reference frame set box) In the video encoding method using multiple reference frames, Determining an optimal reference frame for the P16 × 16 mode of the current macroblock to be encoded; Selecting at least one of a frame from the previous frame to the optimal reference frame determined in the P16 × 16 mode as a reference frame for the sub-mode, based on a distribution of optimal reference frames of adjacent macroblocks of the current macroblock; And And searching for the sub-mode with respect to the selected reference frame. A computer-readable recording medium storing a video encoding method using multiple reference frames according to any one of claims 1 to 6. In the video encoding apparatus using multiple reference frames, A P16x16 mode search unit for determining an optimal reference frame for the P16x16 mode of the current macroblock to be encoded; A candidate determiner for first determining a candidate reference frame to be searched for the sub-mode of the current macroblock adaptively according to the information of the optimal reference frame determined in the P16x16 mode; A final selection unit which finally selects a candidate reference frame among the determined candidate reference frames based on a distribution of optimal reference frames of adjacent macroblocks of the current macroblock; And And a lower mode search unit for searching for the lower mode with respect to the selected final reference frame. The method of claim 8, The candidate determiner determines the candidate reference frame for the lower mode from the previous frame to the optimal reference frame for the P16 × 16 mode. The final selector divides the neighboring macroblocks into 8 × 8 sizes and divides the neighboring macroblocks into 16 neighboring blocks, and counts the number of neighboring blocks belonging to each of the candidate reference frames based on the optimal reference frame information of the 16 neighboring blocks. And finally selecting the candidate reference frame for the sub-mode by comparing the number of neighboring blocks belonging to each of the candidate reference frames. The method of claim 9, The last selected reference frame includes an optimal reference frame in the P16 × 16 mode; The last selector compares the number of adjacent blocks having the previous frame as the best reference frame by multiplying the number of adjacent blocks having the best reference frame of the P16 × 16 mode as the best reference frame and the number of adjacent blocks having the previous reference as the best reference frame. Finally, it is determined whether to include the candidate reference frame, and a comparison is performed from the previous frame to the frame immediately before the optimal reference frame in the P16 × 16 mode according to the following equation to finally attach the candidate reference frame to the candidate reference frame. A video encoding apparatus using multiple reference frames, characterized in that it is determined whether to include: Ref _{i +1} ∈CRF, if λH (Ref _i ) <H (Ref _{i +1} ), (i = 0 to N-2) Here, Ref means a reference frame, Ref _N means the optimal reference frame of the P16 × 16 mode, λ is the specific constant value, H (x) frequency, CRF is the final candidate reference frame set Means)

Images