RU2395174C1 - Method and device for decoding/coding of video signal - Google Patents

Abstract

FIELD: information technologies.

SUBSTANCE: method for decoding of video signal includes stages of video signal coding scheme, reception of configuration information for video signal according to coding scheme, recognition of total number of frames, using information of configuration, recognition of interframe reference information on the basis of total number of frames, and decoding of video signal on the basis of interframe reference information of frame, besides configuration information includes at least frame information for identification of video signal frame.

EFFECT: improved efficiency of coding, decoding of video image by means of video signal coding on the basis of interframe reference information.

8 cl, 49 dwg

Description

Technical field

The present invention relates to a method and apparatus for decoding / encoding a video signal.

State of the art

Compression coding means a number of signal processing methods for transmitting digitized information through a communication circuit or storing the digitized information in a form suitable for a data medium. Compression encoding objects include audio, video, text, and the like. In particular, a method for performing compression encoding with respect to a sequence is called video sequence compression. A video sequence is generally characterized by spatial redundancy and temporal redundancy.

Disclosure of invention

Technical challenge

An object of the present invention is to increase the coding efficiency of a video signal.

Technical solution

An object of the present invention is to encode a video signal in an efficient manner by determining frame information to identify a frame (presentation) of an image.

Another objective of the present invention is to increase the encoding efficiency by encoding a video signal based on inter-frame reference information.

Another objective of the present invention is to provide scalability of a frame (presentation) of a video signal by determining frame level information.

Another object of the present invention is to encode a video signal in an efficient manner by determining an inter-frame synthesis prediction identifier indicating whether to receive an image of a virtual frame.

Useful Results

When encoding a video signal, the present invention provides more efficient encoding by performing inter-frame prediction using frame (presentation) information to identify an image frame. And by redefining the level information indicating information for the hierarchical structure to provide scalability of the frame, the present invention is able to provide a frame sequence suitable for the user. Furthermore, by defining a frame corresponding to the lowest level as a reference frame, the present invention provides compatibility with a conventional decoder. In addition, the present invention improves coding efficiency by determining whether to predict an image of a virtual frame when performing inter-frame prediction. In the case of image prediction of a virtual frame, the present invention provides more accurate prediction, thereby reducing the number of bits to be transmitted.

Brief Description of the Drawings

1 is a schematic block diagram of an apparatus for decoding a video signal according to the present invention.

2 is a diagram of configuration information for a multi-frame video added to an encoded bitstream of a multi-frame video according to an embodiment of the present invention.

FIG. 3 is an internal block diagram of a reference image list creating unit 620 according to an embodiment of the present invention.

4 is a diagram of a hierarchical structure of level information for providing scalability of a video signal frame according to an embodiment of the present invention.

5 is a configuration diagram of a NAL unit including layer information within an extension area of a NAL header according to one embodiment of the present invention.

6 is a diagram of a complete prediction structure of a multi-frame video signal according to an embodiment of the present invention, to describe the concept of a group of inter-frame images.

7 is a diagram of a prediction structure according to an embodiment of the present invention for explaining the concept of a newly defined group of inter-frame images.

FIG. 8 is a schematic block diagram of an apparatus for decoding a multi-frame video using identification information of an inter-frame image group according to an embodiment of the present invention.

9 is a flowchart of a process for constructing a list of reference images according to an embodiment of the present invention.

10 is a diagram for explaining a method of initializing a reference picture list when the current cut is a P-cut according to one embodiment of the present invention.

11 is a diagram for explaining a method of initializing a reference picture list when the current cut is a B-cut according to one embodiment of the present invention.

12 is an internal block diagram of a reference picture list reordering unit 630 according to an embodiment of the present invention.

13 is an internal block diagram of a reference index assignment change module 643B or 645B according to one embodiment of the present invention.

14 is a diagram for explaining a process for reordering a list of reference images using frame information according to one embodiment of the present invention.

15 is an internal block diagram of a reference picture list reordering unit 630 according to another embodiment of the present invention.

FIG. 16 is an internal block diagram of a reference picture list reordering unit 970 for inter-frame prediction according to an embodiment of the present invention.

FIG. 17 and FIG. 18 are syntax diagrams for reordering the reference picture list according to one embodiment of the present invention.

FIG. 19 is a syntax diagram for reordering a list of reference pictures according to another embodiment of the present invention.

FIG. 20 is a diagram for a process for obtaining a lighting difference value of a current block according to one embodiment of the present invention.

21 is a flowchart of a process for performing lighting compensation of a current unit according to an embodiment of the present invention.

FIG. 22 is a process diagram for obtaining a prediction value of a lighting difference of a current block using information for a neighboring block, according to one embodiment of the present invention.

23 is a flowchart of a process for performing lighting compensation using information for a neighboring unit according to one embodiment of the present invention.

24 is a flowchart of a process for performing lighting compensation using information for a neighboring unit according to another embodiment of the present invention.

25 is a process diagram for predicting a current image using an image in a virtual frame according to one embodiment of the present invention.

26 is a flowchart of a process for synthesizing an image in a virtual frame when performing interframe prediction in MVC according to an embodiment of the present invention.

Fig. 27 is a flowchart of a method for performing weighted prediction according to a clipping type when encoding a video signal according to the present invention.

FIG. 28 is a diagram of macroblock types valid in a clipping type when encoding a video signal according to the present invention.

FIG. 29 and FIG. 30 are syntax diagrams for performing weighted prediction according to a newly defined clipping type according to one embodiment of the present invention.

FIG. 31 is a flowchart of a method for performing weighted prediction using flag information indicating whether to perform inter-frame weighted prediction when encoding a video signal according to the present invention.

32 is a diagram for explaining a weighted prediction method according to flag information indicating whether to perform weighted prediction using information for an image in a frame different from that of the current image according to one embodiment of the present invention.

33 is a syntax diagram for performing weighted prediction according to newly defined flag information according to one embodiment of the present invention.

Fig. 34 is a flowchart of a method for performing weighted prediction according to a type of NAL unit (network abstraction layer) according to an embodiment of the present invention.

Fig. 35 and Fig. 36 are syntax diagrams for performing weighted prediction in the case where the NAL unit type is for multi-frame video encoding according to one embodiment of the present invention.

Fig. 37 is a partial block diagram of a video signal decoding apparatus according to a newly determined cut type according to an embodiment of the present invention.

Fig. 38 is a flowchart for explaining a method of decoding a video signal in the device shown in Fig. 37 according to the present invention.

Fig. 39 is a diagram of a macroblock prediction mode according to one embodiment of the present invention.

Fig. 40 and Fig. 41 are diagrams of a syntax having a clipping type and a macroblock mode applied to it according to the present invention.

Fig. 42 is a diagram of embodiments to which the cut types of Fig. 41 are applied.

FIG. 43 is a diagram of various embodiments of a clipping type included in the clipping types shown in FIG. 41.

Fig. 44 is a macroblock diagram valid for a mixed cut type in accordance with a prediction of two mixed predictions according to one embodiment of the present invention.

Figures 45 to 47 are diagrams of a macroblock type for a macroblock existing in a mixed notch in accordance with the prediction of two mixed predictions according to one embodiment of the present invention.

Fig. 48 is a partial block diagram of a video signal encoding apparatus according to a newly determined cut type according to an embodiment of the present invention.

Fig. 49 is a flowchart of a video signal encoding method in the device of Fig. 48 according to the present invention.

The best mode of carrying out the invention

In order to achieve these and other advantages, and in accordance with an object of the present invention, which is realized and described in detail, a video signal decoding method includes the steps of checking a video signal coding scheme, obtaining configuration information for the video signal according to the coding scheme, recognizing the total number of frames using configuration information, recognition of inter-frame reference information based on the total number of frames, and decoding a video signal based on inter-frame reference information ation, wherein the configuration information includes at least frame information for identifying a video frame signal.

To further obtain these and other advantages, and in accordance with an object of the present invention, a video signal decoding method includes the steps of: checking a video signal coding scheme, obtaining configuration information for a video signal according to a coding scheme, checking a level for scalability of a video signal frame from configuration information recognizing inter-frame reference information using configuration information, and decoding the video signal based on the level and inter-frame reference information, whereby, the configuration information includes frame information for identifying an image frame.

Mode for invention

Reference is made below to a detailed description of preferred embodiments of the present invention, examples of which are illustrated in the accompanying drawings.

A video signal compression and encoding technique considers spatial redundancy, temporal redundancy, scalable redundancy, and interframe redundancy. And, it is also possible to perform compression encoding, considering the mutual redundancy between frames in the compression encoding process. A compression coding technique that considers interframe redundancy is only an embodiment of the present invention. And the technical idea of the present invention is applicable to temporary redundancy, scalable redundancy, etc.

If we consider the bitstream configuration in H.264 / AVC, there is a separate layer structure called NAL (network abstraction layer) between the VCL (video coding layer), which deals directly with the encoding process of moving images and a lower (lower level) system that transports and stores encoded information. The output from the encoding process is VCL data and is displayed by the NAL unit before being transported or stored. Each NAL unit includes compressed video data or RBSP (raw byte sequence data: moving image compression result data), which are data corresponding to header information.

The NAL unit mainly includes a NAL header and an RBSP. The NAL header includes flag information (nal_ref_idc) indicating whether the notch is included as a reference image of the NAL unit, and an identifier (nal_unit_type) indicating the type of the NAL unit. The compressed original data is stored in RBSP. And, the RBSP tail bit is added to the last part of the RBSP to represent the RBSP length as an 8-bit multiplication. As the type of NAL unit, there is an IDR image (instant update of decoding), SPS (sequence parameter set), PPS (image parameter set), SEI (additional extension information), or the like.

During standardization, restrictions are set for various profiles and levels in order to allow the implementation of the target product with a suitable cost. In this case, the decoder must satisfy the restriction determined according to the corresponding profile and level. Thus, two concepts are defined - 'profile' and 'level' to indicate a function or parameter to represent - how much the decoder can handle the range of the compressed sequence. And, the profile indicator (profile_idc) can identify that the bitstream is based on the prescribed profile. Profile indicator means a flag indicating the profile on which the bitstream is based. For example, in H.264 / AVC, if the profile indicator is 66, this means that the bitstream is based on the base profile. If the profile indicator is 77, this means that the bitstream is based on the main profile. If the profile indicator is 88, this means that the bitstream is based on an extended profile. And, the profile identifier can be included in the sequence parameter set.

So, in order to deal with multi-frame video, it is necessary to identify whether the profile of the entered bitstream is a multi-frame profile. If the profile of the entered bitstream is a multi-frame profile, you must add syntax to allow the transmission of at least one additional information for multiple frames. In this case, the multi-frame profile indicates the profile mode processing multi-frame video as a technique for amending H.264 / AVC. In MVC, it may be more efficient to add syntax as additional information for MVC mode instead of unconditional syntax. For example, when a profile indicator for AVC indicates a multi-frame profile, if information for a multi-frame video is added, it is possible to increase coding efficiency.

The sequence parameter set indicates header information containing information collected by encoding the entire sequence, such as profile, level, and the like. Entirely compressed moving image, that is, the sequence should begin with the header of the sequence. So, the sequence parameter set corresponding to the header information must arrive at the decoder before the data referencing the parameter set arrives. Namely, the RBSP set of sequence parameters plays the role of header information for the resulting moving image compression data. Once a bitstream is entered, the profile indicator preferably identifies that the entered bitstream is based on one of a plurality of profiles. Thus, adding a part to determine whether the entered bitstream is related to a multi-frame profile (for example, 'IF (profile_idc == MULTI_VIEW_PROFILE)') to the syntax, a decision is made whether the entered bitstream is related to a multi-frame profile. Various kinds of configuration information can only be added if the entered bitstream is defined as being related to a multi-frame profile. For example, it is possible to add a series of full frames, a series of multi-frame reference images (List0 / 1) in the case of a group of inter-frame images, a series of multi-frame reference images (List0 / 1) in the case of a group of non-frame images, etc. And, various information for the frame is used to generate and manage the list of reference images in the decoded image buffer.

Figure 1 illustrates a schematic block diagram of an apparatus for decoding a video signal according to the present invention.

With reference to FIG. 1, a video signal decoding apparatus according to the present invention includes a NAL parser 100, a statistical decoding module 200, an inverse quantization / inverse transform module 300, an intra (intra) prediction module 400, a deblocking filter module 500, a buffer module 600 of decoded images, a module 700 inter (inter) prediction, etc.

The decoded image buffer module 600 includes a reference image storage unit 610, a reference image list creating unit 620, reference image managing unit 650, and the like. And, the reference image list creating module 620 includes a variable acquiring module 625, reference image list initialization module 630, and reference image list reordering module 640.

And, the inter prediction module 700 includes a motion compensation module 710, a lighting compensation module 720, a light difference prediction module 730, a frame synthesis prediction module 740, and the like.

The NAL parser 100 performs parsing by a NAL unit to decode the received video sequence. In general, at least one set of sequence parameters and at least one set of image parameters are transmitted to the decoder before the clipping header (part of the information array) and the clipping data are decoded. In this case, various kinds of configuration information may be included in the NAL header area or the extension area of the NAL header. Since MVC is a correction technique for the conventional AVC technique, it may be more efficient to add configuration information only in the case of the MVC bitstream instead of unconditionally adding. For example, it is possible to add flag information to identify the presence or non-presence of the MVC bitstream in the NAL header region or the NAL header extension region. Only if the input bitstream is a multi-frame video encoded bitstream according to the flag information, is it possible to add configuration information for the multi-frame video. For example, configuration information may include temporal level information, frame level information, interframe image group identification information, frame identification information, and the like. This is described in detail below with reference to FIG. 2.

FIG. 2 shows a diagram of configuration information for a multi-frame video added to an encoded multi-frame video bitstream according to one embodiment of the present invention. The details of the configuration information for multi-frame video are described in the following description.

First of all, temporal level information indicates information for a hierarchical structure to provide temporal scalability from a video signal ((1)). By means of temporal level information, it is possible to provide the user with sequences in relation to different time zones.

The frame level information indicates information for the hierarchical structure to provide scalability of the frame from the video signal ((2)). In multi-frame video, it is necessary to determine the level for time and the level for the frame in order to provide the user with different time and frame sequences. In the case of determining the above level information, it is possible to use temporal scalability and frame scalability. Therefore, the user is able to select a sequence at a specific time and frame, or the selected sequence may be limited by a condition.

Level information can be set in various ways according to a specific condition. For example, level information can be set differently according to the location of the camera (camera) or the alignment of the camera (camera). And, level information can be determined by considering the dependence of the frame. For example, the level for a frame having an I-image in an inter-frame image group is set to 0, the level for a frame having a P-image in an inter-frame image group is set to 1, and the level for a frame having a B-image in an inter-image group, set to 2. In addition, level information may not be accidentally set based on a special condition. Frame level information is described in detail with reference to FIG. 4 and FIG. 5 below.

The identification information of the inter-frame image group indicates information for identifying whether the encoded image of the current NAL unit is the inter-image image group ((3)). In this case, a group of inter-frame images means an encoded image in which all clippings refer only to clippings with the same serial index of the image. For example, a group of inter-frame images means an encoded image that refers to clippings only in a different frame, without reference to clippings in the current frame. In the decoding process, multi-frame video may require inter-frame random access. Identification information of a group of inter-frame images may be necessary in order to realize effective random access. And, inter-frame reference (reference) information may be necessary for inter-frame prediction. So, the identification information of a group of inter-frame images can be used to obtain inter-frame reference information. In addition, the identification information of a group of inter-frame images can be used to add reference images for inter-frame prediction when constructing a list of reference images. In addition, interframe image group identification information can be used to control the added reference images for interframe prediction. For example, reference pictures can be classified into inter-picture picture groups and non-picture picture groups, and classified reference pictures can then be marked so that reference pictures not used for inter prediction should not be used. Meanwhile, the identification information of the inter-frame image group is applicable to a hypothetical reference decoder. Details of the identification information of the inter-frame image group are described with reference to FIG. 6 below.

Frame identification information means information for distinguishing an image in the current frame from an image in an excellent frame ((4)). When encoding a video signal, POC (Image Index) or 'frame_num' can be used to identify each image. In the case of a multi-frame video sequence, inter-frame prediction may be performed. Therefore, identification information is needed to distinguish an image in the current frame from an image in another frame. So, it is necessary to determine the identification information of the frame to identify the image frame. This frame identification information can be obtained from the header area of the video signal. For example, the header area may be a NAL header area, a NAL header extension area, or a cut header area. Information for an image in a frame different from that of the current image is obtained using the identification information of the frame, and a video signal can be decoded using the image information in a different frame. The frame identification information is applicable to the entire video signal encoding / decoding process. And the frame identification information may be applicable to multi-frame video encoding using 'frame_num', which considers a frame instead of considering a specific frame identifier.

Meanwhile, the statistical decoding unit 200 performs statistical decoding on the syntactically parsed bitstream, and then extracts the coefficient of each macroblock, the motion vector, and the like. The inverse quantization / inverse transform unit 300 obtains the converted coefficient value by multiplying the received quantized value by a constant and then inverts the coefficient value to restore the pixel value. Using the reconstructed pixel value, the intra (intra) prediction unit 400 performs intra (intra) prediction from the decoded sample within the current image. Meanwhile, the deblocking filter module 500 is applied to each coded macroblock to reduce block distortion. The filter smooths the edge of the block to improve the image quality of the decoded frame. The choice of the filtering process depends on the level of the border and the gradient of the sample image around the border. Images through filtering are issued or stored in the buffer module 600 of the decoded images to be used as a reference image.

The decoded image buffer module 600 plays a role in storing or opening previously encoded images in order to perform inter (inter) prediction. In this case, to save images in the buffer module 600 of decoded images or to open images, the 'frame_num' and POC (image index) of each image are used. So, since there are images in a frame different from that of the current image among previously encoded images, the frame information for identifying the image frame can be used together with 'frame_num' and POC. The decoded image buffer module 600 includes a reference image storage unit 610, a reference image list creating unit 620, and reference image management unit 650. The reference image storage unit 610 stores images that will be referenced to encode the current image. The reference image list creating unit 620 constructs a reference image list for prediction between images. When encoding multi-frame video, inter-frame prediction may be necessary. So, if the current image refers to an image in another frame, it may be necessary to create a list of reference images for inter-frame prediction. In this case, the reference image list creating unit 620 may use the information for the frame when generating the reference image list for inter-frame prediction. Details of the reference image list creating unit 620 are described below with reference to FIG. 3.

FIG. 3 is an internal block diagram of a reference image list creating unit 620 according to an embodiment of the present invention.

The reference image list creating module 620 includes a variable acquisition (output) module 625, a reference image list initialization module 630, and a reference list reordering module 640.

The variable obtaining unit 625 receives (outputs) the variables used to initialize the reference image list. For example, a variable can be obtained using 'frame_num' indicating the identification number of the image. In particular, the FrameNum and FrameNumWrap variables can be used for each short-term reference image. First of all, the FrameNum variable is equal to the value of the syntax frame_num element. The FrameNumWrap variable may be used for the decoded image buffer module 600 to assign a small number to each reference image. And, the FrameNumWrap variable can be obtained from the FrameNum variable. So, it is possible to get the PicNum variable using the resulting FrameNumWrap variable. In this case, the PicNum variable may mean the identification number of the image used by the decoded image buffer module 600. If you specify a long-term reference image, the LongTermPicNum variable can be used.

In order to create a list of reference pictures for inter-frame prediction, it is possible to obtain (output) the first variable (for example, ViewNum) to create a list of reference pictures for inter-frame prediction. For example, you can get a second variable (for example, Viewld) using 'view_id' to identify the image frame. First of all, the second variable may be equal to the value of the syntax view_id element. And, a third variable (eg, ViewIdWrap) can be used for the decoded image buffer module 600 to assign a small frame identification number to each reference image and can be obtained from the second variable. In this case, the first variable ViewNum may mean the identification number of the image frame used by the buffer module 600 decoded images. However, since the number of reference images used for inter prediction in multi-frame video encoding may be relatively smaller than that used for temporal prediction, another variable may not be defined to indicate the frame identification number of the long-term reference image.

The reference image list initialization module 630 initializes the reference image list using the above variables. In this case, the initialization process for the list of reference images may differ according to the type of clipping. For example, in the case of decoding a P-cut, it is possible to assign a reference index based on the decoding order. In the case of decoding a B-cut, it is possible to assign a reference index based on the image output order. In the case of initializing the reference picture list for inter-frame prediction, it is possible to assign an index to the reference picture based on the first variable, that is, a variable obtained from the frame information.

The reference image list reordering module 640 plays a role in increasing compression efficiency by assigning a lower index to the image that is often referenced in the initialized reference image list. This is because a small bit is assigned if the reference index for encoding becomes smaller.

And, the reference image list reordering module 640 includes a clipping type checking module 642, the reference image list reordering module 643, and the reference image list reordering module 645. If an initialized reference picture list is entered, the clipping type checking module 642 checks the type of clipping to be decoded, and then decides whether to reorder the list-0 of reference images or the list-1 of reference images. So, the reordering module 643, 645 of the reference image list 0/1 performs the reordering of the reference image list 0 if the clipping type is not I-clipping, and also performs the reordering of the reference image list 1 further if the clipping type is B-clipping. Thus, after completion of the reordering process, a list of reference images is created.

The reordering unit 643, 645 of the reference image list 0/1 includes an identification information obtaining unit 643A, 645A and a reference index assignment changing unit 643B, 645B, respectively. The identification information obtaining unit 643A, 645A receives identification information (reordering_of_pic_nums_idc) indicating a reference index assignment method if the reordering of the reference picture list is performed according to flag information indicating whether to reorder the reference picture list. And, the reference index assignment change module 643B, 645B reorders the reference picture list by changing the reference index assignment according to the identification information.

And, the reference picture list reordering module 640 works in a different way. For example, reordering can be performed by checking the type of the NAL block passed before passing through the type checking clipping module 642, and then by classifying the type of the NAL block in the case of MVC NAL and the case of non-MVC NAL.

The reference image control unit 650 controls the reference images to perform inter prediction more flexibly. For example, an operational memory management method and a moving window method are used. This should provide control of the memory of the reference images and the memory of non-reference images by combining the memory blocks into one memory and to realize effective memory management with a small memory. When encoding a multi-frame video, since the images in the frame direction have the same image index, information for identifying the frame of each of the images is suitable for use in marking images in the frame direction. And, reference pictures controlled by the above method can be used by inter prediction unit 700.

Inter prediction unit 700 performs an inter (inter) prediction using reference images stored in the decoded image buffer module 600. The intercoded macroblock can be divided into parts of the macroblock. And each of the parts of the macroblock can be predicted based on one or two reference images. Inter prediction module 700 includes motion compensation module 710, lighting compensation module 720, lighting difference prediction module 730, frame synthesis prediction module 740, weighted prediction module 750, and the like.

The motion compensation module 710 compensates for the motion of the current block using the information transmitted from the statistical decoding module 200. The motion vectors of neighboring blocks of the current block are extracted from the video signal, and then the motion vector prediction block of the current block is derived from the motion vectors of neighboring blocks. And, the motion of the current block is compensated using the obtained motion vector prediction block and the difference motion vector extracted from the video signal. And, it is possible to perform motion compensation using a single reference image or multiple images. When encoding a multi-frame video, if this current image refers to images in excellent frames, it is possible to perform motion compensation using the information of the reference image list for inter-frame prediction stored in the buffer module 600 of the decoded images. And, it is also possible to perform motion compensation using the frame information to identify the frame of the reference image. The direct mode is an encoding mode for predicting motion information of a current block from motion information for an encoded block. Since this method is able to save a number of bits required for encoding motion information, the compression efficiency is improved. For example, the time direction mode predicts motion information for the current block using the correlation of motion information in the time direction. Using a method similar to this method, the present invention is able to predict motion information for the current block using correlation of motion information in the direction of the frames.

Meanwhile, if the entered bitstream corresponds to multi-frame video, since the corresponding sequence of frames was obtained by an excellent camera, the difference in lighting is generated by the internal and external factors of these cameras. To prevent this, the lighting compensation module 720 compensates for the difference in lighting. When performing lighting compensation, it is possible to use flag information indicating whether to perform lighting compensation in relation to a particular video signal level. For example, it is possible to perform lighting compensation using flag information indicating whether to perform lighting compensation with respect to the corresponding clipping or macroblock. When performing lighting compensation using flag information, lighting compensation is applicable to various types of macroblocks (for example, 16 × 16 inter-mode, B-skip mode, direct mode, etc.).

When performing lighting compensation, it is possible to use information for a neighboring block or information for a block in a frame different from that of the current block to restore the current block. And it is also possible to use the lighting difference value of the current block. In this case, if the current block refers to blocks in a different frame, it is possible to perform lighting compensation using the information of the reference picture list for inter-frame prediction stored in the buffer module 600 of the decoded images. In this case, the illumination difference value of the current block indicates the difference between the average pixel value of the current block and the average pixel value of the reference block corresponding to the current block. As an example of using the illumination difference value, the prediction value of the illumination difference of the current block is obtained using neighboring blocks of the current block, and the difference value (residual illumination difference) between the illumination difference value and this illumination difference prediction value is used. Therefore, the decoding module is able to reconstruct the lighting difference value of the current block using the residual lighting difference and the lighting difference prediction value. Upon receipt of the prediction value of the lighting difference of the current block, it is possible to use information for the neighboring block. For example, it is possible to predict the value of the lighting difference of the current block using the value of the lighting difference of the neighboring block. Before the prediction, it is checked whether the reference index of the current block is equal to that of the neighboring block. According to the result of the check, then it is decided which view from the neighboring block or value will be used.

A frame synthesis prediction module 740 is used to synthesize images in a virtual frame using images in a frame adjacent to the frame of the current image and predict the current image using the synthesized images in the virtual frame. The decoding module is able to decide whether to synthesize the image in the virtual frame according to the prediction identifier of the inter-frame synthesis transmitted from the encoding module. For example, if view_synthesize_pred_flag = 1 or view_syn_pred_flag = 1, then a clipping or macroblock in a virtual frame is synthesized. In this case, when the inter-frame synthesis prediction identifier informs that a virtual frame will be generated, it is possible to generate an image in a virtual frame using the frame information to identify the image frame. And, when predicting the current image from the synthesized images in the virtual frame, it is possible to use the frame information to use the image in the virtual frame as a reference image.

A weighted prediction unit 750 is used to compensate for a phenomenon in which the image quality of a sequence is significantly degraded in the case of a coding sequence whose brightness changes over time. In MVC, weighted prediction can be performed to compensate for the difference in brightness from a sequence in a different frame, just as it is for a sequence whose brightness changes over time. For example, a weighted prediction method can be classified into an explicit weighted prediction method and an implicit weighted prediction method.

In particular, an explicit weighted prediction method may use one reference image or two reference images. In the case of using one reference image, the prediction signal is obtained from multiplying the prediction signal corresponding to the motion compensation by a weight coefficient. In the case of using two reference images, the prediction signal is obtained from summing the bias value to the value obtained from multiplying the prediction signal corresponding to the motion compensation by the weight coefficient.

And, implicit weighted prediction performs weighted prediction using the distance from the reference image. As a method of obtaining the distance from the reference image, it is possible to use POC (Image Index), indicating the image output order, for example. In this case, the POC can be obtained by considering the frame identification information of each image. When obtaining a weight coefficient for an image in an excellent frame, it is possible to use the frame information to identify the image frame in order to obtain the distance between the frames of the corresponding images.

When encoding a signal, the depth video information is suitable for use for a specific application or other purpose. In this case, the depth information may mean information capable of indicating an inter-frame mismatch difference. For example, it is possible to obtain a mismatch vector in accordance with inter-frame prediction. And the resulting mismatch vector must be transmitted to the decoding device to compensate for the mismatch of the current block. However, if the depth map is received and then transmitted to the decoding device, the mismatch vector can be logically derived from the depth map (or mismatch card) without transmitting the mismatch vector to the decoding device. In this case, this is advantageous in that the number of bits of depth information that must be transmitted to the decoding device can be reduced. So, getting the mismatch vector from the depth map, it is possible to provide a new way to compensate for the mismatch. Thus, in the case of using the image in an excellent frame during the acquisition of the error vector from the depth map, frame information can be used to identify the image frame.

Inter-predicted or intra-predicted images by the process described above are selected according to the prediction mode in order to restore the current image. In the following description, various embodiments are described that provide an efficient method for decoding a video signal.

4 is a diagram of a hierarchical structure of level information for providing scalability of a video signal frame according to one embodiment of the present invention.

With reference to FIG. 4, level information for each frame can be determined by considering inter-frame reference information. For example, since it is impossible to decode a P-image and a B-image without an I-image, it is possible to assign 'level = 0' to the main view, the group of inter-frame images of which is an I-image, 'level = 1' to the main view, whose group of inter-frame images is P-image, and 'level = 2' the main view, the group of inter-frame images of which is a B-image. However, it is also possible to determine level information at random according to a particular standard.

Level information may be randomly determined according to a specific standard or without a standard. For example, in the case where level information is determined based on the frame, it is possible to set the V0 frame as the main frame to frame level 0, the frame of images predicted using the image in one frame to level 1 of the frame, and the frame of images predicted using the image in many frames, in level 2 frames. In this case, at least one frame sequence may be necessary in order to be compatible with a conventional decoder (e.g., H.264 / AVC, MPEG-2, MPEG-4, etc.). This main frame becomes the basis for coding a multi-frame video, which may correspond to a reference frame for predicting another frame. The sequence corresponding to the main frame in MVC (multi-frame video encoding) can be configured into an independent bitstream by encoding in accordance with a conventional sequence encoding scheme (MPEG-2, MPEG-4, H.263, H.264, etc. ) The sequence corresponding to the main frame is compatible with or may not be H.264 / AVC. However, the sequence in the frame compatible with H.264 / AVC corresponds to the main frame.

As can be seen in FIG. 4, it is possible to set a frame V2 of images predicted using images in a frame V0, a frame V4 of images predicted using images in a frame V2, a frame V6 of images predicted using images in a frame V4, and a frame V7 of images predicted using images in a V6 frame at level 1 of the frame. And, it is possible to set the frame V1 of the images predicted using the images in frames V0 and V2, and the frame V3 predicted in the same way, and the frame V5 predicted in the same way, in level 2 of the frame. So, in the case when the user decoder is not able to view the sequence of multi-frame video, it decodes the sequence in the frame, corresponding only to level 0 frames. In case the user decoder is limited to profile information, it is possible to decode information of only a limited level of frames. In this case, the profile means that the technical elements for the algorithm in the video encoding / decoding process are standardized. In particular, a profile is a set of technical elements required for decoding a bit sequence of a compressed sequence and can be a kind of sub-standardization.

According to another embodiment of the present invention, the level information may change according to the location of the camera. For example, assuming that frames V0 and V1 are sequences obtained by the camera located in front, that frames V2 and V3 are sequences located at the back, that frames V4 and V5 are sequences located to the left, and that frames V6 and V7 are sequences located at the front. on the right, it is possible to set frames V0 and V1 to level 0 frames, frames V2 and V3 to level 1 frames, frames V4 and V5 to level 2 frames, and frames V6 and V7 to level 3 frames. Alternatively, the level information may vary according to the alignment of the camera. Alternatively, level information may not be randomly determined based on a specific standard.

5 is a configuration diagram of a NAL unit including layer information in an extension area of a NAL header according to one embodiment of the present invention.

With reference to FIG. 5, a NAL unit mainly includes a NAL header and an RBSP. The NAL header includes flag information (nal_ref_idc) indicating whether the notch becomes the reference image of the NAL block and an identifier (nal_unit_type) indicating the type of the NAL block. And the NAL header may also include level information (view_level) indicating information for the hierarchical structure to provide scalability of the frame.

The compressed initial data is stored in the RBSP, and the tail of the RBSP is added to the last part of the RBSP to represent the length of the RBSP as an 8-bit multiplication number. As the types of NAL unit, there are IDR (instant update of decoding), SPS (sequence parameter set), PPS (image parameter set), SEI (additional extension information), etc.

The NAL header includes information for the frame identifier. And the video sequence of the corresponding frame level is decoded with reference to the frame identifier during decoding according to the frame level.

The NAL unit includes a NAL header 51 and a cut level 53. The NAL header 51 includes an extension 52 of the NAL header. And, clipping level 53 includes a clipping header 54 and clipping data 55.

The NAL header 51 includes an identifier (nal_unit_type) indicating the type of the NAL unit. For example, an identifier indicating the type of NAL unit may be an identifier for scalable coding and multi-frame coding of video. In this case, the NAL header extension 52 may include flag information distinguishing whether the current NAL is NAL for scalable video encoding or NAL for multi-frame video encoding. And, the NAL header extension 52 may include extension information for the current NAL according to the flag information. For example, in the case where the current NAL is a NAL for encoding multi-frame video according to the flag information, the NAL header extension 52 may include level information (view_level) indicating information for the hierarchical structure to provide scalability of the frame.

6 is a diagram of a complete multi-frame video signal prediction structure according to one embodiment of the present invention, to explain the concept of a group of inter-frame images.

With reference to FIG. 6, T0 to T100 on a horizontal axis indicate frames according to time, and S0 to S7 on a vertical axis indicate frames according to a frame. For example, images at time T0 mean frames captured by different cameras in the same time zone T0, while images at S0 mean sequences captured by a separate camera in different time zones. And, the arrows in the drawing indicate the prediction directions and the prediction orders of the respective images. For example, the image P0 in frame S2 in the time zone T0 is the image predicted from I0, which becomes the reference image of the image P0 in the frame S4 in the time zone T0. And, it becomes a reference image of images B1 and B2 in time zones T4 and T2 in frame S2, respectively.

In a multi-frame video decoding process, inter-frame random access may be necessary. Thus, access to a random frame should be possible by minimizing the decoding effort. In this case, the concept of a group of inter-frame images may be necessary in order to realize effective access. Inter-frame image group means an encoded image in which all clippings refer only to clippings with the same image index. For example, a group of inter-frame images means an encoded image that refers to clippings only on a different frame, without reference to clippings in the current frame. 6, if the image I0 in frame S0 in the time zone T0 is a group of inter-frame images, all images in different frames with respect to the same time zone, i.e., the time zone T0, become multi-frame image groups. As another example, if the image I0 in the frame S0 in the time zone T8 is a group of inter-frame images, all the images in different frames in the same time zone, that is, the time zone T8, are multi-frame image groups. Similarly, all images in T16, ..., T96, and T100 also become multi-frame image groups.

7 illustrates a diagram of a prediction structure according to an embodiment of the present invention to explain the concept of a newly defined group of inter-frame images.

In the general MVC prediction structure, a GOP may start with an I-picture. And this I-image is compatible with H.264 / AVC. So, all groups of inter-frame images compatible with H.264 / AVC can always become an I-image. However, if the I-pictures are replaced by a P-picture, more efficient coding is allowed. In particular, more efficient coding is allowed using a prediction structure allowing a GOP to start with a P-image compatible with H.264 / AVC.

In this case, if the group of inter-frame images is redefined, all the clippings will become an encoded image capable of referring not only to a clipping in a frame in a certain time zone, but also to clipping in the same frame in a different time zone. However, in the case of a reference to a clipping in a different time zone in the same frame, this may be limited to a group of multi-frame images compatible only with H.264 / AVC. For example, the P-image at time point T8 in frame S0 in FIG. 6 may become a newly defined group of multi-frame images. Similarly, the P-image at time point T96 in the S0 frame or the P-image at time point T100 in the S0 frame can become a newly defined group of multi-frame images. And, a group of inter-frame images can be defined only if it has a main frame.

After the inter-frame image group is decoded, all sequentially encoded images are decoded from the images decoded before the multi-frame image group in the output order without inter prediction.

Considering the general multi-frame video encoding structure shown in FIG. 6 and FIG. 7, since the inter-frame reference information of the inter-frame image group is different from that of the non-inter-frame image group, it is necessary to distinguish the inter-frame image group and the non-inter-frame image group from each other according to the identification information of the inter-frame group images.

Inter-frame reference information means information capable of recognizing a prediction structure between inter-frame images. It can be obtained from the video signal data area. For example, it can be obtained from the scope of a sequence parameter set. And, inter-frame reference information can be recognized using the number of reference images and frame information for the reference images. For example, the number of full frames is obtained, and the frame information for identifying each frame can then be obtained based on the number of full frames. And, it is possible to obtain the number of reference images for the link direction for each frame. According to the number of reference images, it is possible to obtain frame information for each of the reference images. Thus, inter-frame reference information can be obtained. And, inter-frame reference information can be recognized by distinguishing between a group of inter-frame images and a group of non-inter-frame images. It can be recognized using the identification information of the inter-frame image group indicating whether the encoded clip in the current NAL is a multi-frame image group. Details of the identification information of the inter-frame image group are described with reference to FIG. 8 below.

FIG. 8 is a schematic block diagram of an apparatus for decoding multi-frame video using a group of inter-frame images identifying information according to one embodiment of the present invention.

With reference to FIG. 8, a decoding apparatus according to one embodiment of the present invention includes a bitstream determining unit 81, an inter-image group identification information obtaining unit 82, and a multi-frame video decoding unit 83.

If a bitstream is input, the bitstream determination unit 81 determines whether the inputted bitstream is an encoded bitstream for scalable video encoding or an encoded bitstream for encoding multi-frame video. This can be determined from the flag information included in the bitstream.

The interframe image group identification information obtaining unit 82 may obtain interframe image group identification information if the input bitstream is a bitstream for encoding a multi-frame video as a result of the determination. If the obtained identification information of the inter-frame image group is 'true', this means that the encoded clipping of the current NAL is a group of inter-frame images. If the obtained identification information of the inter-frame image group is 'false', this means that the encoded clipping of the current NAL is a group of non-inter-frame images. The identification information of a group of inter-frame images can be obtained from the extension area of the NAL header or the region of the cut level.

The multi-frame video decoding unit 83 decodes the multi-frame video according to the identification information of the inter-frame image group. According to the general coding structure of a multi-frame video sequence, the inter-frame reference (reference) information of the inter-frame image group is different from that of the non-frame image group. So, it is possible to use the identification information of the inter-picture group of images to add reference pictures for inter-prediction to generate a list of reference pictures, for example, AND, it is also possible to use the identification information of the group of inter-picture images to control reference pictures for inter-picture prediction. In addition, the identification information of the inter-frame image group is applicable to a hypothetical reference decoder.

As another example of using the identification information of a group of inter-frame images, in the case of using information in a different frame for each decoding process, inter-frame reference information included in the sequence parameter set is used. In this case, information for distinguishing whether the current image is a group of inter-frame images or a group of non-inter-frame images, that is, identification information of a group of inter-frame images, may be required. So, it is possible to use excellent inter-frame reference information for each decoding process.

Fig. 9 is a flowchart of a process for generating a list of reference images according to an embodiment of the present invention.

With reference to FIG. 9, a decoded image buffer module 600 plays a role in storing or opening precoded images to perform inter prediction for an image.

First of all, images encoded prior to the current image are stored in the reference image storage unit 610 to be used as the reference image (S91).

When encoding a multi-frame video, since some of the pre-encoded images are in a frame different from that of the current image, the frame information for identifying the image frame can be used to use these images as a reference image. So, the decoder must obtain frame information to identify the image frame (S92). For example, frame information may include a 'view_id' to identify an image frame.

Buffer module 600 of decoded images is necessary to obtain the variable used in it to form a list of reference images. Since inter-frame prediction may be required for encoding multi-frame video, if the current image refers to an image in a different frame, it may be necessary to create a list of reference images for inter-frame prediction. In this case, the buffer module 600 of the decoded images is necessary to obtain a variable used to generate a list of reference images for inter-frame prediction using the obtained frame information (S93).

The reference picture list for temporal prediction or the reference picture list for inter-frame prediction can be generated in a different way according to the cut type of the current cut (S94). For example, if the cut type is P / SP cut, a list of 0 reference images is formed (S95). In the case where the cut type is B-cut, a list of 0 reference images and a list of 1 reference images are formed (S96). In this case, the reference picture list 0 or 1 may include only the reference picture list for temporal prediction, or both from the reference picture list for temporal prediction and the reference picture list for inter-frame prediction. This is described in detail with reference to FIG. 8 and FIG. 9 below.

The initialized reference picture list is subjected to a lower number assignment process for frequently referenced images to further increase the compression ratio (S97). And, this may be called the reordering process for the reference picture list, which is described in detail with reference to FIGS. 12-19 below. The current image is decoded using a reordered list of reference pictures, and a decoded image buffer module 600 is needed to control the decoded reference pictures in order to use the buffer more efficiently (S98). Reference images controlled by the above process are read by inter prediction unit 700 to be used for inter prediction. When encoding a multi-frame video, inter prediction may include inter-frame prediction. In this case, a reference picture list for inter-frame prediction is used.

Detailed examples for a method of generating a list of reference images according to a clipping type are described with reference to FIG. 10 and FIG. 11 below.

10 is a diagram for explaining a method of initializing a reference picture list when the current cut is a P-cut, according to one embodiment of the present invention.

With reference to FIG. 10, time is denoted by T0, T1, ... TN, while the frame is denoted by V0, V1,., V4. For example, the current image indicates the image at time T3 in frame V4. And, the clipping type of the current image is P-clipping. 'PN' is an abbreviation of the PicNum variable, 'LPN' is an abbreviation of the LongTermPicNum variable, and 'VN' is an abbreviation of the ViewNum variable. A digit added to the end of each variable indicates an index indicating the time of each image (for PN or LPN) or the frame of each image (for VN). This applies to FIG. 11 equally.

The reference picture list for temporal prediction or the reference picture list for inter-frame prediction can be formed in another way according to the cut type of the current cut. For example, the clipping type in FIG. 12 is P / SP clipping. In this case, a list of 0 reference images is formed. In particular, reference picture list 0 may include a reference picture list for temporal prediction and / or a reference picture list for inter-frame prediction. In this embodiment, it is assumed that the reference picture list includes both a reference picture list for temporal prediction and a reference picture list for inter-frame prediction.

There are various methods for arranging reference images. For example, reference images may be arranged according to the decoding or output order of the image. Alternatively, reference images may be arranged based on a variable obtained using frame information. Alternatively, reference images may be arranged according to inter-frame reference information indicating an inter-frame prediction structure.

In the case of a list of reference pictures for temporal prediction, short-term reference pictures and long-term reference pictures can be arranged based on the decoding order. For example, they can be arranged according to the value of the PicNum or LongTermPicNum variable obtained from a value indicating the image identification number (for example, frame_num or Longtermframeidx). First of all, short-term reference images can be initialized to long-term reference images. The order of alignment of the short-term reference images can be set from the reference image having the highest value of the PicNum variable to the reference image having the lowest value of the variable. For example, short-term reference images may be arranged in order PN1 having the highest variable, PN2 having the intermediate variable, and PN0 having the lowest variable among PN0 - PN2. The order of alignment of the long-term reference images can be set from the reference image having the lowest value of the LongTermPicNum variable to the reference image having the lowest value. For example, long-term reference images may be arranged in the order of LPN0 having the highest variable value and LPN1 having the lowest variable value.

In the case of a reference picture list for inter prediction, reference pictures may be arranged based on a first ViewNum variable obtained using frame information. In particular, reference images can be arranged in order from a reference image having the highest value of the first variable (ViewNum) to a reference image having the lowest value of the first variable (ViewNum). For example, reference images may be arranged in the order of VN3 having the highest variable, VN2, VN1, and VN0 having the lowest variable among VN0, VN1, VN2, and VN3.

Thus, both of the list of reference pictures for temporal prediction and the list of reference pictures for inter-frame prediction can be managed as one list of reference pictures. Alternatively, both of the reference picture list for temporal prediction and the reference picture list for inter-frame prediction can be managed as separate reference picture lists, respectively. In the case of managing both of the reference picture list for temporal prediction and the reference picture list for inter-frame prediction as one list of reference pictures, they can be initialized in order or simultaneously. For example, in case of initializing both of the list of reference pictures for temporal prediction and the list of reference pictures for inter-frame prediction according to the order, the list of reference pictures for temporal prediction is preferably initialized, and the list of reference pictures for inter-frame prediction is then initialized further. This concept is also applicable to FIG.

The case where the cut type of the current image is B-cut is described with reference to FIG. 11 below.

11 is a diagram for explaining a method of initializing a reference picture list when the current cut is a B-cut according to one embodiment of the present invention.

With reference to FIG. 9, in case the cut type is B-cut, a list of 0 reference images and a list of 1 reference images are formed. In this case, reference picture list 0 or reference picture list 1 may include only a reference picture list for temporal prediction, or both a reference picture list for temporal prediction and a reference picture list for inter-frame prediction.

In the case of a list of reference images for temporal prediction, the method of building short-term reference images may differ from the method of building long-term reference images. For example, in the case of short-term reference images, reference images can be arranged according to the index of the image (hereinafter abbreviated as POC). In the case of long-term reference images, reference images can be aligned according to the value of the variable (LongtermPicNum). And, short-term reference images can be initialized to long-term reference images.

In the order of arranging the short-term reference images of the reference image list 0, the reference images are preferably aligned from the reference image having the highest POC value to the reference image having the lowest POC among the reference images having POC values smaller than that of the current image, and then lining up from the reference image having the lowest POC value to the reference image having the highest POC among the reference images having POC values greater than m is the current image. For example, reference images can preferably be aligned from PN1 having the highest POC value in reference images PN0 and PN1 having POC values smaller than that of the current image to PN0, and then aligned from PN3 having the lowest POC value in reference images PN3 and PN4 having a POC value smaller than that of the current image, to PN4.

In order to build the long-term reference images of the list 0 of reference images, the reference images are aligned from the reference image having the lowest LongtermPicNum variable to the reference image having the highest variable. For example, reference images are aligned from LPN0 having the lowest value in LPN0 and LPN1 to LPN1 having the second lowest variable.

In the case of a reference picture list for inter prediction, reference pictures may be arranged based on a first ViewNum variable obtained using frame information. For example, in the case of a list of 0 reference pictures for inter-frame prediction, the reference pictures may be aligned from the reference picture having the highest value of the first variable among the reference pictures having the values of the first variable lower than that of the current picture to the reference picture having the lowest value of the first variable. The reference images are then aligned from the reference image having the lowest value of the first variable among the reference images having the first variable value greater than that of the current image to the reference image having the highest value of the first variable. For example, reference images are preferably aligned from VN1 having the highest value of the first variable in VN0 and VN1 having the value of the first variable less than that of the current image to VN0 having the lowest value of the first variable, and then aligned from VN3 having the highest a low value of the first variable in VN3 and VN4 having a first variable value greater than that of the current image, to VN4 having the highest value of the first variable.

In the case of the reference picture list 1, the above-described method of arranging the reference picture list 0 is applicable similarly.

First of all, in the case of the reference picture list for temporal prediction, in order to arrange (arrange) the short-term reference picture of the reference picture list 1, the reference pictures are preferably aligned from the reference picture having the lowest POC value to the reference picture having the highest POC among the reference pictures images having POC values larger than those of the current image, and then line up from the reference image having the highest POC value to the reference image the position having the lowest POC among the reference images having POCs smaller than that of the current image. For example, reference pictures can preferably be aligned from PN3 having the lowest POC value in reference pictures PN3 and PN4 having POC values larger than that of the current picture to PN4, and then aligned from PN1 having the highest POC value in reference pictures PN0 and PN1 having POC values greater than that of the current image to PN0.

In the order of alignment of the long-term reference images of the reference image list 1, the reference images are aligned from the reference image having the lowest LongtermPicNum variable to the reference image having the highest variable. For example, reference images are aligned from LPN0 having the lowest value in LPN0 and LPN1 to LPN1 having the lowest variable.

In the case of a reference picture list for inter prediction, reference pictures may be arranged based on a first ViewNum variable obtained using frame information. For example, in case 1 of the reference picture list for inter-frame prediction, the reference pictures may be aligned from the reference picture having the lowest value of the first variable among the reference pictures having the first variable value greater than that of the current picture to the reference picture having the highest value first variable. The reference images are then aligned (ordered) from the reference image having the highest value of the first variable among the reference images having the value of the first variable less than that of the current image to the reference image having the lowest value of the first variable. For example, reference images are preferably aligned from VN3 having the lowest value of the first variable in VN3 and VN4 having a first variable value greater than that of the current image to VN4 having the highest value of the first variable, and then aligned from VN1 having the highest the value of the first variable in VN0 and VN1 having the value of the first variable less than that of the current image, to VN0 having the lowest value of the first variable.

The reference image list initialized by the above process is passed to the reference image list reordering unit 640. The initialized reference picture list is then reordered for more efficient coding. The reordering process should reduce the bit rate by assigning a small amount to the reference image having the highest probability of being selected as the reference image using a decoded image buffer. Various methods for reordering the reference image list are described with reference to FIGS. 12-19 below.

12 illustrates an internal block diagram of a reference picture list reordering module 640 according to one embodiment of the present invention.

With reference to FIG. 12, the reference image list reordering module 640 mainly includes a clipping type checking module 642, the reference image list reordering module 643, and the reference image list reordering module 645.

In particular, the reference image list reordering unit 643 64 includes a first identification information obtaining unit 643A and a first reference index assignment changing unit 643B. And, the reference picture list reordering module 645 includes a second identification information obtaining module 645A and a second reference index assignment changing module 645B.

The clipping type checking module 642 checks the clipping type of the current clipping. Then a decision is made whether to reorder the list of 0 reference images and / or the list of 1 reference images according to the type of clipping. For example, if the cut type of the current cut is I-cut, both of the list of 0 reference images and the list of 1 reference images are not reordered. If the cut type of the current cut is P-cut, only the list of 0 reference images is reordered. If the cut type of the current cut is B-cut, both of the list of 0 reference images and the list of 1 reference images are reordered.

The reordering module 643 of the reference picture list 0 is activated if the flag information for performing the reordering of the reference picture list is 'true', and if the cut type of the current cut is not I-cut. The first identification information obtaining unit 643A obtains identification information indicating a method for assigning a reference index. The first reference index assignment change module 643B changes the reference index assigned to each reference image of the reference image list 0 according to this identification information.

Similarly, the reordering unit 645 of the reference picture list 1 is activated if the flag information for performing the reordering of the reference picture list 1 is “true” and if the cut type of the current cut is B-cut. The second identification information obtaining unit 645A obtains identification information indicating a method for assigning a reference index. The second reference index assignment change module 645B changes the reference index assigned to each reference image of the reference image list 1 according to this identification information.

Thus, reference image list information used for actual inter prediction is generated by the reordering unit 643 of the reference image list 0 and the reordering unit 645 of the reference image list 1.

A method for changing a reference index assigned to each reference image by a first or second reference index assignment unit 643B or 645B is described with reference to FIG. 13 below.

13 is an internal block diagram of a reference index assignment change module 643B or 645B according to one embodiment of the present invention. In the following description, the reference picture list reordering module 643 64 and the reference picture list reordering module 645 shown in FIG. 12 are described together.

With reference to FIG. 13, each of the first and second reference index assignment change module 643B and 645B includes a temporal prediction reference index assignment change module 644A, a reference index assignment change module 644B for a long-term reference image, and a reference index assignment change module 644C for inter prediction, and the module 644D termination (completion) of the destination change of the reference index. According to the identification information obtained by the first or second modules 643B or 645B, the changes in the assignment of the reference index, the parts in the first or second modules 643B and 645B of the change in the assignment of the reference index are activated, respectively. And the reordering process continues until the identification information is entered to complete the reassignment of the reference index.

For example, if the identification information for changing the reference index assignment for temporal prediction is obtained from the first or second reference index assignment changing module 643B or 645B, the temporal prediction reference index changing module 644A is activated. This temporal prediction reference index assignment change module 644A obtains an image number difference according to the received identification information. In this case, the difference of the image numbers means the difference between the image number of the current image and the number of the predicted image. And the number of the predicted image may indicate the number of the reference image assigned immediately before it. So, it is possible to change the purpose of the reference index using the resulting image number difference. In this case, the difference in image number can be added / subtracted to / from the predicted image number according to the identification information.

For another case, if the identification information for changing the destination of the reference index for the assigned long-term reference image is received, the module 644B changing the destination of the reference index for the long-term reference image is activated. The reference index assignment change module 644B for the long-term reference image obtains a long-term reference image number of the designated image according to the identification number.

For another case, if identification information for changing the assignment of the reference index for inter-frame prediction is received, the module 644C changing the designation of the reference index for inter-frame prediction is activated. Inter-prediction reference index assignment change module 644C obtains a frame information difference according to identification information. In this case, the difference of the frame information means the difference between the frame number of the current image and the number of the predicted frame. And, the predicted frame number may indicate the frame number of the reference image assigned immediately before it. So, it is possible to change the purpose of the reference index using the obtained frame information difference. In this case, the difference of the frame information can be added / subtracted to / from the predicted frame number according to the identification information.

For another case, if identification information is received to complete the change of destination of the reference index, the module 644D of the end of the change of destination of the reference index is activated. The reference index assignment change completion module 644D ends the reference index assignment change according to the received identification information. Thus, the reference picture list reordering module 640 generates reference picture list information.

Thus, the reference images used for inter-frame prediction can be controlled together with the reference images used for temporal prediction. Alternatively, reference images used for inter-frame prediction can be controlled separately from reference images used for temporal prediction. This may require new information to control the reference images used for inter-frame prediction. This will be described with reference to FIGS. 15-19 below.

Details of the inter-frame prediction reference index assignment module 644C are described with reference to FIG. 14 below.

14 is a diagram for explaining a process for reordering a list of reference images using frame information according to one embodiment of the present invention.

With reference to FIG. 14, if the frame number VN of the current picture is 3, if the size of the decoder picture DPBsize buffer is 4, and if the cut type of the current cut is P-cut, the reordering process for the reference picture list 0 is described below.

First of all, the initially predicted frame number is '3', which is the frame number of the current image. And, the initial ordering of the reference picture list 0 for inter-frame prediction is '4, 5, 6, 2' ((1)). In this case, if identification information is received to change the assignment of the reference index for inter-frame prediction by subtracting the difference of the frame information, '1' is obtained as the difference of the frame information according to the received identification information. The newly predicted frame number (= 2) is calculated by subtracting the difference of the frame information (= 1) from the predicted frame number (= 3). In particular, the first index of the reference picture list 0 for inter-frame prediction is assigned to the reference picture having frame number 2. And, the image previously assigned to the first index can be moved to the very back of the reference picture list 0. Thus, the reordered list of reference images is '2, 5, 6, 4' ((2)). Then, if identification information is received to change the assignment of the reference index for inter prediction by subtracting the difference of the frame information, '-2' is obtained as the difference of the frame information according to this identification information. The newly predicted frame number is then calculated by subtracting the difference of the frame information (= -2) from the predicted frame number. In particular, the second index of the reference picture list 0 for inter-frame prediction is assigned to the reference picture having frame number 4. Therefore, the reordered reference picture list 0 is '2, 4, 6, 5' ((3)). Then, if identification information is received to complete the reassignment of the reference index, the reference image list 0 having the rearranged reference image list 0 as an end is formed according to the received identification information ((4)). Therefore, the order of the finally generated reference picture list 0 for inter-frame prediction is '2, 4, 6, 5'.

For another case of reordering the remaining images after the first index of the reference picture list 0 for inter-frame prediction has been assigned, the image assigned to each index can be moved to the right to the position immediately behind that of the corresponding image. In particular, the second index is assigned to the image having the frame number 4, the third index is assigned to the image (frame number 5) to which the second index has been assigned, and the fourth index is assigned to the image (frame number 6) to which the third index has been assigned. Therefore, the reordered list 0 of reference images becomes equal to '2, 4, 5, 6'. And, the subsequent reordering process can be performed in the same way.

The reference picture list generated by the above process is used for inter (external) prediction. Both of the reference picture list for inter-frame prediction and the list of reference pictures for temporal prediction can be managed as one list of reference pictures. Alternatively, each of the reference picture list for inter-frame prediction and the reference picture list for temporal prediction can be managed as a separate reference picture list. This is described with reference to FIG. 15 to 19 below.

15 is an internal block diagram of a reference picture list reordering module 640 according to another embodiment of the present invention.

With reference to FIG. 15, in order to manage the reference picture list for inter-frame prediction as a separate reference picture list, new information may be necessary. For example, the reference picture list for temporal prediction is reordered, and the reference picture list for inter-frame prediction is then reordered in some cases.

The reference picture list reordering module 640 mainly includes a temporal prediction reference picture reordering module 910, a NAL type checking module 960, and an inter-frame prediction reference picture reordering module 970.

The temporal prediction reference image reordering unit 910 includes a clipping type checking module 642, a third identification information obtaining module 920, a third reference index assignment changing module 930, a fourth identification information obtaining module 940, and a fourth reference index assignment changing module 950. The third reference index assignment change module 930 includes a temporal prediction reference index assignment change module 930A, a reference index assignment change module 930B for a long-term reference image, and a reference index assignment change completion module 930C. Similarly, the fourth reference index assignment change module 950 includes a temporal prediction reference index assignment change module 950A, a reference index assignment change module 950B for a long-term reference image, and a reference index assignment change completion module 950C.

The temporal prediction reference picture reordering unit 910 reorders the reference pictures used for temporal prediction. The operations of the reference picture reordering module 910 for temporal prediction are identical to those of the aforementioned reference picture reordering module 640 of FIG. 10, in addition to information for reference pictures for inter-frame prediction. Thus, the details of the temporal prediction reference picture reordering unit 910 are omitted in the following description.

The NAL type verification unit 960 checks the NAL type of the received bitstream. If the NAL type is NAL for multi-frame video encoding, the reference pictures used for inter-frame prediction are reordered by the temporal prediction reference picture reordering unit 970. The generated reference picture list for inter-frame prediction is used for inter prediction together with the reference picture list generated by the temporal prediction reference picture reordering unit 970. However, if the NAL type is not NAL for multi-frame video encoding, the reference picture list for inter-frame prediction is not reordered. In this case, only a list of reference images for temporal prediction is formed. And, the inter-prediction reference picture list reordering unit 970 reorders the reference pictures used for inter-frame prediction. This is described in detail with reference to FIG. 16 below.

16 is an internal block diagram of a reference picture reordering module 970 for inter-frame prediction according to one embodiment of the present invention.

With reference to FIG. 16, a reference picture list reordering module 9 for inter prediction includes a clipping type checking module 642, a fifth identification information obtaining module 971, a fifth reference index assignment changing module 972, a sixth identification information obtaining module 973, and a sixth module 974 changes to the destination of the reference index.

The clipping type checking module 642 checks the clipping type of the current clipping. If so, then a decision is made whether to reorder the reference image list 0 and / or the reference image list 1 according to the clipping type. Details of the clipping type checking module 642 can be inferred from FIG. 10, which are omitted in the following description.

Each of the fifth and sixth identification information obtaining modules 971 and 973 obtains identification information indicating a method of assigning a reference index. And, each of the fifth and sixth modules 972 and 974 change the destination of the reference index changes the reference index assigned to each reference image of the list 0 and / or 1 reference images. In this case, the reference index can only mean the frame number of the reference images. And, identification information indicating a method for assigning a reference index may be flag information. For example, if the flag information is true, the assignment of the frame number changes. If the flag information is false, the reordering of the frame number may be completed. If the flag information is 'true', each of the fifth and sixth reference index assignment change modules 972 and 974 may obtain a frame number difference according to the flag information. In this case, the frame number difference means the difference between the frame number of the current image and the frame number of the predicted image. And, the frame number of the predicted image may mean the frame number of the reference image assigned just before it. You can then change the assignment of the frame number using the difference in frame numbers. In this case, the frame number difference can be added / subtracted to / from the number of frames of the predicted image according to the identification information.

Thus, in order to manage the reference picture list for inter prediction as a separate reference picture list, the syntax structure needs to be redefined. As one embodiment of the content described in FIG. 15 and FIG. 16, the syntax is described with reference to FIG. 17, FIG. 18 and FIG. 19 below.

FIG. 17 and FIG. 18 are syntax diagrams for reordering the reference picture list according to one embodiment of the present invention.

With reference to FIG. 17, the operation of the temporal prediction reference picture list reordering unit 910 shown in FIG. 15 is presented as syntax. Compared to the blocks shown in FIG. 15, the cutting type checking module 642 corresponds to S1 and S6, and the fourth identification information obtaining module 940 corresponds to S7. The indoor units of the third reference index assignment change module 930 correspond to S3, S4, and S5, respectively. And, the internal blocks of the fourth reference index assignment change module 950 correspond to S8, S9, and S10, respectively.

With reference to FIG. 18, the operation of the NAL type checking module 960 and the multi-frame reference picture list reordering module 970 are presented as syntax. Compared with the corresponding blocks shown in FIGS. 15 and 16, the NAL type verification module 960 corresponds to S11, the clipping type verification module 642 corresponds to S13 and S16, the fifth identification information obtaining module 971 corresponds to S14, and the sixth 973 identification information obtaining module corresponds to S17. The fifth reference index assignment change module 972 corresponds to S15, and the sixth reference index assignment change module 974 corresponds to S18.

FIG. 19 is a syntax diagram for reordering a list of reference pictures according to another embodiment of the present invention.

With reference to FIG. 19, the operation of the NAL type checking module 960 and the reordering module 970 of the reference inter-frame picture list are presented as syntax. Compared with the corresponding blocks shown in FIGS. 15 and 16, the NAL type verification module 960 corresponds to S21, the clipping type verification module 642 corresponds to S22 and S25, the fifth identification information obtaining module 971 corresponds to S23, and the sixth 973 identification information obtaining module corresponds to S26. The fifth reference index assignment change module 972 corresponds to S24, and the sixth reference index assignment change module 974 corresponds to S27.

As mentioned in the foregoing description, the reference picture list for inter prediction can be used by the inter prediction unit 700 and is also suitable for use in performing light compensation. Lighting compensation is applicable when performing motion compensation / motion estimation. In the case where the current image uses the reference image in an excellent frame, it is possible to perform lighting compensation more efficiently using the list of reference images for inter-frame prediction. Lighting compensations according to embodiments of the present invention are described below.

FIG. 20 is a process diagram for obtaining a lighting difference value of a current unit according to one embodiment of the present invention.

Lighting compensation means a process for decoding an adaptively motion-compensated video signal according to a change in lighting. And, this applies to the video signal prediction structure, for example, inter prediction, intra prediction, and the like.

Lighting compensation means a process for decoding a video signal using a residual lighting difference and a prediction value of a lighting difference corresponding to a block to be decoded. In this case, the prediction value of the illumination difference can be obtained from the neighboring block to the current block. The process for obtaining the prediction value of the illumination difference from the neighboring block can be determined using reference information for the neighboring block, and the sequence and direction can be taken into account when searching for neighboring blocks. A neighboring block means a block already decoded and also means a block decoded by considering redundancy within the same image for a frame or time, or a sequence decoded by considering redundancy within different images.

When comparing the similarities between the current block and the candidate reference block, the lighting difference between the two blocks must be taken into account. To compensate for the difference in lighting, a new motion estimation / compensation is performed. A new SAD can be found using Formula 1.

[Formula 1]

M _cur =

Mref (p, q) =

[Formula 2]

NewSAD (x, y) =

In this case, 'Mcur' indicates the average pixel value of the current block and 'Mref' indicates the average pixel value of the reference block. 'f (i, j)' indicates the pixel value of the current block and 'r (i + x, j + y)' indicates the pixel value of the reference block. By performing motion estimation based on the new SAD according to Formula 2, it is possible to obtain an average value of the pixel difference between the current block and the reference block. And, the obtained average value of the pixel difference can be called the value of the difference of lighting (IC_offset).

In the case of performing motion estimation, to which the lighting compensation is applied, the value of the difference of illumination and the motion vector are formed. And, lighting compensation is performed according to Formula 3 using the value of the difference between the lighting and the motion vector.

[Formula 3]

NewR (i, j) = {f (i, j) -M _cur } - {r (i + x ', j + y') - M _ref (m + x ', n + y')} =

{f (i, j) -r (i + x ', j + y')} - {M _cur -M _ref (m + x ', n + y')} =

{(f (i, j) -r (i + x ', j + y')} - IC_offset

In this case, NewR (i, j) indicates the illumination-compensated error value (residual) and (x ', y') indicates the motion vector.

The illumination difference value (M _curr -M _ref ) must be transmitted to the decoding module. The decoding module performs lighting compensation as follows.

[Formula 4]

f '(i, j) = {NewR "(x', y ', i, j) + r (i + x', j + y ')} + {M _cur -M _ref (m + x', n + y ')} = {NewR''(x', y ', i, j) + r (i + x', j + y ')} + IC_offset

In Formula 4, NewR "(i, j) indicates the restored lighting compensated error value (residual) and f '(i, j) indicates the pixel value of the restored current block.

To restore the current block, the value of the lighting difference must be transmitted to the decoding module. And the value of the lighting difference can be predicted from the information of neighboring blocks. To further reduce the number of bits for encoding the lighting difference value, only the difference value (RIC_offset) between the lighting difference value of the current block and the lighting difference value of the neighboring block can be sent. This is represented as Formula 5.

[Formula 5]

RIC_offset = IC_offset - predIC_offset

21 is a flowchart of a process for performing lighting compensation of a current unit according to an embodiment of the present invention.

With reference to FIG. 21, first of all, a lighting difference value of a neighboring block indicating an average value of a pixel difference between a neighboring block of the current block and the block referred to by the neighboring block is extracted from the video signal (S2110).

Then, the lighting difference prediction value for compensating the illumination of the current block is obtained using this lighting difference value (S2120). So, it is possible to reconstruct the illumination difference value of the current block using the obtained illumination difference prediction value.

In obtaining the prediction value of the illumination difference, various methods can be used. For example, before the lighting difference value of the current block is predicted from the lighting difference value of the neighboring block, it is checked whether the reference index of the current block is equal to that of the neighboring block. Then you can decide which neighboring block or value will be used according to the result of the check. For another case, when obtaining the prediction value of the lighting difference, flag information (IC_flag) indicating whether to perform lighting compensation of the current block can be used. And, the flag information for the current block can be predicted using also the information of neighboring blocks. For another case, it is possible to obtain a prediction value of a lighting difference using both of a method of checking a reference index and a method for predicting flag information. They are described in detail with reference to FIGS. 22-24 below.

FIG. 22 is a flowchart of a process for obtaining a prediction value of a lighting difference of a current block using information for a neighboring block according to one embodiment of the present invention.

With reference to FIG. 22, information for a neighboring block can be used in obtaining a prediction value of a lighting difference of a current block. In the present disclosure, a block may include a macroblock or a submacroblock. For example, it is possible to predict the value of the lighting difference of the current block using the value of the lighting difference of the neighboring block. Before that, it is checked whether the reference index of the current block is equal to that of the neighboring block. According to the result of the check, then it is possible to decide which neighboring block or value will be used. 22, 'refIdxLX' indicates the reference index of the current block, 'refIdxLXN' indicates the reference index of the block-N. In this case, 'N' is the label of the neighbor block of the current block and indicates A, B or C. And, 'PredIC_offsetN' indicates the value of the lighting difference to compensate for the lighting of the neighboring block-N. If it is impossible to use block C, which is located at the upper right end of the current block, it is possible to use block - D instead of block - C. In particular, the information for block - D is suitable for use as information for block C. If it is impossible to use both of the block - B and block - C, it is possible to use block A instead. Namely, it is possible to use information for block A as information for block - B or block - C.

For another case, when obtaining the prediction value of the lighting difference, you can use the flag information (IC_flag) indicating whether to perform lighting compensation of the current block. Alternatively, it is possible to use both of the reference index checking method and the flag information prediction method when obtaining the prediction value of the illumination difference. In this case, if the flag information for the neighboring block indicates that the lighting compensation has not been performed, that is, if IC_falg == 0, the value of the lighting difference 'PredIC_offsetN' of the neighboring block is set to 0.

23 is a flowchart of a process for performing lighting compensation using information for an adjacent unit, according to one embodiment of the present invention.

With reference to FIG. 23, the decoding module retrieves the average pixel value of the reference block, the reference index of the current block, the reference index of the reference block, and the like. from the video signal, and then can obtain the prediction value of the lighting difference of the current block using the extracted information. The decoding module obtains a difference value (residual lighting difference) between the lighting difference value of the current block and the lighting difference prediction value, and then is able to reconstruct the lighting difference value of the current block using the obtained residual lighting difference and the lighting difference prediction value. In this case, it is possible to use the information for the neighboring block to obtain the prediction value of the lighting difference of the current block. For example, you can predict the value of the lighting difference of the current block using the value of the lighting difference of the neighboring block. Before that, it is checked whether the reference index of the current block is equal to that of the neighboring block. According to the result of the check, you can then decide which neighboring block or value will be used.

In particular, the lighting difference value of the neighboring block indicating the average value of the pixel difference between the neighboring block of the current block and the block referred to by the neighboring block is extracted from the video signal (S2310).

Then, it is checked whether the reference index of the current block is equal to the reference index of one of the plurality of neighboring blocks (S2320).

As a result of check step S2320, if at least one neighboring block exists having the same reference index as that of the current block, it is checked whether one corresponding neighboring block exists or not (S2325).

As a result of the verification step S2325, if there is only one neighboring block having the same reference index of the current block, the lighting difference value of the neighboring block having the same reference index of the current block is assigned to the lighting difference prediction value of the current block (S2330). In particular, this is 'PredIC_offset = PredIC_offsetN'.

If a neighboring block having the same reference index as such of the current block does not exist as a result of verification step S2320, or if there are at least two neighboring blocks having the same reference index as such of the current block as a result of verification step S2325, the average from the lighting difference values (PredIC_offsetN, N = A, B, or C) of the neighboring blocks, the lighting difference prediction value of the current block is assigned (S650). In particular, this is 'PredIC_offset = Average (PredIC_offsetA, PredIC_offsetB, PredIC_offsetC)'.

24 is a flowchart of a process for performing lighting compensation using information for a neighboring unit, according to another embodiment of the present invention.

With reference to FIG. 24, the decoding module must reconstruct the illumination difference value of the current block in order to perform lighting compensation. In this case, it is possible to use the information for the neighboring block to obtain the prediction value of the lighting difference of the current block. For example, it is possible to predict the value of the lighting difference of the current block using the value of the lighting difference of the neighboring block. Before that, it is checked whether the reference index of the current block is equal to that of the neighboring block. According to the result of the check, then it is possible to decide which neighboring block or value will be used.

In particular, the lighting difference value of the neighboring block indicating the average value of the pixel difference between the neighboring block of the current block and the block referred to by the neighboring block is extracted from the video signal (S2410).

Then, it is checked whether the reference index of the current block is equal to the reference index of one of the plurality of neighboring blocks (S2420).

As a result of verification step S720, if at least one neighboring block exists having the same reference index as that of the current block, it is checked whether one corresponding neighboring block exists or not (S2430).

As a result of the verification step S2430, if there is only one neighboring block having the same reference index as such for the current block, the lighting difference value of the neighboring block having the same reference index as such for the current block is assigned to the lighting difference prediction value of the current block (S2440). In particular, this is 'PredIC_offset = PredIC_offsetN'.

If the neighboring block having the same reference index as that of the current block does not exist as a result of the verification step S2420, then the lighting difference prediction value of the current block is set to 0 (S2460). In particular, this is 'PredIC_offset = 0'.

If there are at least two neighboring blocks having the same reference index as such for the current block as a result of the verification step S2430, the neighboring block having a reference index different from that of the current block is set to 0, and the average of the neighboring difference illumination values blocks, including a value set to 0, is assigned to a prediction value of a lighting difference of a current block (S2450). In particular, this is 'PredIC_offset = Average (PredIC_offsetA, PredIC_offsetB, PredIC_offsetC)'. However, if there is a neighboring block having a reference index different from that of the current block, the value '0' may be included in PredIC_offsetA, PredIC_offsetB, or PredIC_offsetC.

Meanwhile, the frame information for identifying the image frame, and the list of reference images for inter-frame prediction is applicable to image synthesis in a virtual frame. In the process for synthesizing an image in a virtual frame, an image in an excellent frame can be referenced. So, if you use the frame information and the list of reference images for inter-frame prediction, you can synthesize the image in a virtual frame more efficiently. The following description describes methods for synthesizing an image in a virtual frame according to embodiments of the present invention.

25 is a flowchart of a process for predicting a current image using an image in a virtual frame, according to one embodiment of the present invention.

With reference to FIG. 25, when performing inter-frame prediction when encoding multi-frame video, it is possible to predict the current image using an image in a frame different from that of the current frame as a reference image. However, the image in the virtual frame is obtained using the images in the frame adjacent to that of the current image, and the current image is then predicted using the obtained image in the virtual frame. If so, the prediction can be performed more accurately. In this case, a frame identifier indicating an image frame may be used to use images in adjacent frames or images in a particular frame. In the case where a virtual frame is generated, there must be special syntax for indicating whether to generate a virtual frame. If the syntax indicates that a virtual frame should be generated, it is possible to form a virtual frame using the frame identifier. The images in the virtual frame obtained by the frame synthesis prediction module 740 are suitable for use as reference images. In this case, a frame identifier may be assigned to images in a virtual frame. In the process of performing motion vector prediction for transmitting the motion vector, neighboring blocks of the current block may refer to images obtained by the frame synthesis prediction module 740. In this case, in order to use the image in the virtual frame as a reference image, a frame identifier indicating the image frame can be used.

Fig. 26 is a flowchart of a process for synthesizing a virtual frame image when performing inter-frame prediction in MVC according to an embodiment of the present invention.

With reference to FIG. 26, an image in a virtual frame is synthesized using images in a frame adjacent to that of the current image. The current image is then predicted using the synthesized image in the virtual frame. If so, a more accurate prediction is possible. In the case where the image in the virtual frame is synthesized, there is a special syntax indicating whether to perform the prediction of the current image by synthesizing the image in the virtual frame. If it is decided whether to predict the current image, more efficient coding is possible. Special syntax is defined as an interframe synthesis prediction identifier, which is described below. For example, an image in a virtual frame is synthesized by a cut level to determine a 'view_synthesize pred_flag' indicating whether to predict the current image. And, the image in the virtual frame is synthesized by the macroblock level to determine 'view_syn_pred_flag' indicating whether to perform prediction of the current image. If 'view_synthesize_pred_flag = 1', the current clipping synthesizes the clipping in the virtual frame using the clipping in the frame adjacent to that of the current clipping. It is then possible to predict the current clipping using a synthesized clipping. If 'view_synthesize_pred_flag = 0', clipping in the virtual frame is not synthesized. Similarly, if 'view_syn_pred_flag = 1', the current macroblock synthesizes the macroblock in the virtual frame using the macroblock in the frame adjacent to that of the current macroblock. It is then possible to predict the current macroblock using the synthesized macroblock. If 'view_syn_pred_flag = 0', the macroblock in the virtual frame is not synthesized. Therefore, in the present invention, the inter-frame synthesis prediction identifier indicating whether to receive an image in a virtual frame is extracted from the video signal. It is then possible to obtain an image in a virtual frame using this inter-frame synthesis prediction identifier.

As mentioned in the foregoing description, frame information for identifying an image frame and a list of reference images for inter-frame prediction can be used by inter prediction unit 700. And, they can also be used when performing weighted prediction. Weighted prediction is applicable to the process for performing motion compensation. Moreover, if the current image uses the reference image in a different frame, it is possible to perform weighted prediction more efficiently using frame information and a list of reference images for inter-frame prediction. Weighted prediction methods according to embodiments of the present invention are described below.

Fig. 27 is a flowchart of a method for performing weighted prediction according to a clipping type when encoding a video signal according to the present invention.

With reference to FIG. 27, weighted prediction is a method of scaling a sample of motion-compensated prediction data in a P-slice or B-slice macroblock. The weighted prediction method includes an explicit mode for performing a weighted prediction for the current image using the weighted coefficient information obtained from the information for the reference images, and an implicit mode for performing the weighted prediction for the current image using the weighted coefficient information obtained from the information for the distance between current image and one of the reference images. The weighted prediction method can be applied in an excellent manner according to the clipping type of the current macroblock. For example, in explicit mode, the weighted coefficient information may be changed according to whether the current macroblock in relation to which the weighted prediction is performed is a P-cut macroblock or a B-cut macroblock. And, the weighted explicit mode coefficient can be determined by the encoder and can be transmitted as included in the clipping header. On the other hand, in implicit mode, a weighted coefficient can be obtained based on the relative time position of List 0 and List 1. For example, if the reference image is close in time to the current image, a large weighted coefficient is applicable. If the reference image is timed away from the current image, a small weighted coefficient is applied.

First of all, the macroblock clipping type for applying weighted prediction to it is extracted from the video signal (S2710).

Then, a weighted prediction can be performed on the macroblock according to the extracted clipping type (S2720).

In this case, the clipping type may include a macroblock to which inter-frame prediction is applied. Inter-frame prediction means that the current image is predicted using information for an image in a frame different from that of the current image. For example, the clipping type may include a macroblock to which time prediction is applied to perform prediction using information for an image in the same frame as that of the current image, a macroblock to which interframe prediction applies, and a macroblock to which both apply from temporal prediction and interframe prediction. And, the clipping type may include a macroblock to which only temporal prediction applies, a macroblock to which only inter-frame prediction applies, or a macroblock to which both of temporal prediction and inter-prediction apply. In addition, the type of clipping may include two of the types of macroblock or all three types of macroblock. This is described in detail with reference to FIG. 28 below. Thus, in the event that the clipping type including the macroblock with the applied inter-frame prediction is extracted from the video signal, a weighted prediction is performed using information for an image in a frame different from that of the current image. In this case, the frame identifier for identifying the image frame can be used to use information for the image in a different frame.

FIG. 28 is a diagram of macroblock types permitted in a clipping type when encoding a video signal according to one embodiment of the present invention.

With reference to FIG. 28, if the P-cut type by inter prediction is defined as VP (View_P), the intra macroblock I, the macroblock P predicted from one image in the current frame, or the macroblock VP predicted from one image in a different frame, valid for this type of P-notch in accordance with inter-frame prediction (2810).

In the case where the type of B-cut according to inter-frame prediction is defined as VB (View_B), a macroblock P or B predicted from at least one image in the current frame, or a macroblock VP or VB predicted from at least one image in excellent frame is acceptable (2820).

In the case that the type of clipping for which the prediction is made using temporal prediction, inter-prediction, or both of temporal prediction and inter-prediction, is defined as “Mixed”, intra-macroblock I, macroblock P or B, predicted from at least one image in the current frame, a macroblock VP or VB predicted from at least one image in a different frame, or a 'Mixed' macroblock predicted using both of the image in the current frame and the image in a different frame, Permissible for mixed-type notch (2830). In this case, in order to use the image in an excellent frame, a frame identifier can be used to identify the image frame.

FIG. 29 and FIG. 30 are syntax diagrams for performing weighted prediction according to a newly defined clipping type according to one embodiment of the present invention.

As mentioned in the previous description of FIG. 28, if the clipping type is defined as VP, VB, or Mixed, the syntax for performing conventional weighted prediction (eg, H.264) can be modified in FIG. 29 or FIG. 30.

For example, if the cut type is P-cut according to temporal prediction, add the part 'if (slice_type! = VP Il slice_type! = VB)' (2910).

If the clipping type is a B-clipping according to temporal prediction, the conditional statement can be changed to 'if (slice_type == B Il slice_type == Mixed)' (2920).

By redefining the cut type VP and the cut type VB, a format similar to FIG. 29 can be added again (2930, 2940). In this case, since information for the frame is added, the syntax elements include parts of the 'frame', respectively. For example, there is 'luma_Iog2_view_weiht_denom, chroma_log2_view_weight_denom'.

FIG. 31 is a flowchart of a method for performing weighted prediction using flag information indicating whether to perform inter-frame weighted prediction when encoding a video signal according to the present invention.

With reference to FIG. 31, when encoding a video signal to which the present invention is applied, in the case of using flag information indicating whether weighted prediction will be performed, more efficient encoding is allowed.

Flag information can be determined based on the type of clipping. For example, there may be flag information indicating whether the weighted prediction will be applied to the P-clipping or SP-clipping, or flag information indicating whether the weighted prediction will be applied to the B-clipping.

In particular, flag information may be defined as 'weighted_pred_flag' or 'weighted_bipred_idc'. If 'weighted_pred_flag = 0', this indicates that weighted prediction does not apply to P-clipping and SP-clipping. If 'weighted_pred_flag = 1', this indicates that weighted prediction applies to P-clipping and SP-clipping. If 'weighted_bipred_idc = 0', this indicates that the default weighted prediction is applied to the B-clipping. If 'weighted_bipred_idc = 1', this indicates that explicit weighted prediction applies to the B-clipping. If 'weighted_bipred_idc = 2', this indicates that implicit weighted prediction is applied to the B-clipping.

When encoding a multi-frame video, flag information indicating whether a weighted prediction will be performed using the information for the inter-frame image can be determined based on the type of clipping.

First of all, the clipping type and flag information indicating whether an inter-frame weighted prediction will be performed is extracted from the video signal (S3110, S3120). In this case, the clipping type may include a macroblock to which temporal prediction to perform prediction using information for an image in the same frame as that of the current image is applied, and a macroblock to which inter-frame prediction is applied to perform prediction using information for an image in a frame different from that of the current image.

Then, a weighted prediction mode can be determined based on the extracted clipping type and the extracted flag information (S3130).

Then, a weighted prediction according to the determined weighted prediction mode (S3140) can be performed. In this case, the flag information may include flag information indicating whether a weighted prediction will be performed using information for an image in a frame different from that of the current image, as well as the aforementioned 'weighted_pred_flag' and 'weighted_bipred_flag'. This is described in detail with reference to FIG. 32 below.

Therefore, in the case where the clipping type of the current macroblock is a clipping type including a macroblock to which inter prediction is applied, more efficient coding is provided than the case of using flag information indicating whether the weighted prediction will be performed using the image information in excellent frame.

32 is a diagram for explaining a weighted prediction method according to flag information indicating whether to perform weighted prediction using information for an image in a frame different from that of the current image according to one embodiment of the present invention.

With reference to FIG. 32, for example, flag information indicating whether a weighted prediction will be performed using information for an image in a frame different from that of the current image can be defined as 'view_weighted_pred_flag' or 'view_weighted_bipred_flag'.

If 'view_weighted_pred_flag = 0', this indicates that weighted prediction does not apply to VP clipping. If 'view_weighted_pred_flag = 1', explicit weighted prediction is applied to the VP clipping. If 'view_weighted_bipred_flag = 0', this indicates that the default weighted prediction is applied to the VB clipping. If 'view_weighted_bipred_flag = 1', this indicates that explicit weighted prediction is applied to the VB clipping. If 'view_weighted_bipred_flag = 2', this indicates that the implicit default weighted prediction is applied to the VB clipping.

In the case where the implicit weighted prediction is applied to the VB clipping, the weighting coefficient can be obtained from the relative distance between the current frame and the excellent frame. In the case where the implicit weighted prediction is applied to the VB clipping, the weighted prediction can be performed using the frame identifier identifying the image frame, or the image index (POC) reproduced by considering the resolution of each frame.

The above flag information may be included in the image parameter set (PPS). In this case, an image parameter set (PPS) means header information indicating an encoding mode of all images (e.g., statistical encoding mode, initial value of a quantization parameter by an image unit, etc.). However, the set of image options does not apply to all images. If the image parameter set does not exist, the image parameter set that exists immediately before is used as header information.

33 is a syntax diagram for performing weighted prediction according to newly defined flag information according to one embodiment of the present invention.

With reference to FIG. 33, when encoding a multi-frame video to which the present invention is applied, in the case where a clipping type including a macroblock applied to inter-frame prediction and flag information indicating whether weighted prediction will be performed using image information in a frame different from that of the current image, are determined, it is necessary to determine which weighted prediction will be performed according to the type of clipping.

For example, if the clipping type, as shown in FIG. 33, extracted from the video signal, is P-clipping or SP-clipping, a weighted prediction can be performed if 'weighted_pred_flag = 1'. In the case where the cut type is B-cut, a weighted prediction can be performed if 'weighted_bipred_flag = 1'. In the case where the cut type is VP cut, a weighted prediction can be performed if 'view_weighted_pred_flag = 1'. In the case where the cut type is VB cut, a weighted prediction can be performed if 'view_weighted_bipred_flag = 1'.

Fig. 34 is a flowchart of a method for performing weighted prediction according to a NAL (network abstraction layer) block according to an embodiment of the present invention.

With reference to FIG. 34, first of all, the type of the NAL unit (nal_unit_type) is extracted from the video signal (S910). In this case, the type of NAL unit means an identifier indicating the type of NAL unit. For example, if 'nal_unit_type = 5', the NAL block is clipping the IDR image. And, IDR (Instant Decoding Recovery) image means the head image of a video sequence.

Then, it is checked whether the extracted NAL unit type is a NAL unit type for multi-frame video encoding (S3420).

If the type of NAL unit is the type of NAL unit for multi-frame video encoding, weighted prediction is performed using information for an image in a frame different from that of the current image (S3430). The type of NAL unit may be a type of NAL unit applicable to both scalable video encoding and multi-frame video encoding, or a type of NAL unit only for multi-frame video encoding. Thus, if the type of NAL unit is intended for encoding multi-frame video, weighted prediction should be performed using information for the image in a frame different from that of the current image. So, you need to define a new syntax. This is described in detail with reference to FIG. 35 and FIG. 36 below.

FIG. 35 and FIG. 36 are syntax diagrams for performing weighted prediction if the type of the NAL unit is for multi-frame video encoding, according to one embodiment of the present invention.

First of all, if the type of NAL block is the type of NAL block for encoding multi-frame video, the syntax for performing conventional weighted prediction (for example, H.264) may change to the syntax shown in FIG. 35 or FIG. 36. For example, reference numeral 3510 indicates a portion of the syntax for performing conventional weighted prediction, and reference numeral 3520 indicates a portion of the syntax for performing weighted prediction when encoding a multi-frame video. So, weighted prediction is performed by syntax part 3520 only if the type of the NAL block is the type of NAL block for multi-frame video encoding. In this case, since the information for the frame is added, each syntax element includes a 'Frame' part. For example, there is 'Iuma_view_log2_weight_denom, chroma_view_log2_weight_denom' or the like. And, reference numeral 3530 in FIG. 36 indicates a portion of the syntax for performing conventional weighted prediction and reference numeral 3540 in FIG. 36 indicates a portion of the syntax for performing weighted prediction in multi-frame video encoding. So, weighted prediction is performed by syntax part 3540 only if the type of the NAL block is the type of NAL block for multi-frame video encoding. Similarly, since information is added for the frame, each syntax element includes a 'frame' part. For example, there is 'luma_view_weight_l1_flag, chroma_view_weight_l1_flag' or the like. Thus, if the type of NAL unit for multi-frame video encoding is determined, more efficient encoding is allowed with respect to the method of performing weighted prediction using information for an image in a frame different from that of the current image.

Fig. 37 is a block diagram of an apparatus for decoding a video signal according to an embodiment of the present invention.

With reference to FIG. 37, a video signal decoding apparatus according to the present invention includes a clipping type extraction module 3710, a prediction mode extraction module 3720, and a decoding module 3730.

Fig. 38 is a flowchart of a method for decoding a video signal in the decoding device shown in Fig. 37, according to one embodiment of the present invention.

With reference to FIG. 38, a video signal decoding method according to one embodiment of the present invention includes a cut type extraction type and a macroblock prediction mode step S3810, and a current macroblock decoding step S3820 according to the cut type and / or macroblock prediction mode.

First, a prediction scheme used by an embodiment of the present invention is described to aid in understanding the present invention. A prediction scheme can be classified into intra (intra) frame prediction (e.g., prediction between images in the same frame) and inter-frame prediction (e.g., prediction between images in different frames). And, intra-frame prediction may be the same prediction scheme as conventional temporal prediction.

According to the present invention, the cut type extraction module 3710 extracts the cut type for the cut including the current macroblock (S3810).

In this case, a clipping type field (slice_type) indicating the clipping type for inter prediction and / or a clipping type field (view_slice_type) indicating the clipping type for inter prediction can be provided as part of the video signal syntax to provide the clipping type. This is described in more detail below with reference to FIGS. 6 (a) and 6 (b). And, each of the clipping type (slice_type) for intra prediction and the clipping type (view_slice_type) for inter prediction may indicate, for example, I-clipping type (I_SLICE), P-clipping type (P_SLICE), or B-clipping type (B_SLICE) .

For example, if the 'slice_type' of a particular cut is B-cut and the 'view_slice_type' is a P-cut, the macroblock in a specific cut is decoded by the B-cut encoding scheme (B_SLICE) in the intraframe direction (i.e., the time direction) and / or the P encoding scheme -cuts (P_SLICE) in the direction of the frame.

Meanwhile, the clipping type is able to include the P-clipping type (VP) for inter prediction, the B-clipping type (VB) for inter prediction, and the Mixed prediction type obtained from mixing both types of prediction. Namely, the mixed-cut type provides prediction using a combination of intra-frame and inter-frame prediction.

In this case, the type of P-cut for inter-frame prediction means the case where each macroblock or part of the macroblock included in the cut is predicted from one image in the current frame or one image in a different frame. The B-cut type for inter prediction means the case where each macroblock or part of the macroblock included in the cut is predicted from 'one or two images in the current frame' or 'one image in a different frame or two images in different frames, respectively'. And, a mixed type of clipping for a prediction obtained from mixing both predictions means the case when each macroblock or part of the macroblock included in the clipping is predicted from 'one or two images in the current frame', 'one image in a different frame or two images in different frames, respectively ', or' of one or two images in the current frame and one image in a different frame or two images in different frames, respectively '.

In other words, the reference (reference) image and the allowed type of macroblock are different for each type of clipping, as described in detail with reference to FIG. 43 and FIG. 44 below.

And, the syntax among the above embodiments of the type of clipping is described in detail with reference to FIG. 40 and FIG. 41 below.

The prediction mode extraction unit 3720 may retrieve a macroblock prediction mode indicator indicating whether the current macroblock is a macroblock in accordance with an intra-frame prediction, a macroblock in accordance with an inter-frame prediction, or a macroblock in accordance with a prediction obtained from mixing both types of prediction (S3820). To this end, the present invention defines a macroblock prediction mode (mb_pred_mode). One embodiment of macroblock prediction modes is described in detail with reference to FIGS. 39, 40 and FIG. 41 below.

Decoding unit 3730 decodes the current macroblock according to the clipping type and / or macroblock prediction mode to receive / receive the current macroblock. In this case, the current macroblock may be decoded according to the macroblock type of the current macroblock determined from the macroblock type information. And, the macroblock type can be determined according to the macroblock prediction mode and the type of clipping.

In the case where the macroblock prediction mode is a mode for intra-frame prediction, the macroblock type is determined according to the clipping type for intra-frame prediction, and the current macroblock is then decoded in accordance with the intra-frame prediction according to the determined macroblock type.

In the case where the macroblock prediction mode is a mode for inter prediction, the macroblock type is determined according to the clipping type for inter prediction, and the current macroblock is then decoded in accordance with the inter prediction according to the determined macroblock type.

In the case where the macroblock prediction mode is a prediction mode obtained from mixing both predictions, the macroblock type is determined according to the clipping type for intra-frame prediction and the type of clipping for inter-frame prediction, and the current macroblock is then decoded in accordance with the prediction obtained by mixing both predictions according to each from certain types of macroblocks.

In this case, the type of macroblock depends on the macroblock prediction mode and the type of clipping. In particular, the prediction scheme to be used for the macroblock type can be determined from the macroblock prediction mode, and the macroblock type is then determined from the macroblock type information according to the clipping type according to the prediction scheme. Namely, one or both of the extracted slice_type and view_slice_type are selected based on the macroblock prediction mode.

For example, if the macroblock prediction mode is the mode for inter-frame prediction, the macroblock type can be determined from the macroblock table of clipping types (I, P, B) corresponding to the clipping type (view_slice_type) for inter-frame prediction. The relationship between the macroblock prediction mode and the macroblock type is described in detail with reference to FIGS. 39, 40 and FIG. 41 below.

Fig. 39 is a diagram of macroblock prediction modes according to embodiments of an example of the present invention.

Fig. 39 (a) shows a table corresponding to one embodiment of the macroblock prediction modes (mb_pred_mode) according to the present invention.

In the case of intra-frame prediction, that is, only temporal prediction for the macroblock is used, '0' is assigned to the value 'mb_pred_mode'. In case only inter-frame prediction is used for the macroblock, '1' is assigned to the value 'mb_pred_mode'. In the case that both temporal distortion and inter-frame prediction are used for the macroblock, '2' is assigned the value 'mb_pred_mode'.

In this case, if the value of 'mb_pred_mode' is '1', that is, if 'mb_pred_mode' indicates inter-frame prediction, the frame direction List0 (ViewList0) or List1 frame direction (ViewList1) is determined as the reference picture list for inter-frame prediction.

Fig. 39 (b) shows the relationship between the macroblock prediction mode and the macroblock type according to another embodiment.

If the value of 'mb_pred_mode' is '0', only temporal prediction is used. And, the macroblock type is determined according to the type of clipping (slice_type) for intra-frame prediction.

If the value of 'mb_pred_mode' is '1', only inter-frame prediction is used. And, the macroblock type is determined according to the type of clipping (view_slice_type) for inter-frame prediction.

If the value of 'mb_pred_mode' is equal to '2', mixed prediction of both temporal and intra-frame prediction is used. And, two types of macroblocks are determined according to the type of clipping (slice_type) for intra-frame prediction and the type of clipping (view_slice_type) for inter-frame prediction.

Based on the macroblock prediction mode, the macroblock type is set based on the clipping type, as shown in tables 1-3 below. [Please insert tables 7-12 to 7-14 in N6540 here as tables 1-3].

In other words, in this embodiment, the prediction scheme used for the macroblock and the type of referenced clipping are determined by the macroblock prediction mode. And, the type of macroblock is determined according to the type of clipping.

Fig. 40 and Fig. 41 are diagrams of embodiments of an example syntax of a portion of a video signal received by a device for decoding a video signal. As shown, the syntax has a clipping type and macroblock prediction mode information according to an embodiment of the present invention.

On Fig shows an exemplary syntax. In this syntax, the 'slice_type' field and the 'view_slice_type' field provide clipping types, and the 'mb_pred_mode' field provides the macroblock prediction mode.

According to the present invention, the slice_type field provides a clipping type for inter prediction and the view_slice_type field provides a clipping type for inter prediction. Each type of cut can be an I-cut type, a P-cut type, or a B-cut type. If the value of 'mb_pred_mode' is '0' or '1', one type of macroblock is determined. However, if this value of 'mb_pred_mode' is equal to '2', it can be seen that another type of macroblock (or two types) is additionally defined. In other words, the syntax shown in (a) in FIG. 40 indicates that 'view_slice_type' is added to further apply the conventional clipping types (I, P, B) for multi-frame video encoding.

FIG. 41 shows another exemplary syntax. In this syntax, the 'slice_type' field is used to provide the clipping type, and the 'mb_pred_mode' field is used to provide the macroblock prediction mode.

According to the present invention, the slice_type field may include, but is not limited to, clipping type (VP) for inter prediction, type-B clipping (VB) for inter prediction, and a mixed clipping type (Mixed) for prediction obtained from mixing both intraframe and inter-frame predictions.

If the value in the 'mb_pred_mode' field is '0' or '1', one type of macroblock is determined. However, if the value in the 'mb_pred_mode' field is '2', it can be seen that an additional (that is, a total of two) type of macroblock is determined. In this embodiment, clipping type information exists in the clipping header, which is described in detail with reference to FIG. In other words, the syntax shown in FIG. 41 indicates that the cut types VP, VB and Mixed are added to the regular cut type (slice_type).

Fig. 42 is a diagram of examples for applying the cut types shown in Fig. 41.

The diagram in FIG. 42 (a) shows that a P-clipping type (VP) for inter prediction, a B-clipping type (VB) for inter prediction, and a mixed Mixed type for prediction obtained from mixing both predictions may exist as a clipping type, in addition to other clipping types, in the clipping header. In particular, the cut type VP, VB, and Mixed according to an embodiment of the example are added to the cut types that may exist in the common cut header.

The diagram in FIG. 42 (b) shows that a P-type clipping (VP) for inter prediction, a B-clipping type (VB) for inter prediction and a mixed prediction clipping type (Mixed) obtained from mixing both predictions may exist as the clipping type in the clipping header for encoding a multi-frame video (MVC). In particular, clipping types according to an exemplary embodiment are defined in a clipping header for encoding a multi-frame video.

The diagram in FIG. 42 (c) shows that the clipping type (VP) for inter prediction, the B-clipping type (VB) for inter prediction, and the mixed prediction clipping type (Mixed) obtained from mixing both predictions may exist as clipping type, in addition to the existing clipping type for scalable video encoding in the clipping header for scalable video encoding (SVC). In particular, the cut type VP, VB, and Mixed according to an embodiment of the example is added to the cut types that may exist in the cut header of the scalable video coding (SVC) standard.

Fig. 43 is a diagram of various examples of cut types included in the cut type shown in Fig. 41.

On Fig (a) shows the case when the type of clipping is predicted from one image in a different frame. So, the clipping type becomes the clipping type (VP) for inter-frame prediction.

On Fig (b) shows the case when the type of clipping is predicted from two images in different frames, respectively. So, the clipping type becomes the B-clipping (VB) type for inter-frame prediction.

Figs. 43 (c) and 43 (f) show the case where the clipping type is predicted from one or two images in the current frame and one image in a different frame. So, the type of clipping becomes a mixed type of clipping (Mixed) for the prediction obtained from mixing both predictions. Also, FIGS. 43 (d) and 43 (e) show a case where the cut type is predicted from one or two images in the current frame and two images in different frames. So, the clipping type also becomes the Mixed clipping type.

Fig. 44 is a macroblock diagram allowed for the cut types shown in Fig. 41.

With reference to FIG. 44, an intramacroblock (I), a macroblock (P) predicted from a single image in the current frame, or a macroblock (VP) predicted from a single image in a different frame, are allowed for the P-cut (VP) type in accordance with the interframe prediction.

Intramacroblock (I), macroblock (P or B) predicted from one or two images in the current frame, or macroblock VP or VB predicted from one image in a different frame or two images in different frames, respectively, are allowed for type B-clipping (VB) in accordance with inter-frame prediction.

And, intramacroblock (I); a macroblock (P or B) predicted from one or two images in the current frame; a macroblock (VP or VB) predicted from one image in a different frame or two images in different frames, respectively, or a macroblock (Mixed) predicted from one or two images in a current frame, one image in a different frame or two images in different frames , respectively, are allowed for the mixed type of cutting (Mixed).

Figures 45-47 are diagrams of a macroblock type for a macroblock existing in a mixed Mixed type according to embodiments of the present invention.

Figures 45 (a) and 45 (b) show configuration diagrams for the macroblock type (mb_type) and the sub-macroblock type (sub_mb_type) for the macroblock existing in the mixed cut, respectively.

Figures 46 and 47 show a binary frame of the macroblock prediction direction (s) existing in the mixed cut and the actual mixed cut prediction direction (s), respectively.

According to an embodiment of the present invention, a macroblock type (mb_type) is prepared by looking at the Partition_Size of the macroblock part and the Direction of predicting the macroblock part.

And, the sub-macroblock type (sub_mb_type) is prepared by looking at and the size (Sub_Partition_Size) of the sub-macroblock part and the prediction direction (Sub_Direction) of each sub-macroblock part.

With reference to FIG. 45 (a), 'Direction0' and 'Direction1' indicate a prediction direction of a first part of a macroblock and a prediction direction of a second part of a macroblock, respectively. In particular, in the case of the 8 × 16 macroblock, 'Direction0' indicates the prediction direction for the left side of the 8 × 16 macroblock, and 'Direction1' indicates the prediction direction for the right side of the 8 × 16 macroblock. The macroblock type configuration principle (mb_type) is described in detail below. First, the first two bits indicate the size of the Partition_Size of the corresponding macroblock, and a value of 0 ~ 3 is available for these first two bits. And, the four bits following the first two bits indicate the direction of prediction (Direction) in the case when the macroblock is divided into parts.

For example, in the case of a 16 × 16 macroblock, four bits indicating a macroblock prediction direction are attached behind the first two bits. In the case of a 16 × 8 macroblock, the four bits after the first two bits indicate the prediction direction (Direction0) of the first part, and the other four bits are attached to the four bits to indicate the prediction direction (Direction1) of the second part. Similarly, in the case of an 8 × 16 macroblock, eight bits are attached behind the first two bits. In this case, the first four bits of the eight bits appended to the first two bits indicate the prediction direction of the first part, and the next four bits indicate the prediction direction of the second part.

With reference to FIG. 45 (b), the prediction direction (Sub_Direction) of the sub-macroblock is used in the same manner as the prediction direction (Direction) of the part of the macroblock shown in FIG. 45 (a). The principle of configuring the sub-macroblock type (sub_mb_type) is described in detail below.

First, the first two bits indicate the size of the part (Partition_Size) of the corresponding macroblock and the second two bits following the two bits indicate the size of the part (Sub_Partition_Size) of the sub-macroblock of the corresponding macroblock. A value of 0-3 is available for each of the first and second two bits. Then, four bits attached to the second two bits indicate the direction of prediction (Sub_Direction) in the case where the macroblock is divided into parts of the sub-macroblock. For example, if the size (Partition_Size) of the part of the macroblock is 8 × 8, and if the size (Sub_Partition_Size) of the part of the macroblock is 4 × 8, the first two bits are 3, the second two bits are 2, the first four bits after the second two bits indicate the prediction direction for the left 4 × 8 block of two 4 × 8 blocks, and the second four bits after the first four bits indicate the prediction direction for the right 4 × 8 block.

With reference to FIG. 46, a macroblock prediction direction is created by four bits. And, you can see that each binary representation becomes equal to '1' according to the case of the image link in the left (L), top (T), right (R) or bottom (B) positions of the current image.

With reference to Fig. 47, for example, in the case where the prediction direction is top (T), the image located above in the frame direction of the current image is a reference. In the case where the prediction direction corresponds to all directions (LTRB), it can be seen that images in all directions (LTRB) of the current image are reference.

Fig. 48 is a block diagram of an apparatus for encoding a video signal according to an embodiment of the present invention.

With reference to FIG. 48 (shown), an apparatus for encoding a video signal according to an embodiment of the present invention. The device includes a macroblock type determining unit 4810, a macroblock forming unit 4820, and an encoding unit 4830.

Fig. 49 illustrates a flowchart of a method of encoding a video signal in the encoding device shown in Fig. 48, according to an embodiment of the present invention.

With reference to FIG. 49, a video signal encoding method according to an embodiment of the present invention includes a step S4910 for determining a first type of macroblock for intra-frame prediction and a second type of macroblock for inter-frame prediction, a step S4920 for generating a first macroblock having a first type of macroblock and a second macroblock, having a second macroblock type, a third macroblock forming step S4930 using the first and second macroblocks, and a macroblock type encoding step S4940 of the current macroblock and the pre-mode macroblock readings.

According to the present invention, the macroblock type determination unit 4810 determines a first macroblock type for intra-frame prediction and a second macroblock type for inter-frame prediction (S4910), as described in detail above.

Then, the macroblock generating unit 4820 generates a first macroblock having a first type of macroblock and a second macroblock having a second type of macroblock (S4920) using known prediction methods, and then forms a third macroblock using the first and second macroblocks (S4930). In this case, the third macroblock is formed according to the average value between the first and second macroblocks.

Finally, the coding unit 4830 encodes the macroblock type (mb_type) of the current macroblock and the macroblock prediction mode (mb_pred_mode) of the current macroblock, comparing the coding efficiencies of the first to third macroblocks (S4940).

In this case, there are various ways to measure coding efficiency. In particular, a method using RD cost (bit rate distortion) is used in this embodiment of the present invention. As is known, in the RD-value method, the corresponding cost is calculated with two components: the number of coding bits generated from the coding of the corresponding block, and the distortion value indicating an error from the actual sequence.

The first and second types of macroblocks can be determined by selecting a type of macroblock having a minimum value above the described RD cost. For example, a macroblock type having a minimum RD value among macroblock types according to intra-frame prediction is defined as a first macroblock type. And, a macroblock type having a minimum RD value among a macroblock type according to inter-frame prediction is defined as a second macroblock type.

In the coding step of the macroblock type and macroblock prediction mode, the macroblock type and prediction mode associated with these first and second macroblocks having lower RD cost can be selected. Then, the RD value of the third macroblock is determined. Finally, the macroblock type and macroblock prediction mode of the current macroblock are encoded by comparing the RD value of the selected first or second macroblock and the RD value of the third macroblock with each other.

If the RD value of the selected first or second macroblock is equal to or greater than the RD value of the third macroblock, the macroblock type becomes the macroblock type corresponding to the selected first or second macroblock.

For example, if the RD value of the first macroblock is less than that of the second and third macroblocks, the current macroblock is set as the first type of macroblock. And, the macroblock prediction mode (i.e., intra-frame prediction) becomes the macroblock prediction circuit corresponding to the RD cost.

For example, if the RD value of the second macroblock is less than that of the first and third macroblocks, the inter prediction scheme as the prediction scheme of the second macroblock becomes the macroblock prediction mode of the current macroblock.

Meanwhile, if the RD cost of the third macroblock is less than the RD cost of the first and second macroblocks, the types of macroblocks correspond to the first and second types of macroblocks. In particular, the macroblock types of intra-frame prediction and inter-frame prediction become the macroblock types of the current macroblock. And, the macroblock prediction mode becomes a mixed prediction scheme obtained from mixing intra-frame prediction and inter-frame prediction.

Accordingly, the present invention provides at least the following effect or advantage.

The present invention is capable of eliminating redundancy information between frames due to various prediction schemes between frames and information such as clipping types, macroblock types, and macroblock prediction modes; thus increasing coding performance / decoding efficiency.

Industrial applicability

Although the present invention has been described and illustrated with reference to preferred embodiments thereof, it will be apparent to those skilled in the art that various modifications and changes can be made without departing from the scope and spirit of the invention. Thus, it is intended that the present invention cover the modifications and variations of this invention as defined by the appended claims and their equivalents.

Claims (8)

1. A method of decoding a video signal, comprising the steps of:
checking the encoding scheme of the video signal;
obtaining configuration information for a video signal according to a coding scheme; recognition of the total number of frames using configuration information;
recognition of inter-frame reference information based on the total number of frames and
decoding a video signal based on inter-frame reference information, wherein the configuration information includes at least frame information for identifying a frame of the video signal. 2. The method according to claim 1, further comprising the step of checking the level for scalability of the frame of the video signal, wherein the video signal is decoded based on the level. 3. The method according to claim 1, in which the configuration information also includes a time level, a flag of a group of inter-frame images and a flag of instant update decoding (IDR). 4. The method according to claim 2, in which at the decoding stage, if the video signal corresponds to the main frame according to the level, decode the video signal of the main frame. 5. The method according to claim 1, further comprising the step of obtaining information for an image in a frame different from that of the current image using frame information, the information for the image in an excellent frame being used when decoding a video signal. 6. The method according to claim 5, in which the step of decoding the video signal further comprises the step of performing motion compensation using information for the image in the excellent frame, wherein the information for the image in the excellent frame includes motion vector information and reference image information. 7. The method according to claim 5, wherein the step of decoding the video signal further comprises the steps of: extracting an inter-frame synthesis prediction identifier indicating whether to obtain an image in a virtual frame using an image in a frame adjacent to that of the current image from the video signal; and acquiring the image in the virtual frame according to the prediction identifier of the interframe synthesis, wherein the information for the image in the excellent frame is used when acquiring the image in the virtual frame. 8. The method according to claim 5, in which the step of decoding the video signal further comprises the step of obtaining depth information between different frames using information for the image in said excellent frame.

Images