KR101276847B1 - Processing multiview video - Google Patents

Abstract

The present invention is directed to a method comprising: receiving a bitstream in which a plurality of viewpoints of a multiview video signal are encoded, extracting flag information related to a portion of the multiview video signal from the bitstream, and lighting compensation according to the flag information For this performed portion, extracting a value associated with a segment in the portion from the bitstream and determining whether illumination compensation of the segment is performed from the extracted value, wherein each of the plurality of views The view includes a plurality of pictures divided into a plurality of segments, wherein the flag information indicates whether illumination compensation is performed on segments within a portion of the multiview video signal. To provide.

Video, multi-view, profile

Description

Processing of multiview video {PROCESSING MULTIVIEW VIDEO}

The present invention relates to the processing of multiview video.

Multiview Video Coding (MVC) deals with compression standards for video images (eg, a series of images or pictures) acquired from multiple cameras. The video image or viewpoints may be coded according to a standard such as MPEG. The picture in the video image may represent a complete video frame or a field of video frames. A slice may be an independently coded portion of a picture that includes some or all macroblocks within the picture. In addition, the macroblock may include blocks composed of picture elements (or pixels).

The video images may be coded as multi-view video images according to the H.264 / AVC codec technology. In addition, many researchers are working with standard additional techniques to provide multi-view video images.

Currently, H.264 defines three profiles that support specific functions. Profile refers to the standardization of technical components that enter algorithms during video encoding / decoding. In other words, it is a kind of sub-standard that is a set of description elements necessary for decoding a bit string of a compressed image. The three profiles refer to a baseline profile, a main profile, and an extended profile. In order for the decoder to be compatible with each profile, various requirements for the encoder and decoder are defined in the H.264 standard.

Looking at the structure of the bit stream in H.264 / AVC, the network abstraction between the video coding layer (VCL) that deals with the video encoding process itself and the subsystem that transmits and stores the encoded information Layer, which is defined as a separate hierarchical structure. The output of the encoding process is VCL data and is mapped in units of NAL before transmission or storage. Each NAL unit includes raw video sequence payload (RBSP), which is data corresponding to compressed video data or header information.

NAL unit includes NAL header and RBSP. The NAL header may include flag information (eg, nal_ref_idc) and identification (ID) information (eg, nal_unit_type). The flag information indicates whether a slice serving as a reference picture of the NAL unit is included, and the identification information indicates the type of the NAL unit. The RBSP stores the compressed original data and adds an RBSP trailing bit at the end of the RBSP to express the length of the RBSP in multiples of 8 bits.

These NAL unit types include Instantaneous Decoding Refresh (IDR) pictures, Sequence Parameter Set (SPS), Picture Parameter Set (PPS), and Supplemental Enhancement Information (SEI). Etc.

In addition, the specification restricts the product to various profiles and levels so that the target product can be implemented at a reasonable cost. The decoder must satisfy the constraints defined in the profile and level.

As such, two concepts, profile and level, have been defined to represent the function or parameter of a compressed video range. Which profile the bitstream is based on may be identified by profile identification information profile_idc. The profile identification information means a flag indicating a profile related to the bitstream. The H.264 / AVC standard may include three profile identifications. For example, if the profile identification information is 66, the bitstream means that the bitstream is based on a baseline profile, if 77, which means that the bitstream is based on the main profile, and if it is 88, it means based on the extended profile. The profile identification information may be included in a sequence parameter set.

In order to efficiently handle a multiview image, the input bitstream may include information for causing a decoding device to determine whether an input bitstream is related to a multiview profile. If the input bitstream is identified as being associated with a multiview profile, it may be necessary to add a syntax to transmit one or more additional information about the multiview image. Here, the multiview profile identification information may indicate a profile mode that handles multiview video as an additional technique of H.264 / AVC.

 Since MVC technology is an additional technology to H.264 / AVC technology, it may be more efficient to add syntax as additional information for MVC mode than unconditional syntax. For example, when the profile identifier of the AVC indicates a multi-view profile, adding information about the multi-view image may increase encoding efficiency.

The sequence parameter set refers to header information that includes information that covers the entire sequence, such as profile and level.

Since the entire compressed video, i.e., the sequence, must start from the sequence header, the sequence parameter set corresponding to the header information must arrive at the decoder before the data referring to the parameter set. The sequence parameter set RBSP (S1 in FIG. 2) serves as header information for the result data of moving picture compression. When the bitstream is input, the profile identifier first identifies which of the plurality of profiles the input bitstream is based on.

Thus, by including profile identification information on the syntax that determines whether the input bitstream is for a multiview profile (eg, "If (profile_idc == MULTI_VIEW_PROFILE)"), the input bitstream is a multiview profile. It is possible to determine whether or not to and to add various attribute information when it is recognized as that for a multi-view profile.

FIG. 1 is a schematic block diagram of a decoding apparatus of a multiview video system for decoding a video signal including a multiview video image according to an embodiment to which the present invention is applied.

The multi-view video system may include a corresponding encoding device (encoder) for providing the multi-view video image. In this case, the multi-view video image is encoded image data contained in a vehicle of information readable by the device (for example, a device readable storage medium, or an energy signal readable by the device propagating between the transmitter and the receiver). It may be provided as a bitstream including a.

According to FIG. 1, the decoding apparatus includes a parsing unit 10, an entropy decoding unit 11, an inverse quantization / inverse transform unit 12, an inter prediction unit 13, an intra prediction unit 14, and deblocking. A filter section 15, a decoded picture buffer section 16, and the like.

The inter prediction unit 13 includes a motion compensator 17, an illumination compensator 18, an illumination compensation offset predictor 19, and the like.

 The parser 10 performs parsing on a NAL basis to decode the received video image. In general, one or more sequence parameter sets and picture parameter sets are transmitted to the decoder before the slice header and slice data are decoded. In this case, various attribute information may be included in the NAL header area or the extension area of the NAL header. For example, temporal level information, view level information, anchor picture identification information, view identifier information, and the like may be included.

Here, the temporal level information refers to information on a hierarchical structure for providing temporal scalability from a video signal. Through such temporal level information, it is possible to provide a user with images of various time zones.

The viewpoint level information refers to information on a hierarchical structure for providing viewpoint scalability from a video signal. In a multi-view video image, it is necessary to define the levels of time and view in order to provide a user with images of various times and views.

When defining the level information in this way, scalability with respect to time and time can be used. Accordingly, the user may view only an image of a desired time and time point, and may view only an image according to another constraint. The level information may be set differently in various ways according to the reference condition. For example, the level information may be set differently according to the position of the camera, or may be set differently according to the arrangement of the camera. In addition, the level information may be arbitrarily set regardless of a special criterion.

An anchor picture refers to an encoded picture in which all slices refer only to slices in frames of the same time zone. For example, an encoded picture refers to only a slice at another viewpoint and no slice at the current viewpoint. In the decoding process of a multiview image, random access between viewpoints may be required.

Access to any point in time should be allowed while minimizing the decryption effort. Here, anchor picture identification information may be needed to realize efficient random access.

The viewpoint identification information refers to information for distinguishing a picture at a current viewpoint from a picture at a different viewpoint. When a video image signal is coded, a picture order count (POC) and frame_num may be used to identify each picture.

In the case of a multiview video image, inter-view prediction may be performed. Thus, an identifier may be used to distinguish a picture at a current time point from a picture at a different time point.

A view identifier indicating a viewpoint of the picture may be defined. The view identifier may be used to obtain information about a picture at a different point in time from the current picture, and the video signal may be decoded using the picture information at a different point in time. This view identifier may be applied throughout the encoding / decoding process of the video signal. In addition, the frame_num may be applied to a multi-view video coding as it is, using a frame_num in consideration of a view rather than a specific view identifier.

In general, since a large amount of data of a multiview image is required, a hierarchical encoding (also called 'view scalability') function of each view may be required to solve this problem. In order to perform the view scalability function, a prediction structure considering the viewpoint of a multiview image may be defined.

The prediction structure may be defined by structuring a prediction order and a direction for a plurality of viewpoint images. For example, when images of various viewpoints to be encoded are given, the center of the entire array may be determined as a base view, and an image of a viewpoint to be encoded hierarchically may be selected. Alternatively, the end of the entire array or any other part can be determined as a reference point.

If the number of camera viewpoints is an exponential power of 2, a hierarchical prediction structure between each viewpoint image may be formed. Alternatively, when the number of camera viewpoints is not an exponential power of 2, a virtual viewpoint may be assumed and a prediction structure may be formed based on the case of the exponential power of 2 which is larger than the actual number and is the smallest. In addition, when the camera array is two-dimensional, it is possible to determine the prediction order alternately in the horizontal and vertical directions.

The parsed bitstream is entropy decoded by the entropy decoding unit 11, and coefficients, motion vectors, and the like of each macroblock are extracted. The inverse quantizer / inverse transformer 12 multiplies the received quantized value by a constant constant to obtain a transformed coefficient value, and inversely transforms the coefficient value to restore the pixel value. The intra prediction unit 14 performs the intra prediction from the decoded samples in the current picture by using the reconstructed pixel value.

The deblocking filter section 15 is applied to each coded macroblock to reduce block distortion. The filter smoothes the edges of the block to improve the quality of the decoded frame. The choice of filtering process depends on the boundary strength and the gradient of the image samples around the boundary. The filtered pictures are output or stored in the decoded picture buffer unit 16 for use as a reference picture.

The decoded picture buffer unit 16 stores or opens previously coded pictures in order to perform inter prediction. At this time, the frame_num and POC (Picture Order Count) of each picture are used to store or open the decoded picture buffer unit 16. Therefore, some of the previously coded pictures in MVC have pictures that are different from the current picture. Therefore, in order to utilize these pictures as reference pictures, not only the frame_num and the POC but also a view identifier indicating the picture's view point may be used. Can be.

The inter prediction unit 13 performs inter prediction using a reference picture stored in the decoded picture buffer unit 16. Inter-coded macroblocks can be divided into macroblock partitions, where each macroblock partition can be predicted from one or two reference pictures.

The motion compensation unit 17 compensates for the motion of the current block by using the information transmitted from the entropy decoding unit 11. A motion vector of blocks neighboring the current block is extracted from the video signal, and a motion vector predictor of the current block is obtained. The motion of the current block is compensated by using the obtained motion vector predictor and the difference vector extracted from the video signal. In addition, such motion compensation may be performed using one reference picture or may be performed using a plurality of pictures.

Therefore, when the reference pictures are pictures that are different from the current view, motion compensation may be performed using a view identifier indicating the view.

In addition, the direct prediction mode is a coding mode for predicting motion information of a current block from motion information of a coded block. This method improves compression efficiency because the number of bits necessary for encoding motion information is saved.

For example, in the temporal direct mode, a temporal direct mode predicts motion information of a current block by using motion information correlation in the time direction. Similar to this method, the decoder may predict the motion information of the current block by using the motion information correlation in the view direction.

In addition, when the input bitstream corresponds to a multi-view image, since the view sequences are images obtained from different cameras, illumination differences may occur due to internal and external factors of the camera. In order to prevent this, the illumination compensation unit 18 performs illumination compensation.

In performing lighting compensation, flag information indicating whether lighting compensation is performed on a predetermined layer of a video signal may be used. For example, lighting compensation may be performed using flag information indicating whether lighting of the slice or the macroblock is performed. In addition, in performing lighting compensation using the flag information, it may be applied to various types of macroblocks (eg, inter16 × 16 mode, B-skip mode, or direct prediction mode).

In addition, in performing the illumination compensation, information of a neighboring block or information on a block that is different from the current block may be used to restore the current block, and an offset value of the current block may be used. Here, the offset value of the current block refers to a difference between the average pixel value of the current block and the average pixel value of the reference block corresponding thereto. As an example of using the offset value, a predicate of an offset value of the current block may be obtained using neighboring blocks of the current block, and a difference between the offset value and the predictor may be used. Accordingly, the decoder may restore the offset value of the current block by using the difference value and the predictor.

In addition, in obtaining the predictor of the current block, information of the neighboring block may be used.

For example, the offset value of the current block may be predicted using the offset value of the neighboring block. Before this, it may be determined whether the reference number of the current block and the reference number of the neighboring block are the same. According to the check result, the illumination compensator 18 may determine which neighboring block to use or what value to use.

In addition, the lighting compensation unit 18 may perform lighting compensation by using the prediction type of the current block. When the current block is predictively coded using two reference blocks, the lighting compensation unit 18 uses the offset value of the current block. An offset value corresponding to each reference block may be obtained.

As such, inter predicted pictures and intra predicted pictures using lighting compensation, motion compensation, and the like are selected according to a prediction mode to reconstruct the current picture.

Hereinafter, specific embodiments of encoding / decoding methods applied to reconstruct the current picture will be described.

2 is a flowchart illustrating a video image encoding method according to an embodiment to which the present invention is applied.

Referring to FIG. 2, as an example of a video image encoding method, an average pixel value of at least one block among reference image blocks at different points of time from blocks adjacent to the current block is obtained (S131). The predicted average pixel value of the current block is derived using one or more modes among a plurality of modes with the obtained value (S132). In operation S133, a difference value between the illumination average prediction value and the illumination average value of the current block is obtained. The coding efficiency of each of the plurality of modes is measured and an optimal mode is selected (S134). Examples of the method of selecting the optimal mode include a method of selecting a mode in which the error value is minimum among the obtained error values, or a method using a rate-distortion (RD) relational expression.

Here, the RD relational expression calculates the cost using two components, the number of encoded bits generated when the corresponding block is encoded and a distortion value representing an error between the actual video. Specifically, it can be obtained by multiplying the number of bits by the Lagrangian multiplier determined by the quantization coefficient and adding the distortion value. After the optimal mode is selected, only the identification information representing the selected mode may be encoded and transmitted, or the error value obtained by the selected mode may be encoded and transmitted together with the identification information representing the selected mode (S135).

FIG. 3 is a block diagram illustrating a process of deriving a predicted average pixel value of a current block from a reference image block at another point in time according to an embodiment of the present invention.

블록 픽셀값들의 평균값을

,

의 평균값을

라고 하고, 나머지 블록들의 평균값도 상기 블록 표시법(notation)에 준해서 표현한다. 주변 정보를 이용하는 방법에 따라

정보를 예측하는 방법이 다양하다.

블록을 부호화할 시, 참조 영상#1(reference frame#1)이 후보 참조 영상(reference frame)이 되는 경우라고 가정하자.According to FIG. 3,

The average of the block pixel values

,

Mean of

The average value of the remaining blocks is also expressed according to the block notation. Depending on how you use your surrounding information

There are many ways to predict information.

When encoding a block, it is assumed that reference picture # 1 becomes a candidate reference frame.

정보를 예측하는 방법의 일실시예로서, 현재 블록에 대응되는 다른 시점의 참조 영상 블록의 조명 평균값으로부터 예측하는 경우(Mode1)를 들 수 있다. 즉, 도 3에서 참조 영상#1(reference frame#1)의 블록

의 평균값으로 예측하는 경우이다. 오차값(difference value)은 다음 수학식 1과 같다.Depending on how you use your surrounding information

As an example of the method of predicting the information, the prediction may be performed from a lighting average value of the reference image block of another view corresponding to the current block (Mode1). That is, the block of the reference picture # 1 (reference frame # 1) in FIG.

It is a case of predicting by the average value of. The difference value is represented by Equation 1 below.

의 조명 평균값과의 차이를, 인접한 블록들(

,

) 끼리의 조명 평균값 차이로 예측하는 경우이다. 오차값은 다음 수학식 2와 같다.According to another embodiment, when the difference between the illumination average value of the current block and the reference image block at different views corresponding thereto is predicted from the difference in illumination average values between adjacent blocks of the current block and the reference image block (Mode2) Can be mentioned. That is, the block of the reference frame # 1 (reference frame # 1) in FIG.

The difference between the average of the illumination and the adjacent blocks (

,

) This is the case of predicting the difference between the average lighting values. The error value is shown in Equation 2 below.

의 조명 평균값과 참조 영상#1( reference frame#1)의 블록

의 조명 평균값 차이로부터 예측하는 경우이다. 오차값은 다음 수학식 3과 같다.As another example, the prediction may be performed based on a difference between an illumination average value of a block adjacent to the current block and an illumination average value of a reference image block at another view corresponding to the current block (Mode3). That is, adjacent blocks

The average value of illumination and the block of reference image # 1 (reference frame # 1)

It is a case where it predicts from the illumination average value difference of. The error value is shown in Equation 3 below.

이 이미 참조 영상#2(reference frame#2)의 블록

을 참조해서 부호화된 경우, 이웃 블록

의 조명 평균값 예측 차이로 예측하는 경우이다. 오차값은 다음 수학식 4와 같다.As another embodiment, a case in which a block adjacent to the current block is encoded from an adjacent block of a reference image block at another point in time may be predicted from the illumination average value prediction difference of the block adjacent to the current block (Mode4). In other words,

This block of reference image # 2 (reference frame # 2)

If encoded with reference to neighboring block

This is the case of predicting the difference in the prediction of the lighting average. The error value is shown in Equation 4 below.

When the adjacent block information is used in the method of Mode2, Mode3, or Mode4, only the information of one block above is used as an example, but in general, information of various neighbors surrounding the current block may be used in combination.

FIG. 4 is a block diagram for generalizing and describing a process of deriving an illumination average prediction value of a current block from a reference image block at another time point as an embodiment to which the present invention is applied.

4 shows already coded blocks that share a boundary with the current block, and blocks that share a boundary with the reference block. At this time, the equations of the Mode2, Mode3, and Mode4 methods may be generalized as shown in Equation 5 below.

Mode 2 :

Mode 2:

Mode 3:

Mode 4:

는

블록의 참조 블록이 참조 프레임(reference frame) #

에 있을 때, 이 블록의 조명 평균값을 나타낸다. In the generalized expression of Mode 4,

The

The reference block of the block is the reference frame #

When is on, it represents the average lighting value of this block.

는 가중치가 되며, 예측에 이용하는 인접한 블록은 상기와 같이 경계선을 공유하는 것들만으로 국한되지 않으며, 상기 이웃들의 이웃들도 포함될 수 있으며, 또한 그 중 일부만 활용할 수도 있다. 이는

로 조절할 수 있다. 이렇게 구한 오차값

는 양자화한 후, 엔트로피 부호화하여 전송한다.In the above formulas

Is a weight, and adjacent blocks used for prediction are not limited to those that share a boundary as described above, and may include neighbors of the neighbors, and may also utilize only some of them. this is

Can be adjusted with The error value

After quantization is performed, entropy encoding is performed.

Determination of the reference frame of the modes Mode1, Mode2, Mode3, and Mode4 is calculated up to the actual bitstream level and found to be optimal in terms of rate and distortion. Examples of the method of selecting the optimal mode include a method of selecting a mode in which the error value is minimum among the obtained error values, or a method using a rate-distortion (RD) relational expression.

In the method using the rate-distortion (RD) relation, all the steps for calculating the actual bitstream for each mode are actually selected, and then the mode that is optimal in terms of rate and distortion is selected. The difference from the conventional method is that when calculating the block residual value, the difference value is calculated from the limits of the average value of each block in the current block and the reference block. That is, the following equation (6).

는 변이 벡터(disparity vector)를 나타내고,

는 조명값(pixel value)을 나타낸다.

는 주변 정보를 통해서 예측한 값과, 오차 값을 양자화한 후, 다시 복원된 값을 더해서 구한 값으로서, 이는 인코더와 디코더에서 동일한 값을 얻기 위함이다.

은 참조 블록(reference block)의 조명 평균값이며, 복호된 영상이므로 인코더와 디코더에서 동일한 값이 된다. 그리고, 실제 적용에서는 시간적으로도 참조를 찾아서 한 후, 시공간 영역에서 최적인 것을 찾게 된다. 따라서, 조명 보상(illumination compensation)을 적용할지 여부의 식별 정보 또한 각 프레임과 각 블록에 대해서 0 또는 1로 지정이 되며, 이 또한 엔트로피 부호화를 적용한다.here,

Represents a disparity vector,

Denotes a pixel value.

Is a value obtained by quantizing an error value, an error value, and then reconstructing the value. This is to obtain the same value at the encoder and the decoder.

Is an average value of illumination of a reference block, and since the decoded image is the same value in the encoder and the decoder. In actual application, the reference is also found in time, and then the optimum one is found in the space-time domain. Therefore, identification information of whether to apply illumination compensation is also designated as 0 or 1 for each frame and each block, and this also applies entropy coding.

After the optimal mode is selected, only the selected mode may be encoded and transmitted. In addition, the error value obtained by the selected mode together with the selected mode may be encoded and transmitted. The selected mode information is expressed as an index, and this information may also be predicted from neighboring mode information, and an error value between the index of the currently selected mode and the index value of the predicted mode may be encoded and transmitted. have.

All of the above modes may be considered, some may be selected, or only one of these modes may be applied. In all possible cases, if only a single method is used, the mode index does not need to be separately coded.

As another embodiment to which the present invention is applied, in deriving the average lighting value and deriving the average lighting prediction value, pixel values that have already been decoded may be used for current blocks that are encoded in a reference image and an image to be encoded.

Basically, when predicting the illumination average value of the current block, the already decoded values of the left and top pixels are used. The illumination average of these pixels can be predicted as the illumination average of the current block. When encoding an actual video image, encoding is performed based on a macro block. In addition, the 16x16 macroblock may be divided into 16x8, 8x16, and 8x8 and encoded according to the complexity of the image, and each 8x8 block may be split into 8x4, 4x8, and 4x4 blocks in the same manner. In each case, a method of predicting an illumination average value of small blocks at each position based on one macro block may vary.

FIG. 5 is a diagram illustrating a 16x16 macroblock illustrating an example of using previously decoded values of pixels on the left and top of a block in obtaining an illumination mean value and deriving an illumination mean prediction value according to an embodiment of the present invention. will be.

According to FIG. 5, a 16x16 macroblock may use all pixel values at the top and left sides. Therefore, when predicting the average illumination value of the current block, it is possible to calculate the illumination average value of all the pixels (v1 ~ v16, h1 ~ h16) on the top, left of the block to predict the average illumination of the current block from it. In this case, the average lighting value of the B16x16 block is expressed by Equation 7 below.

6A and 6B illustrate exemplary embodiments to which the present invention may be applied. In the case where a block is divided, in the obtaining of an illumination average value of the divided block and deriving an illumination average prediction value, all of the pixels surrounding the divided block are used. 16x8 macroblocks for explaining the case of using only pixels surrounding the divided block, respectively.

In the case of FIG. 6A, that is, when all pixels surrounding the divided block are used, the average value of the B16x8_0 and B16x8_1 blocks is expressed by Equation 8 below.

In the case of FIG. 6B, that is, when only pixels surrounding the divided block are used, the illumination average values of the B16x8_0 and B16x8_1 blocks are represented by Equations 9 and 10, respectively.

도 포함하여 계산할 수도 있다. 이 경우 상기 도 6a 및 도 6b 경우의 B16x8_0 조명 평균값은 각각 다음 수학식 11,12와 같다.Also, in the above cases, the corner

It can also be calculated. In this case, the average B16x8_0 illumination value in the case of FIGS. 6A and 6B is represented by Equations 11 and 12, respectively.

와

포함하여 계산할 수도 있다. 이 경우 상기 도 6a 및 도 6b 경우의 B16x8_1 조명 평균값은 각각 다음 수학식 13,14와 같다.Also, in the above cases, the corner

Wow

It can also be calculated. In this case, the average illumination value of B16x8_1 in the case of FIGS. 6A and 6B is represented by Equations 13 and 14, respectively.

7A and 7B illustrate an embodiment to which the present invention is applied. In the case where a block is divided, in the obtaining of an illumination average value of the divided block and deriving an illumination average prediction value, the pixels surrounding the divided block are used. 8x16 macroblocks for explaining the case of using only pixels surrounding the divided block are illustrated. The method of obtaining the average illumination value of each divided block is the same as the method obtained in FIGS. 6A and 6B.

8A and 8B illustrate an embodiment to which the present invention is applied. In the case where a block is divided, in the obtaining of an illumination average value of the divided block and deriving an illumination average prediction value, the pixels surrounding all of the divided blocks are used. 8x8 macroblocks for explaining the case of using only pixels surrounding the divided block are shown. The method of obtaining the average illumination value of each divided block is the same as the method obtained in FIGS. 6A and 6B.

Like the above methods, the 8x8 block may be further divided into subblocks, and the same method may be applied.

,

이라고 한다. In this way, the illumination average values of the current block of the image to be encoded and the corresponding pixel of the reference block are predicted.

,

.

After subtracting the respective illumination average prediction values for all the pixels in each block, the error values of the two blocks can be calculated as in Equation 15 below.

는 변이 벡터(disparity vector)를 의미하고,

는 픽셀값을 표현한다. 이 블록 오차가 가장 작은 참조 영상의 블록을 조명(illumination)이 보상된 최적 블록으로 선택한다. 이때, 추정된 변이 벡터(disparity vector)는

가 된다. 실제 시스템에서는 조명 보상(illumination compensation)을 하지 않는 경우와 결과를 비교하여, 성능이 좋은 것을 선택한다.here,

Means a disparity vector,

Represents a pixel value. The block of the reference image having the smallest block error is selected as the optimal block whose illumination is compensated. In this case, the estimated disparity vector is

. In a real system, the results are compared with those without illumination compensation, and the one with the best performance is selected.

In one variation of the above scheme, the illumination average value of the reference picture block is calculated directly from the illumination average value of all pixels in the actual block, rather than predicted with the surrounding pixel values.

Another variation is the case of increasing the number of upper and left used pixels. That is, not only pixels of one adjacent layer but also pixels of two or more adjacent layers may be used.

에 대한 복호된 값을 먼저 구하고, 상기 예측 방법에 따라 예측 값을 구할 수 있다. 그리고 이 두 값을 더함으로써

(=

+ e)를 복호할 수 있다. 소위 상기 현재 블록의 예측자(predictor)로 불리는 예측 블록인 참조 블록(Reference block)으로부터

을 빼 준 후, 차분 블록(residual block)값의 복호된 값에 더해줌으로써 현재 블록값을 최종적으로 구할 수 있다. 상기 현재 블록은 다음과 같이 복원될 수 있다.The decoder may determine whether illumination compensation of the corresponding block is performed through the identification information. If you perform lighting compensation, the error value

A decoded value for may be obtained first, and a prediction value may be obtained according to the prediction method. And by adding these two values

(=

+ e) can be decoded. From a reference block, which is a prediction block called a predictor of the current block.

After subtracting, the current block value can be finally obtained by adding to the decoded value of the residual block value. The current block may be restored as follows.

+ e), 여기서 B는 현재 블록값을 나타내고, 예측 블록은 상기 현재 블록의 예측자를 나타낸다. 그리고,

은 상기 현재 블록의 조명 보상 오프셋 예측값인 평균 픽셀값의 예측된 차이값을 나타내고, e는 상기에서 설명한 오차값을 나타낸다. 디코딩부는 상기 현재 블록의 조명 보상 오프셋값과 예측된 차이값 사이의 오차값을 획득할 수 있다. 그리고, 상기 획득된 오차값과 상기 예측된 차이값을 이용하여 상기 현재 블록의 조명 보상 오프셋 값을 복원할 수 있다.B = prediction block + residual block + (

+ e), where B represents a current block value and a predictive block represents a predictor of the current block. And,

Denotes a predicted difference value of an average pixel value that is an illumination compensation offset prediction value of the current block, and e denotes an error value described above. The decoding unit may obtain an error value between the illumination compensation offset value and the predicted difference value of the current block. The illumination compensation offset value of the current block may be restored using the obtained error value and the predicted difference value.

9 is a diagram for describing a process of obtaining an offset value of a current block according to an embodiment to which the present invention is applied.

Illumination compensation may be performed in a motion estimation process. When comparing the similarity between the current block and the candidate reference block, the lighting difference between the two blocks should be considered. New motion estimation / motion compensation is performed to compensate for the illumination differences. The new SAD can be obtained by using Equations 16 and 17 below.

Here, M _c represents an average pixel value of the current block and M _r represents an average pixel value of the reference block. I _c (x, y) represents the pixel value at the (x, y) coordinate of the current block, and I _r (x + Δx, y + Δy) is the motion vector (Δx, Δy) of the reference block. Indicates a pixel value. By performing motion estimation based on the new SAD of Equation 16, the average pixel value difference between the current block and the reference block can be obtained. The obtained average pixel value difference is referred to as an offset value IC_offset.

When motion estimation with illumination compensation is performed, an offset value and a motion vector are obtained, and illumination compensation may be performed using Equation 18 using the offset value and the motion vector.

Here, R (x, y) represents an error value in which illumination compensation is performed.

The offset value (IC_offset = M _c M _r ) may be transmitted to the decoding unit. Illumination compensation in the decoding unit may be performed as in Equation 19 below.

R '(x, y) denotes a two restored, the illumination compensation values to perform the error _{(residual), I' c (} x, y) represents the pixel value of the restored current block.

In order to recover the current block, an offset value must be transmitted to the decoding unit, and the offset value can be predicted from information of neighboring blocks. In order to further reduce the number of bits to code the offset value, only the difference value R _{IC_offset} between the offset value IC_offset of the current block and the offset value IC_offset_pred of the neighboring block may be sent. This is shown in Equation 20 below.

R _{IC _ offset} = IC_offset-IC_offset_pred

FIG. 10 is a flowchart illustrating a process of performing lighting compensation on a current block according to an embodiment to which the present invention is applied.

When the lighting compensation flag of the current block is 0, lighting compensation for the current block is not performed. If the flag is 1, a process of restoring the offset value of the current block is performed. At this time, in obtaining the predictor of the current block, information of the neighboring block may be used. First, it is determined whether the reference number of the current block is the same as that of the neighboring block (S210). Based on the determination result, a predictor for lighting compensation of the current block is obtained (S211). The offset value of the current block is restored using the obtained predictor (S212). Here, it is necessary to examine in more detail the step S211 of determining whether the reference number of the current block and the reference number of the neighboring block are the same (S210) and acquiring a predictor based on the determination result. This will be described with reference to FIG. 11.

FIG. 11 is a flowchart illustrating a method of acquiring a predictor based on whether a reference number is identical between a current block and a neighboring block according to an embodiment to which the present invention is applied.

The decoding unit extracts flag information and offset values of neighboring blocks of the current block, reference numbers of corresponding reference blocks of the current block and its neighboring blocks, and the like from the video signal to perform lighting compensation, and uses the current block Obtain a Lock Predictor. The offset value of the current block and the predictor may be obtained, and the offset value of the current block may be restored using the obtained difference value and the predictor.

In this case, in obtaining the predictor of the current block, information of the neighboring block may be used. For example, the offset value of the current block may be predicted by using the offset value of the neighboring block. Before this, whether the reference number of the current block is the same as the reference number of the neighboring block is determined. According to the checking result, which neighboring block or a value to use may be determined. In addition, it is possible to check whether the flag information of the neighboring block is true, and determine whether to use the neighboring block according to the check result.

In a first embodiment, it is determined whether a neighboring block having the same reference number as the reference number of the current block exists (S220). As a result of the determination, when only one neighboring block having the same reference number as the reference number of the current block exists, the offset value of the neighboring block having the same reference number is allocated to the predictor of the current block (S221). When two neighboring blocks having the same reference number as the reference number of the current block exist according to the determination result of step S220, an average value of offset values of two neighboring blocks having the same reference number is assigned to the predictor of the current block. (S222). In addition, when there are three neighboring blocks having the same reference number as the reference number of the current block according to the determination result of step S220, the median of the offset values of the three neighboring blocks having the same reference number is present. It is allocated to the predicate of the block (S223). In addition, when there is no neighbor block having the same reference number as the reference number of the current block according to the determination result of step S220, the predictor of the current block is allocated to "0" (S224). In step S220 of checking the sameness of the above reference numbers, a condition of checking whether the flag of the corresponding neighboring block is 1 may be included.

As a second embodiment, it is checked whether the neighboring block has the same reference number as that of the current block, and whether the flag of the neighboring block is 1 is checked. If there is a neighboring block that satisfies both of the above conditions, the offset value of the neighboring block may be a predictor. In this case, the order of checking the neighboring blocks may be in the order of left-> top-> upper right-> upper left, or in the order of upper-> left-> upper right-> upper left. If there is no neighboring block that satisfies both conditions, and the flags of the three neighboring blocks (for example, the flags of the three neighboring blocks on the left, top, top right (or top left)) are 1 If the median of the offset values of the three neighboring blocks is a predictor, and this is also not satisfied, the predictor of the current block may be allocated as "0".

12 is a flowchart illustrating a method of performing lighting compensation based on a prediction type of a current block according to an embodiment to which the present invention is applied.

The neighboring block referred to may vary according to the prediction type of the current block. For example, when the shape of the current block and the neighboring block is the same, the current block may perform prediction using the median of the neighboring blocks. However, other methods may be applied when the shapes are different.

For example, when a block on the left side of the current block is divided into several blocks, the block located at the top of the block may be used for prediction. Alternatively, when the block on the top of the current block is divided into several blocks, the leftmost block among them may be used for prediction. As such, the prediction value may vary depending on which neighboring block the prediction type of the current block uses. Therefore, in an embodiment of the present invention, first, the neighboring block to be referred may be determined according to the prediction type of the current block (S231). It may be determined whether the reference number of the determined neighboring block is the same as the reference number of the current block (S232). Here, the process of checking whether the same reference number is the same may include a condition of checking whether the flag of the corresponding neighboring block is 1 (S232). A predictor for lighting compensation of the current block may be obtained based on the determination result (S233). Lighting compensation may be performed by restoring the offset value of the current block by using the obtained predictor (S234). Here, a process of performing step S233 based on the result of step S232 will be described with reference to specific embodiments. Specific embodiments of this part may be applied in a manner similar to that described with reference to FIG. 11.

For example, when the prediction type of the current block performs prediction by referring to the left block of the current block, it is determined whether the reference number of the left block of the current block is the same as the reference number of the current block. As a result of the determination, when the reference numbers are the same, the offset value of the left block is allocated to the predictor of the current block. In addition, when the prediction type of the current block performs prediction by referring to the left block and the top two blocks of the current block, or the left block and the top block and the top right block of the current block, respectively. Is applied in the same manner as described in FIG.

FIG. 13 is a flowchart illustrating a method of performing lighting compensation using flag information indicating whether to perform lighting compensation of a corresponding block in an embodiment to which the present invention is applied.

In another embodiment of the present invention, when restoring the offset value of the current block, flag information IC_flag indicating whether the current block performs lighting compensation may also be used. Alternatively, the predictor may be obtained using both the method of confirming the reference number described with reference to FIG. 11 and the method of predicting flag information. First, it may be determined whether a neighboring block having the same reference number as the reference number of the current block exists (S241). A predictor for lighting compensation of the current block may be obtained based on the determination result. In this case, whether the flag of the neighboring block is 1 may be included as a criterion of determination (S242). In addition, the flag information of the current block may be predicted based on the determination result (S243). Lighting compensation may be performed by restoring the offset value of the current block by using the obtained predictor and the predicted flag information (S244). Here, the step S242 may be applied in the same manner as described in FIG. The step S243 will be described in detail later with reference to FIG. 14.

FIG. 14 is a flowchart illustrating a method of predicting flag information of a current block based on whether a reference number of a current block and a neighboring block is the same as an embodiment to which the present invention is applied.

In an embodiment of the present invention, first, it may be determined whether a neighboring block having the same reference number as that of the current block exists. As a result of the determination, when only one neighboring block having the same reference number as the reference number of the current block exists, the flag information of the current block may be predicted from the flag information of the neighboring block having the same reference number (S251). When there are two neighboring blocks having the same reference number as the reference number of the current block according to the determination result of step S250, the current block may be selected from the flag information of any one of flag information of two neighboring blocks having the same reference number. The flag information can be predicted (S252).

In addition, when there are three neighboring blocks having the same reference number as the reference number of the current block according to the determination result of step S250, the current value is determined from the median of the flag information of the three neighboring blocks having the same reference number. The flag information of the block can be predicted (S253). In addition, when there is no neighbor block having the same reference number as the reference number of the current block according to the determination result of step S250, flag information prediction of the current block may not be performed (S254).

15 is a flowchart illustrating a method of performing lighting compensation when a current block is predictively coded using two reference blocks according to an embodiment to which the present invention is applied.

In performing the illumination compensation, when the current block is predictively coded using two reference blocks, the decoder cannot directly know an offset value corresponding to each reference block. This is because, when the offset value of the current block is obtained, the pixel value obtained by averaging the two reference blocks is used. Therefore, in an embodiment of the present invention, an offset value corresponding to each reference block may be obtained to enable more accurate prediction. First, the offset value of the current block is restored using the predictor and the error value of the current block (S261). If the current block is predictively coded using two reference blocks, an offset value corresponding to each reference block is obtained using the offset value (S262). This is shown in Equation 21 below.

IC_Offset = - * - *

IC_Offset =-*-*

-

= IC_Offset + (

- 1)*

+

*

IC_OffsetL0 =

-

= IC_Offset + (

- One)*

+

*

-

= IC_Offset +

*

+ (

- 1)*

.IC_OffsetL1 =

-

= IC_Offset +

*

+ (

- One)*

.

Here, m _c represents the average pixel value of the current block, and m _{r , 1} , m _{r , 2} represent the average pixel value of the reference block, respectively. w ₁ and w ₂ represent weight coefficients in bi-predictive coding.

As an embodiment of the illumination compensation method, the system can separately obtain and use an accurate offset value corresponding to each reference block, thereby enabling more accurate prediction coding. When restoring the offset value of the current block (S262), the offset value may be obtained by adding the restored offset difference value and the predictor value. In this case, a predictor for each of the reference picture of List 0 and each of the reference pictures of List 1 may be obtained, and a combination of these may be used to obtain a predictor required for restoring the offset value of the current block.

As another embodiment, the present invention may be applied to a skip macroblock. In this embodiment, prediction is performed to obtain information regarding lighting compensation. The flag information indicating whether the lighting compensation is performed may use a value predicted in the neighboring block, and the offset value of the current block may use an offset value predicted in the neighboring block. For example, if the flag information is true, the offset value is added to the reference block. As a specific example, in the case of the macroblock to which the P-Skip mode is applied, the flag and the offset value of the macroblock may be obtained by predicting using the flags and the offset values of the neighboring blocks on the left and the top. If only one block flag is 1, the flag and offset value of the current block may be set to the flag and offset value of the block. If both blocks have the flag 1, the flag of the current block is 1, and the offset value may be set to an average value of offset values of two neighboring blocks.

As another embodiment, the present invention may be applied to a direct prediction mode (eg, Direct, B-Skip mode). In this embodiment, prediction is performed to obtain information regarding lighting compensation. Each of the predictors can be obtained by applying the various flag and offset prediction methods described above. You can set this value to the actual flag and offset value of the current block. If each block has only a pair of flag and offset information, one prediction value for each block may be obtained. In this case, when there are two reference blocks, when the reference number is checked, it is checked whether each reference number of the current block and the neighboring block is identical. In addition, when each reference block has an offset value, the predicted first flag information, the predicted first offset value, the predicted second flag information, and the predicted second offset value may be obtained. In this case, the flag information may write a predicted value in the neighboring block, and the offset values of the two reference blocks may write the predicted first offset value and the second offset value, respectively. Here, the offset value of the current block may use an average value of the offset values of each reference block.

In the case of the direct prediction mode and the skip macroblock, flag information may be encoded / decoded. That is, the offset value may or may not be added according to the flag value. In addition, an error value between the offset value and the predicted offset value may be encoded / decoded. In this case, it can be restored more accurately, and it is possible to select the optimal one in terms of RD. In addition, when the reference picture is not available in the prediction process described herein, that is, when the reference picture number is less than 1, the flag information or the predicted flag information may be false, and the offset value or the predicted offset value may be It can be set to zero.

As another embodiment, the present invention may be applied to entropy coding. Regarding the flag information, three contexts can be considered according to the flag value of the neighboring block (for example, the left block and the upper block of the current block).

If the flag value is true, it is converted to 1, and if it is false, it is converted to 0. Flag information can be encoded / decoded using these three contexts. For the prediction error value of the offset, the same method as in transform coefficient levels coding can be used, for example. That is, binarization may be performed to UEG0, and one context model may be applied to the first bin value and one context model may be applied to the remaining bin values of the unary prefix part. The sign bit may be encoded / decoded in a bypass mode. As another embodiment of the flag information, two contexts may be considered according to the predicted value of the flag information and may be encoded / decoded using the context.

FIG. 16 is a flowchart illustrating a process of performing lighting compensation using flag information indicating whether lighting compensation is performed on a current block and an offset value of the current block according to an embodiment to which the present invention is applied.

In order to perform lighting compensation, the decoding unit extracts various information, for example, flag information and offset values of neighboring blocks of the current block, index information of corresponding reference blocks of the current block and its neighboring blocks, and the like from the video signal. Can be. In addition, the information of the current block may be obtained using the information. The decoder may obtain a difference between the offset value of the current block and the predictor and restore the offset value of the current block by using the obtained difference value and the predictor. In this case, when restoring the offset value of the current block, flag information IC_flag indicating whether illumination compensation of the current block is performed may be used.

The decoding unit obtains flag information indicating whether to perform illumination compensation of the current block from the video signal (S271). When illumination compensation is performed according to the flag information, the offset value of the current block indicating the difference between the average pixel value of the current block and the average pixel value of the reference block may be restored (S272). As such, the illumination compensation technique codes difference values of average pixel values of blocks belonging to different pictures. When a flag indicating whether the illumination compensation technique is applied to each block is used, when the corresponding block is a block belonging to a P slice, one flag information and one offset value may be encoded / decoded. However, if the corresponding block is a block belonging to a B slice, various methods may be possible. Hereinafter, the detailed description will be made with reference to FIGS. 17A to 17B.

17A to 17B illustrate an embodiment to which the present invention is applied to explain a method of performing illumination compensation using flag information and offset values for blocks belonging to a P slice and a B slice, respectively.

In FIG. 17A, "C" represents a current block C, "N" represents a block neighboring the current block C, "R" represents a block referred to by the current block C, and "S". Denotes a block referenced by a block N neighboring the current block C. "m _c " represents an average pixel value of the current block, and "m _r " represents an average pixel value of the block referenced by the current block. If the offset value of the current block (C) is called "IC_offset", then IC_offset = m _c- m _r becomes

Similarly, if the offset value of the neighboring block N is "IC_offset_pred", the encoding unit does not transmit the value as it is to restore "IC_offset", which is the offset value of the current block C, but instead offsets the offset value (IC_offset) of the current block. Only the difference value R _{IC_offset} between the offset value IC_offset_pred and the neighboring block may be transmitted. Wherein, R _{IC _ offset} is shown in Equation 20.

Various methods may be applied when generating the predicate of the current block from the flag information or the offset value of the neighboring block. For example, information of only one neighboring block may be used, or information of two or more neighboring blocks may be used. When using information of two or more neighboring blocks, an average value may be used or a median may be used. As such, when the current block is predictively coded using one reference block, lighting compensation may be performed using one offset value and one flag information.

However, if the corresponding block is a block belonging to a B slice, that is, the current block is predictively coded using two or more reference blocks, various methods may be possible.

For example, in FIG. 17B, "C" represents a current block C, "N" represents a block neighboring the current block C, and "R0" corresponds to reference picture 1 of List 0 referenced by the current block. For a reference block, it is assumed that "S0" represents a reference block in reference picture 1 of List 0 referenced by a neighboring block. In addition, it is assumed that "R1" represents a reference block in reference picture 3 of List 1 referred to by the current block, and "S0" represents a reference block in reference picture 3 of List 1 referred to by the neighboring block. In this case, since the flag information and the offset value of the current block exist for each reference block, two values exist. Accordingly, at least one of the flag information and the offset value may be used.

As a first example, the predictor of the current block may be obtained by combining information about two reference blocks through motion compensation. Here, whether to apply lighting compensation to the current block is indicated by one flag information, and when the flag information is true, one offset value may be obtained from the current block and the predictor to perform encoding / decoding.

As a second example, it may be determined whether to apply illumination compensation to two reference blocks in the process of performing motion compensation. Flag information is provided for each reference block, and one offset value obtained using the flag information may be encoded / decoded. In this case, two flag information may be used based on the reference block and one offset value may be used based on the current block.

As a third example, whether or not to apply lighting compensation to this block based on the current block may be indicated in one flag information. Each offset value may be encoded / decoded with respect to two reference blocks. In the encoding process, when illumination compensation is not applied to any one of the reference blocks, the corresponding offset value is set to zero. In this case, one flag information may be used based on the current block and two offset values may be used based on the reference block.

As a fourth example, each flag information and an offset value may be encoded / decoded for each reference block. In this case, both the flag information and the offset value may be used in each of two reference blocks.

In the case of the first to fourth examples, the offset value is not encoded as it is, and after predicting from the offset value of the neighboring block, only the error value can be encoded.

18 is a flowchart illustrating a method of performing lighting compensation when a current block is predictively coded using two reference blocks according to an embodiment to which the present invention is applied.

If the current block is a block belonging to a B slice, in order to perform lighting compensation, flag information and offset values of neighboring blocks of the current block, index information of corresponding reference blocks of the current block and its neighboring blocks, etc. are extracted from the video signal. In addition, the information of the current block may be obtained using the information. The offset value of the current block and the predictor may be obtained, and the offset value of the current block may be restored using the obtained difference value and the predictor. In this case, when restoring the offset value of the current block, flag information IC_flag indicating whether illumination compensation of the current block is performed may be used.

The decoding unit may obtain flag information indicating whether to perform illumination compensation of the current block from the video signal (S291). When illumination compensation is performed according to the flag information, the offset value of the current block indicating the difference between the average pixel value of the current block and the average pixel value of the reference block may be restored (S292).

However, when the current block is predictively coded using two reference blocks as described above, the decoder cannot directly know an offset value corresponding to each reference block. This is because, when the offset value of the current block is obtained, the pixel value obtained by averaging the two reference blocks is used. Therefore, in an embodiment of the present invention, an offset value corresponding to each reference block may be obtained to enable more accurate prediction. Therefore, when the current block is predictively coded using two reference blocks, an offset value corresponding to each reference block may be obtained using the offset value (S293). This is shown in Equation 22 below.

IC_offset = m _c -w ₁ * m _{r , 1} -w ₂ * m _{r , 2}

IC_offsetL0 = m _c -m _{r , 1} = IC_offset + (w ₁ -1) * m _{r , 1} + w ₂ * m _{r , 2}

IC_offsetL1 = m _c -m _{r , 2} = IC_offset + w ₁ * m _{r , 1} + (w ₂ -1) * m _{r , 2} .

Here, m _c represents the average pixel value of the current block, and m _{r , 1} , m _{r , 2} represent the average pixel value of the reference block, respectively. w ₁ and w ₂ represent weight coefficients in bi-predictive coding.

When the illumination compensation is performed in this manner, the accurate offset value corresponding to each reference block can be obtained and used separately, thereby enabling more accurate predictive coding. When restoring the offset value of the current block, the offset value may be obtained by adding the restored offset difference value and the predictor value. At this time, a predictor for each of List0 and List1 may be obtained, and the combination may be combined to obtain a predicator required when restoring the offset value of the current block.

19 is a flowchart illustrating a process of performing lighting compensation by using flag information indicating whether lighting compensation is performed on a current block according to an embodiment to which the present invention is applied.

Lighting compensation techniques are intended to compensate for lighting or color differences between cameras. This technique is further extended and can be applied between sequence images obtained from the same camera. This technique ensures that lighting or color differences do not significantly affect motion estimation. However, in actual encoding, flag information indicating whether lighting compensation is applied is used. The scope of illumination compensation may be a sequence, a view, a group of pictures (GOP), a picture, a slice, a macroblock, a sub-block, and the like. have.

If the lighting compensation technique is used in a small area range, it is possible to adjust the more local area, but instead, it takes a lot of bits for the flag information. Also, in many cases, lighting compensation techniques may not be needed. Therefore, by assigning a flag bit indicating whether the illumination compensation technique is used for each region range, it is possible to effectively use the illumination compensation technique. First, flag information for inducing lighting compensation for a predetermined layer of a video signal may be obtained (S301).

For example, flag information may be set for each region range as follows. "Seq_IC_flag" for sequence layer, "view_IC_flag" for view layer, "GOP_IC_flag" for GOP layer, "pic_IC_flag" for picture layer, "slice_IC_flag" for slice layer, "mb_IC_flag" for macroblock layer, block For the layer, flag bits may be allocated with "blk_IC_flag". This will be described in detail with reference to FIGS. 20A to 20C. In operation S302, a predetermined layer of the video signal on which illumination compensation is performed may be decoded according to the flag information.

20A to 20C are diagrams for describing a range of use of flag information for inducing lighting compensation of a current block as an embodiment to which the present invention is applied.

An application range of flag information for inducing lighting compensation may be divided by layers. For example, as illustrated in FIGS. 20A to 20B, "seq_IC_flag" 311 for the sequence layer, "view_IC_flag" 312 for the view layer, "GOP_IC_flag" 313 for the GOP layer, and "" for the picture layer are described. pic_IC_flag "314," slice_IC_flag "315 for the slice layer," mb_IC_flag "316 for the macroblock layer, and" blk_IC_flag "317 for the block layer.

Here, each flag is 1 bit information, and at least one flag may exist. In addition, flags such as sequence / view / picture / slice range may be located in the corresponding parameter set or header, or in other parameter sets. For example, "seq_IC_flag" 311 may be located in a sequence parameter set, "view_IC_flag" 312 may be located in a view parameter set, and "pic_IC_flag" ( 314 may be located in a picture parameter set. Also, "slice_IC_flag" 315 may be located in the slice header.

When two or more flag information exist, whether to perform lighting compensation of a lower layer among two or more predetermined layers may be controlled by performing lighting compensation of a higher layer. That is, when each flag bit value is set to 1, it indicates that the lighting compensation technique may be applied in the lower range.

For example, if pic_IC_flag = 1, the slice_IC_flag value can be set to 1 or 0 for each slice within the corresponding picture range, or the mb_IC_flag value can be set to 1 or 0 for each macroblock, or each A blk_IC_flag value may be set to 1 or 0 for a block. If there is a viewpoint parameter set, if seq_IC_flag = 1, the view_IC_flag value can be set to 1 or 0 for each viewpoint. However, when view_IC_flag = 1, as shown in FIG. 20A, a flag bit value for each GOP, picture, slice, macroblock, or block belonging to the corresponding view may be set to 1 or 0, or may not be the case. . In this case, otherwise, the flag for each GOP, picture, slice, macroblock, or block is not controlled by the flag information on the viewpoint.

If the flag bit value of the upper range is 0, the flag bit values of the lower range are automatically 0. For example, when seq_IC_flag = 0, this means that the lighting compensation technique is not applied to the sequence, and therefore, view_IC_flag, GOP_IC_flag, pic_IC_flag, slice_IC_flag, mb_IC_flag, and blk_IC_flag are all set to zero. In addition, only one mb_IC_flag and blk_IC_flag may be used according to a specific implementation method of the lighting compensation technique. In addition, the flag information view_IC_flag about the viewpoint may be applied when a viewpoint parameter set is newly applied in multiview video coding. The offset value of the current block may be additionally encoded / decoded according to the flag bit value in the macroblock or subblock which is the lowest unit.

Also, as shown in FIG. 20C, an embodiment in which a flag for applying an IC technology is applied only to a slice and a macroblock layer may be considered. For example, when slice_IC_flag is 0, it may mean that the IC is not applied to the slice. If slice_IC_flag is 1, it may mean that the IC is applied to the corresponding slice. In this case, if mb_IC_flag is 1, IC_offset is restored in the corresponding macroblock. If mb_IC_flag is 0, this may mean that the IC is not applied to the corresponding macroblock.

According to another embodiment of the present invention, when flag information of a layer higher than a macroblock layer is true, an offset value of a current block indicating a difference between an average pixel value of a current block and an average pixel value of a reference block may be obtained. In this case, flag information on the macroblock layer or the block layer may not be used. When applying the lighting compensation technology, whether or not to apply the lighting compensation technology to each block may be indicated through the flag information. However, it may be sufficient to represent only that value, such as a motion vector. This may be operated together with the application range of the lighting compensation technology, and may indicate whether lighting compensation is applied in the lower range through flag information on the upper range (sequence, view, GOP, picture). In the lowest block, the macroblock layer or block layer, the offset value may be sufficient without using the flag bit. It can basically perform prediction and encoding in a manner similar to a motion vector. For example, when performing predictive coding on the current block, the offset value of the neighboring block is assigned to the offset value of the current block. In the case of bidirectional predictive coding, an offset value for each reference block is obtained through calculation with reference blocks found in List 0 and List 1. Therefore, when encoding offset values of the current block, only offset errors are encoded / decoded without directly encoding the offset value for each reference block by using offset values of neighboring blocks. The prediction method of the offset value may use the above offset prediction method or a method of obtaining a median used in motion vector prediction. Even in the direct mode of the bidirectional prediction, offset values may be obtained through the information already given in the same manner as the motion vector without encoding / decoding the additional information.

In addition, as another embodiment of the present invention, when using a decoder other than the MVC decoder (for example, H.264), since the view image compatible with the existing decoder should be able to be decoded by the existing decoder Must be set to "view_IC_flag = false or 0" Here we need to explain the concept of base views. We may need at least one view sequence to be compatible with the H.264 / AVC decoder. Therefore, it is necessary to define time points that can be independently decoded, which is called a reference time point. This reference time point becomes a reference of encoding among multiviews, which corresponds to a reference view. In MVC, an image corresponding to a reference time point is encoded by a conventional general video encoding method (MPEG-2, MPEG-4, H.263, H.264, etc.) to form an independent bitstream. The image corresponding to the reference time point may or may not be compatible with H.264 / AVC. However, an image of a viewpoint compatible with H.264 / AVC is always a reference viewpoint.

FIG. 21 is a flowchart illustrating a process of obtaining a motion vector in consideration of an offset value of a current block according to an embodiment to which the present invention is applied.

According to FIG. 21, the system may obtain an offset value of a current block from a video signal (S321). The system may search for a reference block that optimally matches the current block using the offset value (S322). The system may obtain and encode a motion vector from the searched reference block (S323). Here, there are considerations when performing motion estimation when applying illumination compensation. For example, in the case of comparing the similarity of two blocks by canceling the average pixel value between blocks, when the motion estimation is performed, the similarity degree is determined by subtracting the average pixel value of each block from the pixel values of each block. Can be calculated In this case, since the offset value between the two blocks is separately coded, such a cost should be reflected in the motion estimation process. The existing cost calculation is shown in Equation 23 below.

In the case of applying illumination compensation, sum of absolute differences (SAD) may be calculated as in Equation 24 below.

Here, I _c and I _r represent pixel values of the current block and the reference block, respectively. M _c and M _r represent average pixel values of the current block and the reference block, respectively. The cost of the offset value can be reflected in the SAD calculation.

SAD _IC = α | offset-offset_pred | + ∑ ｜ (I _c (m, n) -M _c )-(I _r (m, n)-M _r ) ｜

는 가중치가 되며, 1일 때, 오프셋 값의 절대치가 반영된다. 또한, 조명 보상 비용을 반영하는 다른 방법으로서, 오프셋 값을 부호화하는 데에 쓰이는 비트수를 추정해서 반영하는 방법이 있다. here,

Is a weight, and when 1, the absolute value of the offset value is reflected. As another method of reflecting the illumination compensation cost, there is a method of estimating and reflecting the number of bits used to encode the offset value.

This estimates and reflects the offset encoding bits as shown in Equation 27 below. Here, the coded bits can be estimated to be proportional to the magnitude of the offset difference value.

In this case, the new cost can be obtained using Equation 28 below.

Cost =

Cost =

FIG. 1 is a schematic block diagram of a decoding apparatus of a multiview video system for decoding a video signal including a multiview video image according to an embodiment to which the present invention is applied.

2 is a flowchart illustrating a video image encoding method according to an embodiment to which the present invention is applied.

FIG. 3 is a block diagram illustrating a process of deriving a predicted average pixel value of a current block from a reference image block at another point in time according to an embodiment of the present invention.

FIG. 4 is a block diagram for generalizing and describing a process of deriving an illumination average prediction value of a current block from a reference image block at another time point as an embodiment to which the present invention is applied.

FIG. 5 is a diagram illustrating a 16x16 macroblock illustrating an example of using previously decoded values of pixels on the left and top of a block in obtaining an illumination mean value and deriving an illumination mean prediction value according to an embodiment of the present invention. will be.

6A and 6B illustrate exemplary embodiments to which the present invention may be applied. In the case where a block is divided, in the obtaining of an illumination average value of the divided block and deriving an illumination average prediction value, all of the pixels surrounding the divided block are used. 16x8 macroblocks for explaining the case of using only pixels surrounding the divided block, respectively.

7A and 7B illustrate an embodiment to which the present invention is applied. In the case where a block is divided, in the obtaining of an illumination average value of the divided block and deriving an illumination average prediction value, the pixels surrounding the divided block are used. 8x16 macroblocks for explaining the case of using only pixels surrounding the divided block are illustrated.

8A and 8B illustrate an embodiment to which the present invention is applied. In the case where a block is divided, in the obtaining of an illumination average value of the divided block and deriving an illumination average prediction value, the pixels surrounding all of the divided blocks are used. 8x8 macroblocks for explaining the case of using only pixels surrounding the divided block are shown.

9 is a diagram for describing a process of obtaining an offset value of a current block according to an embodiment to which the present invention is applied.

FIG. 10 is a flowchart illustrating a process of performing lighting compensation on a current block according to an embodiment to which the present invention is applied.

FIG. 11 is a flowchart illustrating a method of acquiring a predictor based on whether a reference number is identical between a current block and a neighboring block according to an embodiment to which the present invention is applied.

12 is a flowchart illustrating a method of performing lighting compensation based on a prediction type of a current block according to an embodiment to which the present invention is applied.

FIG. 13 is a flowchart illustrating a method of performing lighting compensation using flag information indicating whether to perform lighting compensation of a corresponding block in an embodiment to which the present invention is applied.

FIG. 14 is a flowchart illustrating a method of predicting flag information of a current block based on whether a reference number of a current block and a neighboring block is the same as an embodiment to which the present invention is applied.

15 is a flowchart illustrating a method of performing lighting compensation when a current block is predictively coded using two reference blocks according to an embodiment to which the present invention is applied.

FIG. 16 is a flowchart illustrating a process of performing lighting compensation using flag information indicating whether lighting compensation is performed on a current block and an offset value of the current block according to an embodiment to which the present invention is applied.

17A to 17B illustrate an embodiment to which the present invention is applied to illustrate a method of performing lighting compensation using flag information and offset values for blocks belonging to a P slice and a B slice, respectively.

18 is a flowchart illustrating a method of performing lighting compensation when a current block is predictively coded using two reference blocks according to an embodiment to which the present invention is applied.

19 is a flowchart illustrating a process of performing lighting compensation by using flag information indicating whether lighting compensation is performed on a current block according to an embodiment to which the present invention is applied.

20A to 20C are diagrams for describing a range of use of flag information for inducing lighting compensation of a current block as an embodiment to which the present invention is applied.

FIG. 21 is a flowchart illustrating a process of obtaining a motion vector in consideration of an offset value of a current block according to an embodiment to which the present invention is applied.

The present invention is directed to a method comprising: receiving a bitstream in which a plurality of viewpoints of a multiview video signal are encoded, extracting flag information related to a portion of the multiview video signal from the bitstream, and lighting compensation according to the flag information For this performed portion, extracting a value associated with a segment in the portion from the bitstream and determining whether illumination compensation of the segment is performed from the extracted value, wherein each of the plurality of views The view includes a plurality of pictures divided into a plurality of segments (eg, an image block segment such as a single block or macroblock, or a segment such as a slice of an image), wherein the flag information is part of the multi-view video signal. Whether lighting compensation is performed on segments within Provides a multi-view video signal decoding method characterized in that the indicating.

In addition, the present invention may include one or more of the following features.

In the present invention, the segments are composed of image blocks.

The present invention obtains an illumination compensation prediction value of the first block, using an illumination compensation offset value of at least one neighboring block adjacent to the first block, for a first block associated with a value indicating that illumination compensation is performed. Characterized in that it further comprises the step.

In the present invention, the illumination compensation offset value of the neighboring block is obtained by adding the difference value and the illumination compensation prediction value of the neighboring block.

In the present invention, the obtaining of the lighting compensation prediction value of the first block by using the lighting compensation offset value of at least one neighboring block adjacent to the first block may be performed according to the predetermined order among the neighboring blocks. And selecting a neighboring block.

In the present invention, the step of selecting the at least one neighboring block according to the predetermined order includes one or more neighboring blocks in one or more vertical or horizontal directions, and then one or more neighboring blocks in one or more diagonal directions. Determining whether the conditions are satisfied for the neighboring block.

In the present invention, the flag information, characterized in that the illumination compensation for one or more of the sequence, the viewpoint, the group of pictures, the picture and the slice containing the block.

In the present invention, the flag information, characterized in that to enable illumination compensation for the slice containing the block.

In the present invention, the extracted value is characterized by consisting of flag information of the macroblock including the block or flag information of the block.

In the present invention, the extracted value is characterized by consisting of flag information of a macroblock including the block.

In addition, the present invention provides a method of receiving a bitstream in which a plurality of viewpoints of a multiview video signal are encoded, and illuminating the first segment using an illumination compensation offset value of at least one neighboring segment adjacent to the first segment. Obtaining a compensation prediction value, wherein each viewpoint of the plurality of viewpoints includes a plurality of pictures divided into a plurality of segments, and obtaining the prediction value is performed according to a predetermined order among the neighboring segments. It provides a multi-view video signal decoding method comprising the step of selecting at least one neighboring segment.

In addition, the present invention may include one or more of the following features.

In the present invention, the first segment and the at least one neighboring segment are formed of image blocks.

In the present invention, the illumination compensation offset value of the neighboring block is obtained by adding the difference value and the illumination compensation prediction value of the neighboring block.

In the present invention, the step of selecting the at least one neighboring block according to the predetermined order includes one or more neighboring blocks in one or more vertical or horizontal directions, and then one or more neighboring blocks in one or more diagonal directions. Determining whether the conditions are satisfied for the neighboring block.

In the present invention, the step of selecting the at least one neighboring block according to the predetermined order, in the order of the left neighboring block, the upper neighboring block, the upper right neighboring block, the upper left neighboring block, one or more conditions for the neighboring block Determining whether it is satisfied.

In the present invention, determining whether one or more conditions are satisfied for a neighboring block includes extracting from the bitstream a value related to the neighboring block indicating whether illumination compensation of the neighboring block is performed. It is characterized by.

In the present invention, the extracted value is characterized by consisting of flag information of the macroblock including the block or flag information of the block.

In the present invention, obtaining the illumination compensation prediction value includes determining whether to use an illumination compensation offset value of one neighboring block or a plurality of illumination compensation offset values of each neighboring block. It is done.

In the present invention, when the plurality of illumination compensation offset values are used, the method may further include acquiring an illumination compensation prediction value of the first block by using the plurality of illumination compensation offset values.

In the present invention, the step of using the plurality of illumination compensation offset values comprises the step of taking an average or median of the plurality of illumination compensation offset values.

The present invention also provides a method of receiving a bitstream encoded with multiple views of a multiview video signal, obtaining an illumination compensation offset value of a first segment with respect to a reference picture, and performing illumination compensation of the first segment. Decoding the bitstream using a plurality of pictures, wherein each view of the plurality of views includes a plurality of pictures divided into a plurality of segments, and the offset value is at least one determined based on characteristics associated with a neighboring segment. Predicted using an illumination compensation offset value of a neighboring segment of, wherein the decoding step comprises: predicting pixels of the first segment obtained from the reference picture, difference values of the pixels of the first segment, and corresponding illumination compensation; A multiview video signal decoding room comprising adding an offset value It provides.

In addition, the present invention may include one or more of the following features.

In the present invention, the first segment and the at least one neighboring segment are formed of image blocks.

In the present invention, the illumination compensation offset value of the neighboring block is obtained by adding the difference value and the illumination compensation prediction value of the neighboring block.

The present invention is directed to at least one neighboring block based on whether one or more conditions are satisfied for the neighboring block, in the order of one or more vertical or horizontal neighboring blocks, followed by one or more diagonal neighboring blocks. It characterized in that it further comprises the step of selecting.

In the present invention, the step of selecting the at least one neighboring block according to the predetermined order, in the order of the left neighboring block, the upper neighboring block, the upper right neighboring block, the upper left neighboring block, one or more conditions for the neighboring block Determining whether it is satisfied.

In the present invention, determining whether one or more conditions are satisfied for a neighboring block includes extracting from the bitstream a value related to the neighboring block indicating whether illumination compensation of the neighboring block is performed. It is characterized by.

In the present invention, the extracted value is characterized by consisting of flag information of the macroblock including the block or flag information of the block.

In the present invention, selecting at least one neighboring block includes determining whether to use an illumination compensation offset value of one neighboring block or a plurality of illumination compensation offset values of each neighboring block. It features.

In the present invention, if a plurality of lighting compensation offset values are used, the method further comprises obtaining the lighting compensation prediction value of the first block by using the plurality of lighting compensation offset values.

In the present invention, the step of using the plurality of illumination compensation offset values is characterized in that it comprises taking the average value or the median of the plurality of illumination compensation offset values.

In addition, the present invention provides a method comprising: receiving a bitstream in which a plurality of viewpoints of a multiview video signal are encoded, obtaining an illumination compensation prediction value of a first segment related to a reference picture, and an illumination compensation prediction value of the first segment; And determining the illumination compensation offset value of the first segment by adding the difference value and the difference value and decoding the bitstream using the illumination compensation of the first segment, wherein each of the plurality of viewpoints is multiple. And a plurality of pictures divided into four segments, wherein the decoding comprises: predicting pixels of the first segment obtained from the reference picture, difference values of pixels of the first segment, and corresponding illumination compensation offsets; It provides a multi-view video signal decoding method comprising the step of adding a value.

In addition, the present invention may include one or more of the following features.

In the present invention, the segments are composed of image blocks.

In the present invention, the use of the illumination compensation of the first segment may include obtaining an illumination compensation offset value of the neighboring block by adding a difference value and an illumination compensation prediction value of the neighboring block.

In the present invention, at least one neighbor based on whether one or more conditions are satisfied for the neighboring block, in the order of one or more vertical or horizontal neighboring blocks, followed by one or more diagonal neighboring blocks. The method may further include selecting a block.

In the present invention, the step of selecting the at least one neighboring block, in the order of the left neighboring block, the upper neighboring block, the upper right neighboring block, the upper left neighboring block, determines whether one or more conditions are satisfied for the neighboring block. Characterized in that it comprises a step.

In the present invention, determining whether one or more conditions are satisfied for a neighboring block includes extracting from the bitstream a value related to the neighboring block indicating whether illumination compensation of the neighboring block is performed. It is characterized by.

In the present invention, the extracted value is characterized by consisting of flag information of the macroblock including the block or flag information of the block.

In the present invention, selecting at least one neighboring block includes determining whether to use an illumination compensation offset value of one neighboring block or a plurality of illumination compensation offset values of each neighboring block. It features.

In the present invention, when a plurality of lighting compensation offset values are used, the method may further include obtaining a prediction value for performing lighting compensation of the first block by using the plurality of lighting compensation offset values.

In the present invention, the step of using the plurality of illumination compensation offset values comprises the step of taking an average or median of the plurality of illumination compensation offset values.

The present invention also relates to receiving a bitstream in which multiple views of a multiview video signal are encoded, and whether or not a reference picture associated with a first segment is the same as a reference picture associated with a neighboring segment. Obtaining an illumination compensation prediction value of a first segment with respect to a reference picture using an illumination compensation offset value of at least one neighboring segment adjacent to, wherein each viewpoint of the plurality of viewpoints is divided into a plurality of segments; It provides a multi-view video signal decoding method comprising a number of pictures.

In addition, the present invention may include one or more of the following features.

In the present invention, the segments are composed of image blocks.

In the present invention, the use of the illumination compensation of the first segment may include obtaining an illumination compensation offset value of the neighboring block by adding a difference value and an illumination compensation prediction value of the neighboring block.

The present invention is directed to at least one neighboring block based on whether one or more conditions are satisfied for the neighboring block, in the order of one or more vertical or horizontal neighboring blocks, followed by one or more diagonal neighboring blocks. It characterized in that it further comprises the step of selecting.

In the present invention, the step of selecting the at least one neighboring block, in the order of the left neighboring block, the upper neighboring block, the upper right neighboring block, the upper left neighboring block, determines whether one or more conditions are satisfied for the neighboring block. Characterized in that it comprises a step.

In the present invention, determining whether one or more conditions are satisfied for a neighboring block includes extracting from the bitstream a value related to the neighboring block indicating whether illumination compensation of the neighboring block is performed. It is characterized by.

In the present invention, the extracted value is characterized by consisting of flag information of the macroblock including the block or flag information of the block.

In the present invention, selecting at least one neighboring block includes determining whether to use an illumination compensation offset value of one neighboring block or a plurality of illumination compensation offset values of each neighboring block. It features.

In the present invention, when a plurality of lighting compensation offset values are used, the method may further include obtaining a prediction value for performing lighting compensation of the first block by using the plurality of lighting compensation offset values.

In the present invention, the step of using the plurality of illumination compensation offset values comprises the step of taking an average or median of the plurality of illumination compensation offset values.

The present invention also provides, for each decoding method, a video signal encoding method characterized by generating a bitstream in which the video signal can be decoded by the respective decoding method.

For example, the present invention provides a method of generating a bitstream in which a plurality of viewpoints of a multiview video signal are encoded, providing flag information related to a portion of the multiview video signal, and lighting compensation according to the flag information. For this portion to be performed, providing a value associated with a segment in the portion, and determining whether illumination compensation of the segment is performed from the provided value, wherein each of the plurality of viewpoints is a plurality of segments. And a plurality of pictures, each of which is divided into a plurality of pictures, wherein the flag information indicates whether lighting compensation is performed on segments within a portion of the multi-view video signal.

The present invention also provides a method of generating a bitstream in which a plurality of viewpoints of a multiview video signal are encoded, and using illumination compensation offset values of at least one neighboring segment adjacent to the first segment, illuminating the first segment. Providing a compensated prediction value, wherein each viewpoint of the plurality of viewpoints includes a plurality of pictures divided into a plurality of segments, and providing the prediction value comprises: determining the prediction value according to a predetermined order of the neighboring segments. Selecting at least one neighboring segment provides a video signal encoding method.

The present invention also provides a method of generating a bitstream in which a plurality of viewpoints of a multiview video signal are encoded, providing an illumination compensation offset value of a first segment with respect to a reference picture, and lighting compensation of the first segment. Providing information for each of the plurality of viewpoints, wherein each viewpoint of the plurality of viewpoints comprises a plurality of pictures divided into a plurality of segments, wherein the offset value is at least one neighbor determined based on characteristics associated with the neighboring segment. Wherein the information providing step comprises: predicting the pixels of the first segment obtained from the reference picture, a difference value of the pixels of the first segment, and a corresponding illumination compensation offset; A video signal encoding method is provided based on adding a value.

The present invention also provides a method of generating a bitstream in which a plurality of viewpoints of a multiview video signal are encoded, providing an illumination compensation prediction value of a first segment related to a reference picture, and an illumination compensation prediction value of the first segment. And providing an illumination compensation offset value of the first segment by adding a difference value and a difference value, and providing information for illumination compensation of the first segment, wherein each viewpoint of the plurality of viewpoints is a plurality of segments. And the information providing step comprises: predicting pixels of the first segment obtained from the reference picture, difference values of the pixels of the first segment, and corresponding illumination compensation offset values; It provides a video signal encoding method comprising the step of adding.

The present invention also provides a method of generating a bitstream in which a plurality of viewpoints of a multiview video signal are encoded and whether the reference picture associated with the first segment is the same as the reference picture associated with the neighboring segment. Providing an illumination compensation estimate of the first segment with respect to the reference picture using an illumination compensation offset value of at least one neighboring segment adjacent to, wherein each of the plurality of viewpoints is divided into a plurality of segments. It provides a video signal encoding method comprising a number of pictures.

In addition, for each decoding method, the computer program stored in the computer readable medium instructs the computer to perform the respective decoding method.

In addition, for each decoding method, the image data included in the apparatus-readable information carrier can be decoded into a video signal by the respective decoding method.

In addition, for each decoding method, the decoder comprises means for performing said respective decoding method.

In addition, for each decoding method, the encoder comprises means for generating a bitstream to be decoded into a video signal by said respective decoding method.

In addition, the present invention, (a) obtaining an average of the illumination (illumination) for at least one block of the reference image blocks at a different time point than the blocks adjacent to the current block, and (b) at least one obtained Deriving an illumination average prediction value of the current block from the illumination average value for the block; and (c) obtaining an error value between the illumination average prediction value of the current block and the illumination average value of the current block. Provide a method.

In addition, the present invention, (l) obtaining an error value for restoring the illumination average value of the current block from the video signal, and (m) calculating the illumination average prediction value of the current block based on the reference image blocks at different time points And (n) restoring an illumination average value of the current block from the illumination average prediction value and the error value.

In addition, the present invention, the illumination average value acquisition unit for obtaining the illumination average value of the reference image blocks at a different time point than the blocks adjacent to the current block, and the illumination average value prediction unit for inducing the illumination average prediction value of the current block from the obtained average value And an error value encoder which obtains an error value between the illumination average prediction value and the illumination average value of the current block.

The present invention also provides an error value decoder for obtaining an error value from a received bitstream, an illumination average value predictor for inducing an illumination average predicted value of a current block based on a reference image block at a different view, and the error value and The present invention provides a video image decoding apparatus comprising an illumination compensator for restoring an illumination average value of a current block from an illumination average prediction value.

The present invention also provides a method for acquiring a predictor for lighting compensation of the current block by using an offset value of a neighboring block adjacent to a current block, and using the predictor to offset the current block. And reconstructing the signal, wherein the predictor is determined according to whether a reference number of a current block and a reference number of the neighboring blocks are the same.

The present invention also provides a method of restoring an offset value of a current block indicating a difference between an illumination average value of a current block and an illumination average value of a reference block, and when the current block is predictively coded using two or more reference blocks, the offset And obtaining offset values for each reference block of the current block by using a value.

The present invention also provides a method of acquiring flag information indicating whether or not to perform illumination compensation of a current block, and when illumination compensation is performed according to the flag information, And restoring the offset value of the current block indicating the difference.

The present invention also includes obtaining flag information for inducing illumination compensation for a certain layer of a video signal, and decoding a layer of the video signal for which illumination compensation has been performed according to the flag information. The predetermined layer of the video signal may include a sequence layer, a view layer, a group of picture layer, a picture layer, a slice layer, a macroblock layer, or A video signal decoding method is provided which is any one of a block layer.

The present invention also provides a method of obtaining an offset value of a current block representing a difference between an average pixel value of a current block and an average pixel value of a reference block, and searching for a reference block that is optimally matched to the current block by using the offset value. And acquiring and encoding a motion vector from the searched reference block.

The above embodiments may have one or more of the following advantages.

The encoding / decoding method or apparatus for the video image predicts an average value of a block to be encoded from a neighboring block and transmits only the difference value, thereby minimizing information to be transmitted for illumination compensation. can do.

In addition, the present invention can improve coding efficiency by performing efficient illumination compensation in a multiview video image requiring a large amount of data. In addition, it is possible to construct an efficient encoding and decoding system by using correlations between blocks or viewpoints.

Since the viewpoint images of the multiview video data are generated by different cameras, illumination differences occur due to internal and external factors of the cameras. Therefore, in the present invention, by predicting the offset value of the current block using the information of the neighboring block and transmitting only the difference value, it is possible to minimize the information to be transmitted for lighting compensation. In addition, when predicting the offset value of the current block, it is possible to perform a more accurate prediction by checking whether the reference number of the current block and the neighboring block is the same.

In addition, the present invention can also minimize the information to be transmitted by predicting the flag information indicating whether the illumination compensation of the current block is performed only by transmitting the difference value. Similarly, by checking whether the current block and the neighboring block have the same reference number, more accurate prediction can be performed. Through the present invention as described above, efficient coding is enabled by using correlations between blocks or viewpoints.

Since the viewpoint images of the multiview video data are generated by different cameras, illumination differences occur due to internal and external factors of the cameras. Therefore, in the present invention, by predicting the offset value of the current block using the information of the neighboring block and transmitting only the difference value, it is possible to minimize the information to be transmitted for lighting compensation. According to the present invention, it is possible to minimize the information to be transmitted by predicting flag information indicating whether the current block performs lighting compensation and transmitting only the difference value.

In addition, when prediction is coded using two or more reference blocks, more efficient coding may be enabled by applying offset values and flag information in at least one or more methods, respectively. In addition, by assigning a flag bit indicating whether to use the illumination compensation technique for each region range of the video signal, it is possible to effectively use the illumination compensation technique. In addition, the present invention, it is possible to perform more accurate predictive coding by calculating the cost by reflecting the illumination difference in the process of performing the motion estimation.

Other features and advantages will appear more clearly in the description and claims.

It will be understood by those skilled in the art that the present invention may be embodied in other specific forms without departing from the spirit or essential characteristics thereof.

Therefore, it is to be understood that the embodiments described above are exemplary in all respects and not restrictive. The scope of the present invention is defined by the appended claims rather than the detailed description and all changes or modifications derived from the meaning and scope of the claims and their equivalents are to be construed as being included within the scope of the present invention do.

Claims (50)

Receiving a bitstream in which a plurality of viewpoints of a multiview video signal are encoded; Extracting first flag information for a current slice of the multiview video signal from the bitstream; Extracting second flag information for a current block in the current slice from the bitstream when lighting compensation is performed on the current slice according to the first flag information; Deriving an illumination compensation prediction value of the current block using an illumination compensation offset value of at least one neighboring block adjacent to the current block when illumination compensation is performed on the current block according to the second flag information; Extracting an illumination compensation difference value of the current block from the bitstream; And Obtaining an illumination compensation offset value of the current block by adding an illumination compensation prediction value of the current block and an illumination compensation difference value of the current block, The first flag information indicates whether lighting compensation is performed on the current slice, And the second flag information indicates whether lighting compensation is performed on the current block. The method of claim 1, Predicting a pixel value of the current block using an illumination compensation offset value of the current block; And And decoding the current block using the predicted pixel value of the current block. The method of claim 1, The illumination compensation prediction value of the current block is derived based on whether the reference number of the current block and the reference number of the neighboring block is the same. The method of claim 1, The illumination compensation offset value of the neighboring block is obtained by adding the illumination compensation prediction value of the neighboring block and the illumination compensation difference value of the neighboring block. The method of claim 3, And wherein the neighboring block is selected according to a predetermined order among neighboring blocks adjacent to the current block. 6. The method of claim 5, The neighboring block is a multi-view video signal decoding method characterized in that the block is located in the order of the upper, left, upper right and the upper left of the current block. The method of claim 1, wherein the deriving an illumination compensation prediction value of the current block comprises: And determining whether to use the illumination compensation offset value of one neighboring block or the illumination compensation offset values of the plurality of neighboring blocks. 8. The method of claim 7, wherein when using the illumination compensation offset values of the plurality of neighboring blocks, the illumination compensation prediction value of the current block is derived from an average value or a median value of the illumination compensation offset values of the plurality of neighboring blocks. Multi-view video signal decoding method. delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete

Images