KR20190112249A - Method and apparatus for image encoding/decoding - Google Patents

Abstract

A prediction method in a screen according to the present invention may identify a reference pixel area designated for obtaining correlation information, determine reference pixel processing setting based on the availability determination of the reference pixel area, and perform prediction in a screen according to the determined reference pixel processing.

Description

Image Encoding / Decoding Method and Apparatus {METHOD AND APPARATUS FOR IMAGE ENCODING / DECODING}

The present invention relates to a method and apparatus for image encoding / decoding.

Recently, the demand for high resolution and high quality images such as high definition (HD) and ultra high definition (UHD) images is increasing in various applications, and high efficiency image compression technologies are being discussed.

An inter-screen prediction technique for predicting pixel values included in the current picture from a picture before or after the current picture using an image compression technology, an intra-prediction technology for predicting pixel values included in the current picture using pixel information in the current picture, Various techniques exist, such as an entropy encoding technique for allocating a short code to a high frequency of appearance and a long code to a low frequency of appearance, and the image data can be effectively compressed and transmitted or stored.

An object of the present invention for solving the above problems is to provide an image encoding / decoding method and apparatus for performing intra prediction based on the availability of a reference pixel.

According to an aspect of the present invention, there is provided a method of performing intra prediction according to an embodiment of the present invention. The method may include: identifying a reference pixel region designated for obtaining correlation information, and setting reference pixel processing based on a determination of the availability of the reference pixel region. The method may include determining an image, and performing an intra prediction based on the determined reference pixel processing setting.

In the case of using the method of performing intra prediction based on the availability of the reference pixel of the present invention as described above, encoding performance may be improved.

1 is a conceptual diagram of an image encoding and decoding system according to an embodiment of the present invention.
2 is a block diagram of a video encoding apparatus according to an embodiment of the present invention.
3 is a block diagram of an image decoding apparatus according to an embodiment of the present invention.
4 is an exemplary diagram illustrating an intra prediction mode according to an embodiment of the present invention.
5 is a conceptual diagram illustrating in-screen prediction of a directional mode and a non-directional mode according to an embodiment of the present invention.
6 is a conceptual diagram illustrating an intra prediction according to a color copy mode according to an embodiment of the present invention.
FIG. 7 is an exemplary diagram illustrating a corresponding block of each color space and an area adjacent thereto in relation to a color copy mode according to an embodiment of the present invention.
8 is an exemplary diagram for setting a region for obtaining correlation information of a color copy mode according to an embodiment of the present invention.
9 is an exemplary diagram for describing a configuration of a reference pixel used for intra prediction according to an embodiment of the present invention.
10 is a conceptual diagram illustrating a block adjacent to a target block of intra prediction according to an embodiment of the present invention.
11 is an exemplary diagram for describing a possibility of using a reference pixel in a color copy mode according to an embodiment of the present invention.
12 is an exemplary diagram for describing a possibility of using a reference pixel in a color copy mode according to an embodiment of the present invention.
13 is an exemplary diagram for describing a possibility of using a reference pixel in a color copy mode according to an embodiment of the present invention.
14 is a flowchart illustrating an intra prediction method of a color copy mode according to an embodiment of the present invention.
15 is an exemplary diagram for describing prediction in color copy mode according to an embodiment of the present invention.
16 is an exemplary diagram for describing prediction in color copy mode according to an embodiment of the present invention.
17 is an exemplary diagram for explaining prediction in color copy mode according to an embodiment of the present invention.
18 is a flowchart of a process of performing correction in a color copy mode according to an embodiment of the present invention.
19 is an exemplary diagram for describing a type of filter applied to a correction target pixel according to an exemplary embodiment of the present invention.
20 is an exemplary diagram for describing a type of filter applied to a correction target pixel according to an exemplary embodiment of the present invention.

As the present invention allows for various changes and numerous embodiments, particular embodiments will be illustrated in the drawings and described in detail in the written description. However, this is not intended to limit the present invention to specific embodiments, it should be understood to include all modifications, equivalents, and substitutes included in the spirit and scope of the present invention.

Terms such as first, second, A, and B may be used to describe various components, but the components should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, the first component may be referred to as the second component, and similarly, the second component may also be referred to as the first component. The term and / or includes a combination of a plurality of related items or any item of a plurality of related items.

When a component is referred to as being "connected" or "connected" to another component, it may be directly connected to or connected to that other component, but it may be understood that other components may be present in between. Should be. On the other hand, when a component is said to be "directly connected" or "directly connected" to another component, it should be understood that there is no other component in between.

The terminology used herein is for the purpose of describing particular example embodiments only and is not intended to be limiting of the present invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In the present invention, the terms "comprise" or "have" are intended to indicate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, and one or more other features. It is to be understood that the present invention does not exclude the possibility of the presence or the addition of numbers, steps, operations, components, components, or a combination thereof.

Unless defined otherwise, all terms used herein, including technical or scientific terms, mean the same as generally understood by one of ordinary skill in the art. Terms such as those defined in the commonly used dictionaries should be construed in accordance with their meaning in the context of the related art and should not be construed in an ideal or excessively formal sense unless explicitly defined in the present invention.

Typically, one or more color spaces may be configured according to a color format of an image. It may consist of one or more pictures with a certain size or one or more pictures with other sizes, depending on the color format. For example, in the YCbCr color configuration, color formats such as 4: 4: 4, 4: 2: 2, 4: 2: 0, and Monochrome (only Y) may be supported. For example, in the case of YCbCr 4: 2: 0, one luminance component (Y in this example) and two chrominance components (Cb / Cr in this example) may be configured, wherein the chrominance component and the luminance component are configured. The ratio may have a length of 1: 2. For example, 4: 4: 4 may have the same ratio of width and length. When the picture is composed of one or more color spaces as described above, the picture may be divided into each color space.

Images can be classified into I, P, B, etc. according to the image type (eg, picture type, slice type, tile type, etc.). The I image type is an image which is encoded / decoded by itself without using a reference picture. The P image type may mean an image that is encoded / decoded using a reference picture but allows only omnidirectional prediction, and the B image type may be encoded / decoded by using a reference picture. Although it may mean an image that allows direction prediction, a part of the type may be combined (P and B are combined) or another type of image type may be supported according to the encoding / decoding setting.

1 is a conceptual diagram of an image encoding and decoding system according to an embodiment of the present invention.

Referring to FIG. 1, the video encoding apparatus 105 and the decoding apparatus 100 may include a personal computer (PC), a notebook computer, a personal digital assistant (PDA), and a portable multimedia player (PMP). Player), PlayStation Portable (PSP: PlayStation Portable), wireless communication terminal (Wireless Communication Terminal), smart phone (Smart Phone) or a user terminal such as a TV, or a server terminal such as an application server and a service server, etc. A communication device such as a communication modem for communicating with a wired / wireless communication network, a memory (memory, 120, 125) or a program for storing various programs and data for inter or intra prediction for encoding or decoding an image and executing the operation And various devices including processors (processors 110 and 115) for controlling.

In addition, the image encoded in the bitstream by the image encoding apparatus 105 is real-time or non-real-time via the wired or wireless communication network (Network), such as the Internet, local area wireless communication network, wireless LAN network, WiBro network or mobile communication network, or cable or general purpose The image decoding apparatus 100 may be transmitted to the image decoding apparatus 100 through various communication interfaces such as a universal serial bus (USB), and may be decoded by the image decoding apparatus 100 to restore and reproduce the image. Also, an image encoded in the bitstream by the image encoding apparatus 105 may be transferred from the image encoding apparatus 105 to the image decoding apparatus 100 through a computer-readable recording medium.

The above-described image encoding apparatus and the image decoding apparatus may be separate apparatuses, but may be made of one image encoding / decoding apparatus according to implementation. In this case, some components of the image encoding apparatus may be implemented to include at least the same structure or to perform at least the same function as substantially the same technical elements as some components of the image decoding apparatus.

Therefore, in the following detailed description of the technical elements and their operation principle, overlapping description of the corresponding technical elements will be omitted. Also, since the image decoding apparatus corresponds to a computing device applying the image encoding method performed by the image encoding apparatus to the decoding, the following description will focus on the image encoding apparatus.

The computing device may include a memory for storing a program or a software module for implementing an image encoding method and / or an image decoding method, and a processor connected to the memory and executing a program. Here, the image encoding apparatus may be referred to as an encoder, and the image decoding apparatus may be referred to as a decoder.

2 is a block diagram of a video encoding apparatus according to an embodiment of the present invention.

Referring to FIG. 2, the image encoding apparatus 20 may include a predictor 200, a subtractor 205, a transformer 210, a quantizer 215, an inverse quantizer 220, an inverse transformer 225, An adder 230, a filter 235, an encoded picture buffer 240, and an entropy encoder 245 may be included.

The prediction unit 200 may be implemented using a prediction module, which is a software module, and generates a prediction block with an intra prediction or inter prediction for a block to be encoded. can do. The prediction unit 200 may generate a prediction block by predicting a current block to be currently encoded in an image. In other words, the prediction unit 200 predicts the pixel value of each pixel of the current block to be encoded in the image according to the intra prediction or the inter prediction. Can generate a prediction block with In addition, the prediction unit 200 may transmit information necessary for generating a prediction block, such as information about a prediction mode such as an intra prediction mode or an inter prediction mode, to the encoder so that the encoder encodes the information about the prediction mode. Can be. In this case, the processing unit for which the prediction is performed, the processing method for which the prediction method and the details are determined may be determined according to the encoding / decoding setting. For example, the prediction method, the prediction mode, etc. may be determined in the prediction unit, and the performance of the prediction may be performed in the transformation unit.

The inter prediction unit may be classified into a translation motion model and an affine motion model according to a motion prediction method. In the case of the moving motion model, the prediction is performed by considering only the parallel movement. In the case of the out-of-movement model, the prediction can be performed by considering not only the parallel movement but also movements such as rotation, perspective, and zoom in / out. have. Assuming one-way prediction, one motion vector may be required for a motion model, but one or more motion vectors may be required for an out-of-motion motion model. In the case of an out-of-movement motion model, each motion vector may be information applied to a predetermined position of the current block, such as an upper left vertex and an upper right vertex of the current block, and a position of an area to be predicted of the current block through the corresponding motion vector. May be obtained in a pixel unit or a sub block unit. The inter prediction unit may apply some processes to be described later in common according to the motion model, and some processes may be applied separately.

The inter prediction unit may include a reference picture configuration unit, a motion estimation unit, a motion compensator, a motion information determiner, and a motion information encoder. The reference picture configuration unit may include, in the reference picture lists L0 and L1, pictures encoded before or after the current picture. A prediction block may be obtained from a reference picture included in the reference picture list, and the current picture may also be configured as a reference picture and included in at least one of the reference picture lists according to encoding settings.

In the inter prediction unit, the reference picture component may include a reference picture interpolator, and may perform an interpolation process for a fractional pixel according to interpolation precision. For example, an 8-tap DCT based interpolation filter may be applied in the case of luminance components, and a 4-tap DCT based interpolation filter may be applied in the case of chrominance components.

In the inter prediction unit, the motion estimation unit searches a block having a high correlation with the current block through a reference picture, and various methods such as a full search-based block matching algorithm (FBMA) and a three step search (TSS) may be used. The motion compensator means a process of obtaining a prediction block through a motion estimation process.

The motion information determiner may perform a process for selecting the optimal motion information of the current block in the inter prediction unit, and the motion information may be a skip mode, a merge mode, or a competition mode. It may be encoded by a motion information encoding mode. The mode may be configured by combining the supported modes according to the motion model, skip mode (movement), skip mode (movement), merge mode (movement), merge mode (movement), competition mode (movement), Competitive mode (outside the move) may be an example. Some of the modes may be included in the candidate group according to the encoding setting.

In the motion information encoding mode, a prediction value of motion information (motion vector, reference picture, prediction direction, etc.) of the current block may be obtained from at least one candidate block, and optimal candidate selection information is provided when two or more candidate blocks are supported. May occur. In the skip mode (no residual signal) and the merge mode (the residual signal present), the prediction value may be used as the motion information of the current block, and in the contention mode, difference information between the motion information of the current block and the prediction value may be generated. .

The candidate group for the motion information prediction value of the current block may be adaptive and have various configurations according to the motion information encoding mode. The motion information of a block spatially adjacent to the current block (eg, left, top, top left, top right, bottom left block, etc.) may be included in the candidate group, and the motion information of a temporally adjacent block may be included in the candidate group, and the spatial candidate And mixed motion information of temporal candidates may be included in the candidate group.

The temporally adjacent block may include a block in another image corresponding to (or corresponding to) the current block, and mean a block located in a left, right, up, down, top, right, left, bottom, and bottom block with respect to the block. Can be. The mixed motion information may mean information obtained by an average, a median value, etc. through motion information of spatially adjacent blocks and motion information of blocks adjacent in time.

There may be a priority for constructing the motion information prediction value candidate group. The order of inclusion in the prediction value candidate group configuration may be determined according to the priority, and the candidate group configuration may be completed when the number of candidate groups (determined according to the motion information encoding mode) is filled according to the priority. In this case, priority may be determined in order of motion information of spatially adjacent blocks, motion information of temporally adjacent blocks, and mixed motion information of spatial candidates and temporal candidates, but other modifications are also possible.

For example, among spatially adjacent blocks, candidates may be included in the candidate group in the order of left-up-top-right-bottom-left-top block, and candidate blocks in order of right-bottom-middle-right-bottom block among temporally adjacent blocks. It can be included in.

The subtraction unit 205 may generate a residual block by subtracting the prediction block from the current block. In other words, the subtractor 205 calculates a difference between the pixel value of each pixel of the current block to be encoded and the predicted pixel value of each pixel of the prediction block generated through the prediction unit, and is a residual signal in the form of a block. Residual blocks can be created. In addition, the subtraction unit 205 may generate the residual block according to units other than the block unit obtained through the block division unit described below.

The transform unit 210 may convert a signal belonging to a spatial domain into a signal belonging to a frequency domain, and a signal obtained through a transformation process is called a transformed coefficient. For example, a transform block having a transform coefficient may be obtained by transforming the residual block having the residual signal received from the subtractor, but the input signal is determined according to an encoding setting, which is not limited to the residual signal.

The transform unit can transform the residual block using transformation techniques such as a Hadamard transform, a Discrete Sine Transform (DST Based-Transform), or a Discrete Cosine Transform (DCT Based-Transform). The present invention is not limited thereto, and various transformation techniques may be used.

At least one of the transformation schemes may be supported, and at least one detailed transformation scheme may be supported in each transformation scheme. In this case, the detailed changed technique may be a transformation technique in which a part of the basis vector is configured differently in each transformation technique.

For example, in the case of DCT, one or more detailed transformation schemes of DCT-I through DCT-VIII may be supported, and in the case of DST, one or more detailed transformation schemes of DST-I through DST-VIII may be supported. A portion of the detailed transformation scheme may be configured to form a transformation technique candidate group. For example, the transformation may be performed by configuring DCT-II, DCT-VIII, and DST-VII as candidate candidates for transformation techniques.

The conversion can be performed in the horizontal / vertical direction. For example, one-dimensional transform is performed in the horizontal direction using the transform technique of DCT-II, and a total two-dimensional transform is performed by performing one-dimensional transform in the vertical direction using the transform technique of DST-VIII. The pixel value of can be converted into the frequency domain.

The transformation can be performed using one fixed transformation technique, or the transformation can be performed by adaptively selecting the transformation scheme according to the encoding / decoding. In this case, the adaptation method may be selected using an explicit or implicit method. In the explicit case, each transformation technique selection information or transformation technique set selection information applied to the horizontal and vertical directions may occur in a unit such as a block. In an implicit case, encoding settings may be defined according to an image type (I / P / B), a color component, a size, a shape of a block, an intra prediction mode, and a predetermined transformation scheme may be selected.

In addition, it may be possible that the partial transform is omitted depending on the encoding setting. That is, one or more of the horizontal / vertical units may be omitted, either explicitly or implicitly.

In addition, the transformer may transmit information necessary to generate a transform block to the encoder to encode the information, and store the information in the bitstream and transmit the information to the decoder, and the decoder of the decoder parses the information and inversely transforms the information. Can be used for the process.

The quantization unit 215 may quantize the input signal. In this case, the signal obtained through the quantization process is referred to as a quantized coefficient. For example, a quantization block having a quantization coefficient may be obtained by quantizing a residual block having a residual transform coefficient received from a transform unit. The received signal is determined according to an encoding setting, which is not limited to the residual transform coefficient.

The quantization unit may quantize the transformed residual block using a quantization technique such as dead zone uniform threshold quantization, quantization weighted matrix, and the like, but is not limited thereto. Quantization techniques can be used.

The quantization process may be omitted according to the encoding setting. For example, the quantization process may be omitted (including the inverse process) according to the encoding setting (for example, the quantization parameter is 0. That is, the lossless compression environment). As another example, the quantization process may be omitted when the compression performance through quantization is not exhibited according to the characteristics of the image. In this case, the region in which the quantization process is omitted among the quantization blocks M x N may be an entire region or a partial region (M / 2 x N / 2, M x N / 2, M / 2 x N, etc.) and may be quantized. The omission selection information may be determined implicitly or explicitly.

The quantization unit may transmit information necessary for generating a quantization block to the encoding unit and encode the information. The quantization unit stores the information in a bitstream and transmits the information to the decoder, and the decoder of the decoder parses the information and dequantizes it. Can be used for the process.

Although the above example has been described under the assumption that the residual block is transformed and quantized through the transform unit and the quantization unit, the residual block may be transformed from the residual signal to generate a residual block having transform coefficients, and the quantization process may not be performed. Not only may the quantization process be performed without converting the residual signal into transform coefficients, but neither the transformation nor the quantization process may be performed. This may be determined according to the encoder setting.

The inverse quantization unit 220 inverse quantizes the residual block quantized by the quantization unit 215. That is, the inverse quantizer 220 inversely quantizes the quantized frequency coefficient sequence to generate a residual block having the frequency coefficient.

The inverse transform unit 225 inversely transforms the residual block inversely quantized by the inverse quantization unit 220. That is, the inverse transformer 225 inversely transforms frequency coefficients of the inversely quantized residual block to generate a residual block having a pixel value, that is, a reconstructed residual block. Here, the inverse transform unit 225 may perform inverse transform by using the transformed method used in the transform unit 210 as the inverse.

The adder 230 reconstructs the current block by adding the prediction block predicted by the predictor 200 and the residual block reconstructed by the inverse transform unit 225. The reconstructed current block may be stored as a reference picture (or a reference block) in the coded picture buffer 240 and may be used as a reference picture when encoding the next block of the current block, another block in the future, or another picture.

The filter unit 235 may include one or more post-processing filter processes such as a deblocking filter, a sample adaptive offset (SAO), an adaptive loop filter (ALF), and the like. The deblocking filter may remove block distortion generated at the boundary between blocks in the reconstructed picture. The ALF may perform filtering based on a value obtained by comparing the reconstructed image with the original image after the block is filtered through the deblocking filter. The SAO may restore the offset difference from the original image on a pixel basis with respect to the residual block to which the deblocking filter is applied. Such a post-processing filter may be applied to the reconstructed picture or block.

The encoded picture buffer 240 may store a block or a picture reconstructed by the filter unit 235. The reconstructed block or picture stored in the encoded picture buffer 240 may be provided to the prediction unit 200 that performs intra prediction or inter prediction.

The entropy encoder 245 scans the generated quantization frequency coefficient sequence according to various scan methods to generate a quantization coefficient sequence, and outputs the encoded quantization coefficient sequence by encoding it using an entropy encoding technique. The scan pattern may be set to one of various patterns such as zigzag, diagonal lines, and rasters. In addition, encoded data including encoded information transmitted from each component may be generated and output as a bitstream.

3 is a block diagram of an image decoding apparatus according to an embodiment of the present invention.

Referring to FIG. 3, the image decoding apparatus 30 includes an entropy decoder 305, a predictor 310, an inverse quantizer 315, an inverse transformer 320, an adder and subtractor 325, a filter 330, and the like. The decoded picture buffer 335 may be configured.

In addition, the prediction unit 310 may be configured to include an intra prediction module and an inter prediction module.

First, when an image bitstream transmitted from the image encoding apparatus 20 is received, the image bitstream may be transferred to the entropy decoder 305.

The entropy decoder 305 may decode the bitstream to decode the decoded data including the quantized coefficients and the decoded information transmitted to each component.

The prediction unit 310 may generate a prediction block based on the data transferred from the entropy decoding unit 305. In this case, the reference picture list using a default construction technique may be constructed based on the reference picture stored in the decoded picture buffer 335.

The inter prediction unit may include a reference picture component unit, a motion compensator, and a motion information decoder, and some of the processes may be performed in the same process as the encoder, and some may be reversed.

The inverse quantizer 315 may inverse quantize the quantized transform coefficients provided in the bitstream and decoded by the entropy decoder 305.

The inverse transform unit 320 may generate a residual block by applying inverse transform techniques of inverse DCT, inverse integer transform, or the like to a transform coefficient.

In this case, the inverse quantization unit 315 and the inverse transform unit 320 perform the processes performed by the transform unit 210 and the quantization unit 215 of the image encoding apparatus 20 described above, and may be implemented in various ways. have. For example, the same process and inverse transform shared with the transform unit 210 and the quantization unit 215 may be used, and information about the transform and quantization process from the image encoding apparatus 20 (for example, transform size and transform). Shape, quantization type, etc.) may be used to reverse the transform and quantization processes.

The residual block that has undergone inverse quantization and inverse transformation may be added to the prediction block derived by the prediction unit 310 to generate an image block reconstructed. This addition can be made by the adder and subtractor 325.

The filter 330 may apply a deblocking filter to the reconstructed image block to remove blocking if necessary, and further add other loop filters to improve video quality before and after the decoding process. Can also be used.

The reconstructed and filtered image block may be stored in the decoded picture buffer 335.

Although not shown in the figure, the picture encoding / decoding apparatus may further include a picture divider and a block divider.

The picture splitter may divide (or partition) a picture into at least one processing unit such as a color space (eg, YCbCr, RGB, XYZ, etc.), a tile, a slice, a basic coding unit (or a maximum coding unit), or the like. The block splitter may divide the basic coding unit into at least one processing unit (for example, encoding, prediction, transform, quantization, entropy, in-loop filter unit, etc.).

The basic coding unit may be obtained by dividing a picture at regular intervals in a horizontal direction and a vertical direction. Based on this, division of tiles, slices, etc. may be performed, but is not limited thereto. The division unit such as the tile and the slice may be configured as an integer multiple of the basic coding block, but an exceptional case may occur in the division unit located at the image boundary. To this end, adjustment of the basic coding block size may occur.

For example, a picture may be divided into basic coding units and then divided into the units, or a picture may be divided into basic units and then divided into basic coding units. In the present invention, a description will be given on the assumption that the division and division order of each unit is the former, but the present invention is not limited thereto. In the latter case, the size of the basic coding unit may be changed to an adaptive case according to the division unit (tile, etc.). That is, it means that the basic coding block having a different size for each division unit can be supported.

In the present invention, a case of dividing a picture into basic coding units as a default setting will be described below. The default setting may mean that the picture is not divided into tiles or slices or the picture is one tile or one slice. However, as described above, when each division unit (tile, slice, etc.) is first partitioned and then divided into basic coding units based on the obtained unit (that is, each division unit is not an integer multiple of the basic coding unit, etc.). It should be understood that various embodiments may be applied in the same or modified form.

In the case of the slice unit, a slice may be composed of a bundle of at least one consecutive block according to a scan pattern, and in the case of a tile, the slice may be composed of a rectangular bundle of spatially adjacent blocks, and other additional division units are supported. And can be constructed by definition accordingly. The slice and the tile may be divided units supported for the purpose of parallel processing, and for this purpose, references between the divided units may be limited (that is, cannot be referred to).

A slice may generate split information of each unit as information about a start position of consecutive blocks, and in the case of a tile, generate slice information about horizontal and vertical split lines or tile position information (for example, upper left corner). , Upper right, lower left, and lower right positions).

In this case, the slice and the tile may be divided into a plurality of units according to the encoding / decoding.

For example, some units <A> may be units containing setting information that affects the encoding / decoding process (ie, including tile headers or slice headers), and some units <B> do not include setting information. May be a unit. Alternatively, some units <A> may be units that cannot refer to other units in the encoding / decoding process, and some units <B> may be units that may be referred to. In addition, some units <A> may be in a vertical relationship including other units <B> or some units <A> may be in a relationship equivalent to other units <B>.

Here, A and B may be slices and tiles (or tiles and slices). Alternatively, A and B may be composed of one of slices or tiles. For example, A may be a slice / tile <type 1> and B may be configured as a slice / tile <type 2>.

Here, Type 1 and Type 2 may each be one slice or tile. Alternatively, type 1 may be a plurality of slices or tiles (including type 2) (slice set or tile set), and type 2 may be one slice or tile.

As described above, the present invention is described assuming that a picture is composed of one slice or tile. However, when two or more division units occur, the above description may be applied to an embodiment to be described below. In addition, A and B are examples of properties that a division unit may have, and an example in which A and B of each example are mixed is also possible.

Meanwhile, the block divider may be divided into blocks of various sizes. In this case, the block may be composed of one or more blocks according to the color format (for example, one luminance block and two chrominance blocks, etc.), and the size of the block may be determined according to the color format. In the following description, for convenience of description, a block corresponding to one color component (luminance component) will be described.

The following description applies to one color component but differs in proportion to the color format (e.g., in the case of YCbCr 4: 2: 0, the ratio of the width and height of the luminance component and the chrominance component is 2: 1). It should be understood that modifications may be made to the color components. It should also be understood that block partitioning that is dependent on other color components (e.g., depending on the block partitioning result of Y in Cb / Cr) may be possible, but block partitioning independent of each color component may be possible. . In addition, although one common block division setting (which is proportional to the length ratio) can be used, it is also necessary to consider and consider that the individual block division setting is used according to the color component.

The block may have a variable size such as M × N (M and N are integers such as 4, 8, 16, 32, 64, 128, etc.) and may be a unit (coding block) for performing encoding. In detail, the present invention may be a unit that is a basis for prediction, transform, quantization, and entropy encoding, and in the present invention, this is collectively referred to as a block. Herein, the block may be understood as a broad concept including not only a rectangular block but also various types of regions such as a triangle and a circle, and the present invention will be described based on the case of a rectangular shape.

The block divider may be set in relation to each component of the image encoding apparatus and the decoding apparatus, and the size and shape of the block may be determined through this process. In this case, the set block may be defined differently according to the configuration unit, and a prediction block in the prediction unit, a transform block in the transform unit, and a quantization block in the quantization unit may correspond to this. Can be. However, the present invention is not limited thereto, and a block unit according to another component may be further defined. In the present invention, it is assumed that the inputs and outputs are blocks (that is, rectangular shapes) in each component, but some components may be input / output in other forms (eg, squares, triangles, etc.). .

The size and shape of the initial (or starting) block of the block division may be determined from higher units. For example, in the case of a coding block, a basic coding block may be an initial block, and in the case of a prediction block, a coding block may be an initial block. In addition, in the case of a transform block, a coding block or a prediction block may be an initial block, which may be determined according to a sub / decoding setting.

For example, when the encoding mode is intra, the prediction block may be an upper unit of the transform block, and in the inter case, the prediction block may be an independent unit of the transform block. The initial block, which is the start unit of the partition, may be divided into blocks of small size, and when the optimal size and shape according to the partition of the block is determined, the block may be determined as the initial block of the lower unit. The initial block, which is the start unit of the division, may be referred to as the initial block of the upper unit. Here, the upper unit may be a coding block and the lower unit may be a prediction block or a transform block, but is not limited thereto. When the initial block of the lower unit is determined as in the above example, a partitioning process for searching for a block having an optimal size and shape like the upper unit may be performed.

In summary, the block division unit may divide the basic coding unit (or the largest coding unit) into at least one coding unit (or the lower coding unit). In addition, the coding unit may perform division in at least one prediction unit and may perform division in at least one transformation unit. The coding unit may perform splitting into at least one coding block, the coding block may perform splitting into at least one prediction block, and may split into at least one transform block. The prediction unit may perform division into at least one prediction block, and the transform unit may perform division into at least one transform block.

In this case, some blocks may be combined with other blocks to perform one division process. For example, when the coding block and the transform block are combined into one unit, a partitioning process is performed to obtain an optimal block size and shape, which is optimized for the transform block as well as the optimal size and shape of the coding block. Size and shape. Alternatively, the coding block and the transform block may be combined in one unit, the prediction block and the transform block may be combined in one unit, the coding block, the prediction block and the transform block may be combined in one unit, Combination of other blocks may be possible. However, the combination or not is not applied collectively in an image (picture, slice, tile, etc.), but detailed conditions in units of blocks (for example, image type, encoding mode, block size / shape, prediction mode information, etc.). ) May be adaptively determined.

As described above, when a block having an optimal size and shape is found, mode information (for example, split information, etc.) for this may be generated. The mode information may be stored in the bitstream together with information generated by the component to which the block belongs (for example, prediction related information and transform related information) and transmitted to the decoder, and may be parsed in units of the same level by the decoder. Can be used in the video decoding process.

Hereinafter, the division scheme will be described. For convenience of explanation, it is assumed that the initial block is in the form of a square, but the same may be similarly or similarly applied to the case where the initial block is in the form of a rectangle.

Various methods for block partitioning may be supported, but the present invention will be described with emphasis on tree-based partitioning, and at least one tree partitioning may be supported. In this case, a quad tree (Quad Tree. QT), a binary tree (BT), a ternary tree (TT), etc. may be supported. When one tree method is supported, it may be referred to as a single tree split, and when two or more tree methods are supported, a multi-tree method.

In the case of quad-tree splitting, this means that the blocks are divided into two in the horizontal and vertical directions, and in the case of binary tree splitting, the blocks are divided into two in either the horizontal or vertical directions. This means that the system is divided into three directions in either the horizontal or vertical direction.

In the present invention, when the pre-division block is M x N, the quad tree division is divided into four M / 2 x N / 2, and the binary tree division is divided into two M / 2 x N or M x N / 2, In the case of ternary tree partitioning, it is assumed that the partition is divided into M / 4 x N / M / 2 x N / M / 4 x N or M x N / 4 / M x N / 2 / M x N / 4. However, the splitting result is not limited to the above case, and various modification examples may be possible.

One or more of the tree divisions may be supported according to the encoding / decoding setting. For example, it can support quad-tree splitting, or can support quad-tree splitting and binary tree splitting, or it can support quad-tree splitting and ternary tree splitting, or it can support quad-tree splitting and binary tree splitting and ternary tree splitting. Can support

The above example is a case where the basic partitioning scheme is a quad tree and binary tree partitioning and ternary tree partitioning are included in the additional partitioning scheme depending on whether other trees are supported, but various modifications may be possible. In this case, information on whether or not the other tree is supported (bt_enabled_flag, tt_enabled_flag, bt_tt_enabled_flag, etc., may have a value of 0 or 1, 0 is not supported, 1 is supported) or is implicitly determined according to the encoding / decoding setting or sequence, It may be determined explicitly in units of a picture, a slice, a tile, and the like.

The split information may include information on whether to split (tree_part_flag. Or qt_part_flag, bt_part_flag, tt_part_flag, bt_tt_part_flag. The value may be 0 or 1, and 0 may be split and 1 is split). In addition, depending on the partitioning scheme (binary tree, ternary tree), the split direction (dir_part_flag., Or bt_dir_part_flag, tt_dir_part_flag, bt_tt_dir_part_flag. Vertical>) may be added, which may be information that may occur when splitting is performed.

When a plurality of tree partitions are supported, various partition information configurations may be possible. The following description assumes an example of how the partition information is configured at one depth level (ie, for the convenience of explanation, although recursive partitioning may be possible because one or more supported partition depths are set to one or more). do.

As an example (1), information on whether to divide is checked. At this time, if the division is not performed, the division ends.

If the split is performed, the selection information on the split type (for example, tree_idx. 0 is QT, 1 is BT, 2 is TT) is checked. At this time, additionally check the split direction information according to the selected split type, and start again from the beginning if additional splitting is possible due to the next step (when the split depth has not reached the maximum, or end splitting if splitting is not possible). Go to).

In one example (2), the information on whether to split some tree methods (QT) is checked and the process proceeds to the next step. In this case, if the split is not performed, information on whether or not the partial tree method BT is split is checked. In this case, if the split is not performed, information on whether or not the partial tree method TT is split is checked. At this time, if the division is not performed, the division ends.

If splitting of some tree type (QT) is performed, the process proceeds to the next step. In addition, if the splitting of some tree methods BT is performed, the split direction information is checked and the process proceeds to the next step. In addition, if the split of some tree split schemes (TT) is performed, the split direction information is checked and the process proceeds to the next step.

In one example (3), the information on whether to split for some tree method (QT) is checked. At this time, if the split is not performed, the information on whether or not to split some tree methods (BT and TT) is checked. At this time, if the division is not performed, the division ends.

If splitting of some tree type (QT) is performed, the process proceeds to the next step. In addition, if splitting of some tree methods BT and TT is performed, the split direction information is checked and the process proceeds to the next step.

The above example may be the case where the priority of the tree split exists (examples 2 and 3) or does not exist (example 1), but various modification examples may be possible. In addition, in the above example, the division of the current step is an example for explaining a case that is not related to the division result of the previous step, but it may also be possible to set the division of the current step depending on the division result of the previous step.

For example, in the case of the first to third examples, if the tree-type partitioning (QT) is performed in the previous step and the current step is transferred to the current step, the splitting of the same tree method (QT) may be supported in the current step.

On the other hand, if some tree-type splitting (QT) is not performed in the previous step and another tree-type splitting (BT or TT) is carried out to the current step, except for some tree-type splitting (QT), some tree-type splitting is performed. It may also be possible that the division of (BT and TT) is supported for subsequent steps, including the current step.

In the above case, since the tree configuration supported for block partitioning may be adaptive, the partitioning information configuration described above may also be configured differently. (Examples to be described below are assumed to be examples 3) In other words, if the splitting of some tree methods (QT) is not performed in the previous step, the splitting process may be performed without considering some tree methods (QT) in the present step. Can be. In addition, splitting information (eg, splitting information, splitting direction information, etc.) regarding the related tree method may be configured by removing the splitting information.

The above example is adaptive for the case where block partitioning is allowed (e.g., block size is within the range between the maximum and minimum values, the partitioning depth of each tree method does not reach the maximum depth <allowed depth>, etc.). It is related to the structure of partitioning information, and adapts to the case where the block partitioning is limited (for example, the block size does not exist in the range between the maximum value and the minimum value, the partitioning depth of each tree method reaches the maximum depth, etc.). Partitioning information may be configured.

As already mentioned, tree-based partitioning in the present invention can be performed using a recursive manner. For example, when the partition flag of the coding block having the partition depth k is 0, encoding of the coding block is performed in the coding block having the partition depth k, and when the partition flag of the coding block having the partition depth k is 1, the coding block. The encoding of is performed in N sub-coding blocks having a division depth of k + 1 (wherein N is an integer of 2 or more, such as 2, 3, 4) according to a division scheme.

The sub coded block may be set again to a coded block k + 1 and divided into sub coded blocks k + 2 through the above process. It can be determined according to.

In this case, the bitstream structure for expressing the partition information may be selected from one or more scan methods. For example, the bitstream of the split information may be configured based on the split depth order, or the bitstream of the split information may be configured based on whether the split information is divided.

For example, in the case of the partition depth ordering method, after obtaining partitioning information at the current level of depth based on the first block, the partitioning information is obtained at the next level of depth. A method of preferentially acquiring additional splitting information in the received block, and another additional scanning method may be considered. In the present invention, it is assumed that a bitstream of partitioning information is configured based on partitioning.

As described above, various cases regarding block partitioning have been described, and a fixed or adaptive setting regarding block partitioning may be supported.

In this case, the setting regarding block division may explicitly include related information in units of a sequence, a picture, a slice, a tile, and the like. Alternatively, the block division setting may be implicitly determined according to the sub / decoding setting, wherein the sub / decoding setting includes various sub / decoding elements such as image type (I / P / B), color component, type of division, and depth of division. It may be configured according to one or a combination of two or more.

In the image encoding method according to an embodiment of the present invention, the intra prediction may be configured as follows. The intra prediction of the prediction unit may include a reference pixel construction step, a prediction block generation step, a prediction mode determination step, and a prediction mode encoding step. In addition, the image encoding apparatus may be configured to include a reference pixel constructing unit, a predictive block generating unit, and a prediction mode encoding unit for implementing a reference pixel constructing step, a predictive block generating step, a prediction mode determining step, and a prediction mode encoding step. Some of the processes described above may be omitted or other processes may be added, and may be changed in an order other than the order described above.

4 is an exemplary diagram illustrating an intra prediction mode according to an embodiment of the present invention.

Referring to FIG. 4, it is assumed that 67 prediction modes are configured as prediction mode candidate groups for intra prediction, of which 65 are directional modes and 2 are non-directional modes (DC, Planar). Not limited to this, various configurations may be possible. In this case, the directional mode may be divided into tilt (eg, dy / dx) or angle information (Degree). In addition, all or part of the prediction mode may be included in the prediction mode candidate group of the luminance component or the chrominance component, and other additional modes may be included in the prediction mode candidate group.

In the present invention, the direction of the directional mode may mean a straight line, and the curvature directional mode may further be configured as a prediction mode. In addition, in the non-directional mode, the DC mode for obtaining a prediction block with an average (or weighted average, etc.) of pixels of adjacent neighboring blocks (eg, left, top, top left, top right, bottom left blocks, etc.) of the current block. The pixel of the block may include a planar mode for obtaining a prediction block through linear interpolation.

In the DC mode, the reference pixel used to generate the prediction block may be obtained from blocks that are bound in various combinations such as left, top, left + top, left + left, top + top, left + top + bottom + top, and the like. The reference pixel acquisition block position may be determined according to a sub / decoding setting defined by an image type, a color component, and a size / shape / position of a block.

Here, in the planar mode, the pixel used for generating the prediction block includes a region consisting of reference pixels (eg, left, top, top left, top right, bottom left, etc.) and a region not consisting of reference pixels (eg, right, Lower, lower right, etc.), in the case of areas not configured as reference pixels (i.e., uncoded), one or more pixels are used in the area configured as reference pixels (e.g., copy as is, weighted average, etc.) It is possible to explicitly generate information on at least one pixel of an area which may be obtained implicitly or not constituted of a reference pixel. Thus, a prediction block can be generated using an area composed of reference pixels and an area not composed of reference pixels as described above.

5 is a conceptual diagram illustrating in-screen prediction of a directional mode and a non-directional mode according to an embodiment of the present invention.

Referring to FIG. 5A, intra prediction is shown according to modes in the vertical 5a, horizontal 5b, and diagonal 5c to 5e directions. Referring to FIG. 5B, intra prediction according to the DC mode is illustrated. Referring to FIG. 5C, the intra prediction according to the planar mode is illustrated.

Additional non-directional modes other than the above description may be included, and the present invention will be described based on the linear directional mode and the non-directional modes of DC and Planar. However, changes to other cases may also be possible.

4 may be a prediction mode fixedly supported regardless of the size of a block. In addition, the prediction mode supported according to the size of the block may be different from FIG. 4.

For example, the number of prediction mode candidates is adaptive (for example, the angles between the prediction modes are equally spaced, but the angles are set differently. 9, 17, 33, 65, 129, etc. based on the directional mode). The number of prediction mode candidates may be fixed but may be of different configuration (eg, directional mode angle, non-directional type, etc.).

In addition, FIG. 4 may be a prediction mode fixedly supported regardless of the shape of a block. In addition, the prediction mode supported according to the shape of the block may be different from FIG. 4.

For example, the number of prediction mode candidates is adaptive (e.g., the number of prediction modes derived from the horizontal or vertical direction depending on the aspect ratio of the block is small or high) or the number of prediction mode candidates is It may be fixed but in a different configuration (e.g., more precisely set the prediction mode derived from the horizontal or vertical direction depending on the aspect ratio of the block).

Alternatively, the prediction mode of the longer block may support a larger number, and the prediction mode of a shorter block may support a smaller number. In the case of a longer block, the prediction mode interval may be represented in FIG. 4. The mode located to the right of mode 66 (for example, a mode with an angle of +45 degrees or more relative to mode 50, that is, a mode with numbers such as 67 to 80) or the mode to the left of mode 2 ( For example, a mode having an angle of -45 degrees or more based on mode 18, that is, a mode having a number such as -1 to -14 may be supported. This can be determined by the ratio of the width to the length of the block, and vice versa.

In the present invention, the prediction mode will be described with reference to the case where the prediction mode is fixedly supported (regardless of any sub / decoding factor) as shown in FIG. 4, but it is also possible to set the prediction mode that is adaptively supported according to the encoding setting. .

In addition, when classifying prediction modes, horizontal and vertical modes (modes 18 and 50), some diagonal modes (Diagonal up right <2>, Diagonal down right <34>, Diagonal down left <66>, etc.) Etc. may be a reference, which may be a classification method performed based on some directionality (or angle, 45 degrees, 90 degrees, etc.).

In addition, some modes (modes 2 and 66) located at both ends of the directional mode may be a mode used as the reference mode for classification of prediction modes, which is an example when the intra prediction mode is configured as shown in FIG. 4. That is, when the prediction mode configuration is adaptive, an example in which the reference mode is changed may also be possible. For example, mode 2 may be replaced with a mode with a number less than or greater than 2 (-2, -1, 3, 4, etc.), or mode 66 with a number less than or greater than 66 (64, 66). , 67, 68, etc.).

In addition, an additional prediction mode regarding the color component may be included in the prediction mode candidate group. The following describes the color copy mode and the color mode as an example of the prediction mode through the following.

(Color Copy Mode)

A prediction mode related to a method of obtaining data for generating a prediction block from an area located in another color space may be supported.

For example, a prediction mode for a method of obtaining data for generating a prediction block in another color space using correlation between color spaces may be an example.

6 is a conceptual diagram illustrating an intra prediction according to a color copy mode according to an embodiment of the present invention. Referring to FIG. 6, the current block C of the current color space M may perform prediction using data of a corresponding area D of another color space N at the same time t.

In this case, the correlation between the color space may mean the correlation between Y and Cb, Y and Cr, and Cb and Cr when YCbCr is taken as an example. That is, in the case of the color difference component Cb or Cr, a reconstructed block of the luminance component Y corresponding to the current block may be used as the prediction block of the current block (basic setting of an example in which color difference vs luminance is described later). Alternatively, a reconstructed block of some color difference components Cb or Cr corresponding to the current block of some color difference components Cr or Cb may be used as a prediction block of the current block.

Here, in some color formats (eg, YCbCr 4: 4: 4, etc.), an area corresponding to the current block may have the same absolute position in each image. Alternatively, in some color formats (eg, YCbCr 4: 2: 0, etc.), relative positions within each image may be the same. The corresponding position may be determined according to the ratio of width and height according to the color format, and the corresponding pixel in the color space different from the pixel of the current image has width and height according to the color format in each component of the coordinates of the current pixel. It can be obtained by multiplying or dividing by the length ratio of.

For convenience of description, some color formats (4: 4: 4) will be mainly described, but it should be understood that the position of the corresponding area in another color space may be determined according to the width / length length ratio according to the color format. do.

In the color copy mode, a reconstructed block of another color space may be used as a prediction block as it is, or a block obtained in consideration of correlation between color spaces may be used as a prediction block. A block obtained by considering correlation between color spaces refers to a block that can be obtained by performing correction on an existing block. Specifically, in the formula {P = a * R + b}, a and b denote values used for correction, and R and P denote values obtained from different color spaces and prediction values of the current color space, respectively. can do. Here, P means a block obtained in consideration of the correlation between color spaces.

In this example, it is assumed that data obtained by using the correlation between color spaces is used as a prediction value of the current block, but it may also be possible when the data is used as a correction value applied to the prediction value of a current block that already exists. have. That is, the prediction value of the current block may be corrected using the residual value of another color space.

In the present invention, it is assumed that the former case is described, but the present invention is not limited thereto, and the same or modified application to the case used as a correction value may be possible.

The color copy mode may be supported explicitly or implicitly depending on the encoding / decoding setting. In this case, the encoding / decoding setting may be defined according to one or more combinations of the image type, color component, position / size / shape of the block, aspect ratio of the block, and the like. In an explicit case, related information may be included in units of a sequence, a picture, a slice, a tile, and the like. In addition, depending on the encoding / decoding setting, whether or not the color copy mode is supported may be implicitly determined in some cases, and related information may be explicitly generated in some cases.

In the color copy mode, correlation information (a and b, etc.) between color spaces may be explicitly generated or implicitly obtained according to the encoding / decoding setting.

In this case, an area to be compared (or referenced) to obtain correlation information may be a corresponding area (D of FIG. 6) of a different color space from the current block (C of FIG. 6). Alternatively, adjacent areas of the current block (left, upper, upper left, upper right, lower left block, etc. centering on C of FIG. 6) and adjacent areas (left, upper, Upper left, upper right, lower left block, etc.).

In the former case, since the correlation information needs to be directly obtained by using data of a block corresponding to the current block, it may correspond to an example of explicitly processing related information. That is, the data of the current block may be a case where correlation information has to be generated because encoding has not been completed yet. In the latter case, since correlation information may be obtained indirectly using data of an adjacent area of a block corresponding to an adjacent area of the current block, this may correspond to an example of implicitly processing related information.

In summary, in the former case, correlation information is obtained by comparing a block corresponding to the current block, and in the latter case, correlation information is obtained by comparing regions adjacent to the block corresponding to the current block. Data obtained by applying the correlation information to the corresponding block may be used as the prediction pixel of the current block.

Here, in the former case, the correlation information may be encoded as it is, or information on the difference may be encoded using the correlation information obtained by comparing adjacent regions as a prediction value. The correlation information may be information that may occur when the color copy mode is selected as the prediction mode.

The latter case may be understood as an example of an implied case in which no additional information is generated except that the color copy mode is selected as the optimal mode in the prediction mode candidate group. That is, this may be an example possible under the setting in which one correlation information is supported.

If two or more types of correlation information are supported, selection information for correlation information may be required in addition to selecting a color copy mode as an optimal mode. Like the above example, a configuration in which the explicit case and the implicit case are mixed according to the encoding / decoding setting may be possible.

In the present invention, description will be made mainly on the case where the correlation information is obtained indirectly, and the correlation information obtained may be N or more (N is an integer of 1 or more such as 1, 2, 3). The setting information about the number of correlation information may be included in units of a sequence, a picture, a slice, a tile, and the like. In some of the examples described below, it should be understood that the case where k correlation information is supported may mean the same as the case where k color copy modes are supported.

FIG. 7 is an exemplary diagram illustrating a corresponding block of each color space and an area adjacent thereto in relation to a color copy mode according to an embodiment of the present invention. Referring to FIG. 7, examples (p and q) corresponding to pixels in a current color space M and a different color space N are shown, and examples that may occur in some color formats (4: 2: 0) It can be understood as. In addition, the corresponding relationship 7a for obtaining correlation information and the corresponding relationship 7b for applying a predicted value may be confirmed.

The following follows the description related to obtaining correlation information in color copy mode. In order to obtain the correlation information, pixel values of pixels in a predetermined area (all or part of adjacent areas of each block corresponding to the current block) of each color space may be compared (or used) (ie, 1: 1). Pixel value comparison process). In this case, the compared pixel values may be obtained based on corresponding pixel positions of each color space. The pixel value may be a value derived from at least one pixel in each color space.

For example, in some color formats (4: 4: 4), the pixel value of one pixel in the chrominance space and the pixel value of one pixel in the luminance space may be used as the pixel value corresponding to the correlation information acquisition process. Alternatively, in the case of some color formats (4: 2: 0), a pixel value of one pixel of a chrominance space and a pixel value derived from one or more pixels of a luminance space (that is, obtained through a down sampling process) are obtained from the correlation information acquisition process. It may be used as a pixel value corresponding to.

Specifically, in the former case, p [x, y] of the chrominance space can be compared with q [x, y] of the luminance space. In this case, the brightness value of one pixel may be used as the pixel value. In the latter case, p [x, y] in the chrominance space is q [2x, 2y], q [2x, 2y + 1], q [2x + 1, 2y], q [2x + 1, 2y + 1 in the luminance space. ] And the like.

Since a 1: 1 pixel value comparison must be performed, one of the plurality of pixels may be used as a value for comparing the pixel value of the color difference pixel in the luminance space. That is, the brightness value of one of the plurality of pixels is used as it is. Alternatively, one pixel value from two or more pixels (k. K is an integer of 2 or more, such as 2, 4, 6, etc.) may be used as a value for deriving and comparing the pixel values. That is, a weighted average (weights may be equally assigned or non-uniformly assigned) to two or more pixels may be applied.

As described above, when there are a plurality of corresponding pixels, a pixel value of one preset pixel or a pixel value derived from two or more pixels may be used as a comparison value. In this case, one of the above two methods for deriving pixel values compared in each color space may be used alone or in combination according to a sub / decoding setting.

The following description may be based on the assumption that pixel values of one pixel in the current color space are used for comparison, and that one or more pixels may be used for deriving pixel values in another color space. For example, assume that the color format is YCbCr 4: 2: 0, the current color space is the chrominance space, and the other color space is the luminance space. The method for deriving pixel values will be described based on other color spaces.

For example, it may be determined according to the shape of the block (horizontal / vertical length ratio). As a detailed example, p [x, y] of the color difference space adjacent to the longer side of the current block (or predicted block) can be compared with q [2x, 2y] of the luminance space, and p [x, y] can be compared with the average of q [2x, 2y] and q [2x + 1, 2y] of the luminance space.

In this case, the above content may be applied to some block shapes (rectangular) regardless of the horizontal / vertical length ratio, or the ratio of horizontal / vertical length may be a certain ratio (k: 1 or 1: k, for example, 2: 1, 4: 1, etc.). Similarly, k may be applied only when the value is greater than or equal to 2) or more.

For example, it may be determined according to the size of the block. As a detailed example, p [x, y of chrominance space when the size of the current block is greater than or over a certain size (M x N. For example, 2 ^m x 2 ⁿ where m and n are integers greater than or equal to 2 to 6). ] Can be compared with q [2x + 1, 2y] of the luminance space, and in the case of less than / less than a certain size, p [x, y] of the chrominance space is q [2x, 2y], q [2x, 2y + 1].

In this case, one boundary value for size comparison may exist as in the above example, or an adaptive setting such as two or more (M ₁ x N ₁ , M ₂ x N _2, etc.) may be possible.

The above examples are some cases that can be considered in terms of calculation amount, and examples of various modifications are possible, including cases opposite to the above examples.

For example, it may be determined according to the position of the block. As a detailed example, when the current block is located inside a preset area (assuming a maximum coding block in this example), p [x, y] of the color difference space is q [2x, 2y] of the luminance space. It can be compared with the average of q [2x + 1, 2y], q [2x, 2y + 1], q [2x + 1, 2y + 1], and the boundary of the preset area (assuming the upper left boundary in this example). ), P [x, y] of the chrominance space can be compared with q [2x + 1, 2y + 1] of the luminance space. The preset area may mean an area set based on a slice, a tile, a block, and the like. In detail, it may be obtained based on an integer multiple of a slice, a tile, and a maximum coding / prediction / transformation block.

As another example, when the current block is located at some boundary of the region (assuming an upper boundary in this example), p [x, y] of the color difference space adjacent to the some boundary (upper) is q [2x + 1, 2y + 1], and p [x, y] of the color difference space adjacent to the inside (left) of the area is q [2x, 2y] of the luminance space. It can be compared with an average of q [2x + 1, 2y], q [2x, 2y + 1], q [2x + 1, 2y + 1].

The above examples are some cases that can be considered in terms of memory, and examples of various modifications are possible, including cases opposite to the above examples.

Through the above-described example, various cases regarding derivation of pixel values compared in each color space have been described. As described above, the pixel value derivation setting for acquiring the correlation information may be determined in consideration of various sub / decoding elements as well as the size / shape / position of the block.

As described above, the region compared for acquiring correlation information has been described in which one or two reference pixel lines of a block corresponding to the current block are used as shown in FIG. 7. That is, in the case of YCbCr 4: 4: 4, one reference pixel line is used, and in other formats, in some color spaces, two reference pixel lines are used as in some color spaces <color N> of FIG. 7. Means a case, but is not limited to this, various modifications may be possible.

The following will focus on the reference pixel lines of the current color space, and it should be understood that the reference pixel lines may be determined according to the color format in other color spaces. For example, the same number of reference pixel lines may be used or twice the reference pixel lines may be used.

In the color copy mode of the present invention, k reference pixel lines (where k is an integer greater than or equal to 1 and 2) may be used (or compared) to obtain correlation information. In addition, k reference pixel lines may be fixedly used or adaptively used. Next, various examples of setting the number of reference pixel lines will be described.

For example, it may be determined according to the shape of the block (horizontal / vertical length ratio). As a detailed example, two reference pixel lines adjacent to the longer side of the current block may be used, and one reference pixel line adjacent to the shorter side of the current block may be used.

In this case, the above content may be applied to some block shapes (rectangular) regardless of the horizontal / vertical length ratio, or the ratio of horizontal / vertical length may be a certain ratio (k: 1 or 1: k, for example, 2: 1, 4: 1, etc.). Likewise, k may be applied only if the value is greater than or greater than 2). In addition, two or more boundary values for the horizontal / vertical length ratio exist, so that in the case of 2: 1 or 1: 2, two reference pixel lines adjacent to the long side (or the short side) are used, and 4: In the case of 1 or 1: 4, expansion such as using three reference pixel lines adjacent to the long side (or the short side) may be possible.

In the above example, the long side (or the short side) uses s reference pixel lines and the short side (or the long side) uses t reference pixel lines according to the ratio of width to length. It may be an example for the case where s is greater than or equal to t (that is, s and t are integers of 1 or more).

For example, it may be determined according to the size of the block. As a detailed example, two reference pixel lines may be used when the size of the current block is greater than or over a certain size (M x N. For example, 2 ^m x 2 ⁿ where m and n are integers greater than or equal to 2 to 6). And one reference pixel line may be used when the size is less than / less than a predetermined size.

In this case, one boundary value for size comparison may exist as in the above example, or an adaptive setting such as two or more (M ₁ x N ₁ , M ₂ x N _2, etc.) may be possible.

For example, it may be determined according to the position of the block. As a detailed example, when the current block is located inside a preset area (which can be derived from a previous description related to obtaining correlation information. In this example, it is assumed to be the largest coding block), two reference pixel lines may be used. One reference pixel line may be used when located at a boundary (assuming an upper left boundary in this example).

As another example, when the current block is located at some boundary of the region (assuming an upper boundary in this example), one reference pixel line adjacent to the some boundary (upper) may be used, and 2 adjacent to the inside of the region (left) Reference pixel lines may be used.

The above examples are some cases that can be considered in terms of accuracy of the correlation information, memory, and the like, and examples of various modifications are possible, including cases opposite to the above examples.

In the above-described example, various cases regarding reference pixel line settings used to obtain correlation information in each color space have been described. As described above, the reference pixel line setting for obtaining correlation information may be determined in consideration of various sub / decoding elements as well as the size / shape / position of a block.

Next, another case of a region to be compared (or referenced) for obtaining correlation information will be described. The area to be compared may be a pixel adjacent to a position of a left, an upper, a left top, a right top and a bottom left adjacent to a current block of a current color space.

In this case, the area to be compared may be set to include all blocks of the left, top, top left, top right, and bottom left positions. Alternatively, the reference pixel area may be configured by a combination of blocks of some positions. For example, left / top / left + top / left + top + top / left + left / top + top / left + left + top + bottom / top + top + top / left + top + top / left + top + bottom, etc. It is possible to construct regions that are compared by a combination of adjacent blocks.

8 is an exemplary diagram for setting a region for obtaining correlation information of a color copy mode according to an embodiment of the present invention. In the case of a to e of FIG. 8, it may correspond to the above-described example (left + top, top + right, left + left, left + top + right, left + top + bottom), and blocks of some positions may be one or more. An example (f and g of FIG. 8) may be possible in which the sub-block is divided into a part of which is set as an area for obtaining correlation information. That is, an area for obtaining correlation information may be set to one or more subblocks located in some directions. Alternatively, regions for obtaining correlation information may be set in one or more blocks located in some directions (a) and one or more sub blocks located in some directions (b), where a and b mean different directions. Also, an example of area setting (h, i in FIG. 8) for acquiring correlation information with non-contiguous blocks may also be possible.

In summary, the areas to be compared for obtaining correlation information may be configured as preset areas. Alternatively, the present invention may be configured in various combinations of some regions. That is, the area to be compared may be fixed or adaptively configured according to the negative / decryption setting.

Next, we will look at various examples of which directions of adjacent areas are composed of reference areas around the current block of the current color space. Here, it is assumed that in a corresponding block of another color space, which direction adjacent areas are configured as the reference area is determined according to the configuration of the reference area of the current color block. In addition, it is assumed that the basic reference region is composed of left and upper blocks.

For example, it may be determined according to the shape of the block (horizontal / vertical length ratio). As a detailed example, when the current block is long horizontally, the left, top, and right upper blocks may be set as the reference area, and when the vertical block is long, the left, upper, lower left blocks may be set as the reference area.

In this case, the above content may be applied to some block shapes (rectangular) regardless of the horizontal / vertical length ratio, or the ratio of horizontal / vertical length may be a certain ratio (k: 1 or 1: k, for example, 2: 1, 4: 1, etc.). Likewise, k may be applied only if the value is greater than or greater than 2). In addition, two or more boundary values for the horizontal / vertical length ratio exist so that in the case of 2: 1 (or 1: 2), the left, top, and right top (or left, top, bottom left) blocks are set as reference regions. , 4: 1 (or 1: 4) may be extended such that upper, upper right (or left, lower left) blocks are set as reference regions.

For example, it may be determined according to the size of the block. As a detailed example, when the size of the current block is greater than or equal to a certain size (M x N. For example, 2 ^m x 2 ⁿ where m and n are one or more integers such as 2 to 6), the left and upper blocks are referred to as reference regions. In the case of being set to less than / less than a predetermined size, the left, top, and top left blocks may be set as reference regions.

In this case, one boundary value for size comparison may exist as in the above example, or an adaptive setting such as two or more (M ₁ x N ₁ , M ₂ x N _2, etc.) may be possible.

For example, it may be determined according to the position of the block. As a detailed example, when the current block is located inside a preset area (which can be derived from a previous description related to obtaining correlation information. In this example, it is assumed to be the largest coding block), the left, top, top left, top right, and bottom left blocks are referred to as reference areas. If it is set and located at the boundary of the preset area (assuming the upper left boundary in this example), the left and upper blocks may be set as the reference area.

As another example, if the current block is located at some boundary of the region (assuming the upper boundary in this example), except for the block adjacent to the some boundary (upper boundary), the left and lower blocks adjacent to the inside of the region are referred to as the reference region. Can be set. That is, the left and lower left blocks may be set as reference regions.

The above examples are some cases that can be considered in terms of calculation amount, memory, and the like, and examples of various modifications are possible, including cases opposite to the above examples.

In the above-described example, various cases regarding the reference region setting used to obtain correlation information in each color space have been described. As described above, the reference region setting for obtaining correlation information may be determined in consideration of various sub / decoding elements as well as the size / shape / position of the block.

In addition, the comparison area may be a pixel adjacent to a current block of a current color space. Here, all of the reference pixels may be used to obtain correlation information, or some of them may be used.

For example, when the current block (based on color M of FIG. 7) is a block having a pixel range of (a, b) to (a + 7, b + 7) (that is, 8 × 8), the comparison is performed. It is assumed that the region (the corresponding block is omitted since the description can be derived according to the color format) is one reference pixel line of the left and upper blocks of the current block.

In this case, all of the pixels within the range of (a, b-1) to (a + 7, b-1) and (a-1, b) to (a-1, b + 7) may be included in the region to be compared. have. Or (a, b-1), (a + 2, b-1), (a + 4, b-1), (a + 6, b-1) and (a-1) which are some pixels in the above range , b), (a-1, b + 2), (a-1, b + 4), and (a-1, b + 6). Alternatively, some pixels in the range may include (a, b-1), (a + 4, b-1), and (a-1, b), (a-1, b + 4).

The above example may be applicable for the purpose of reducing the amount of computation required for obtaining correlation. In addition to the many examples already described above, various sub / decoding elements such as the size / shape / position of a block may be considered in setting the reference pixel sampling of the compared area for obtaining correlation information. The examples are derivable, and thus detailed descriptions are omitted.

Through the various examples described above, various factors (such as deriving corresponding pixel values, number of reference pixel lines, reference region direction setting, reference pixel sampling, etc.) that influence the acquisition of correlation information have been described. The number of cases where the above examples are alone or mixed to affect the acquisition of correlation information may be possible.

The above description may be understood as a preset process for obtaining one piece of correlation information. In addition, as mentioned above, one or more correlation information may be supported according to encoding / decoding settings. In this case, two or more correlation information may be supported by leaving two or more of the above-described presetting (ie, a combination of factors affecting acquisition of correlation information).

In summary, parameter information based on correlation information may be derived from an adjacent area of a block corresponding to an adjacent area of the current block. That is, at least one parameter (eg, <a1, b1>, <a2, b2>, <a3, b3>, etc.) may be generated based on the correlation information, which may be generated in a restored block of another color space. It can be used as a value that is multiplied or added to the pixel.

The following is a description of the linear model applied in color copy mode. By applying the parameters obtained through the above process, prediction based on a linear model may be performed.

pred_sample_C (i, j) = a x rec_sample_D (i, j) + b

In the above formula, pred_sample_C means a predicted pixel value of a current block of a current color space, and rec_sample_D means a reconstructed pixel value of a corresponding block of another color space. a and b may be obtained by minimizing a regression error between the adjacent area of the current block and the corresponding area of the block, and may be calculated by the following equation.

In the above formula, D (n) means an adjacent area of the corresponding block, C (n) means an adjacent area of the current block, and N is a value set based on the horizontal or vertical length of the current block (this example Is assumed to be twice the minimum of width and length).

In addition, various methods, such as a straight-line equation, may be used to obtain correlation information based on minimum and maximum values of adjacent areas of each color space. In this case, one model may be used as a model for obtaining correlation information, or one of a plurality of models may be selected. In this case, the selection of one of the plurality of models means that the model information may be considered as a sub / decoding element for parameter information based on the correlation information. That is, when a plurality of parameters are supported, it may mean that even if the remaining correlation information related settings are the same, different correlation information may be divided according to different models for obtaining correlation.

In the case of some color formats (not 4: 4: 4), one pixel of the current block may correspond to one or more (two, four, etc.) pixels of the corresponding block. For example, in the case of 4: 2: 0, p [x, y] in the color space is q [2x, 2y], q [2x, 2y + 1], q [2x + 1, 2y], q in the luminance space. [2x + 1, 2y + 1] and the like.

For one prediction pixel value, one pixel value may be derived (7b) from a predetermined pixel value (or a prediction value) or two or more pixels among the corresponding plurality of pixels. That is, a reconstruction value may be obtained before applying the correlation information in at least one corresponding pixel in another color space to obtain one prediction pixel value. The number of various cases may be possible according to the encoding / decoding setting, and since the related description may be derived from the corresponding pixel value derivation 7a in order to obtain correlation information, the detailed description thereof will be omitted. However, the same or different settings may be applied to 7a and 7b.

(Color mode)

A prediction mode related to a method of obtaining a prediction mode for generating a prediction block from a region located in another color space may be supported.

For example, a prediction mode for a method of obtaining a prediction mode for generating a prediction block in another color space using correlation between color spaces may be an example. That is, the color mode may be a mode that does not have any particular prediction direction or prediction method, but is adaptively determined according to a prediction mode of a block corresponding to another color space using existing prediction directions and methods.

In this case, various color modes may be obtained according to the block division setting.

For example, in a setting where the block division for some color components (color difference) is implicitly determined according to the result of the block division for some color components (luminance), that is, when the block division of luminance components is explicitly determined, One block of some color components (color difference) may correspond to one block of some color space (luminance). Therefore (assuming 4: 4: 4. For other formats, the description of this example can be derived from the ratio of width to height.) The current block (color difference) is (a, b) to (a + m , b + n), a block may be referred to any pixel position within the pixel range of (a, b) to (a + m, b + n) of the corresponding block (luminance). In this case, one prediction mode may be obtained in a block including the corresponding pixel.

Alternatively, when separate block division is supported according to each color component (that is, when the block division of each color space is explicitly determined), one block of some color components (color difference) is used in some color space (luminance). It may correspond to one or more blocks. Therefore, even if the current block (color difference) has the same pixel range as the above example, the corresponding block (luminance) may be composed of one or more blocks according to the block division result. Thus, different prediction modes (ie, one or more modes) may be obtained in the corresponding block indicated by the pixel according to the pixel position within the pixel range of the current block.

If one color mode is supported in the intra prediction mode candidate group for the color difference component, it may be set at which position of the corresponding block to obtain the prediction mode.

For example, the prediction mode may be obtained at a position such as center-upper left-upper right-lower left-lower right of the corresponding block. That is, when the prediction mode is brought in the above order but the corresponding block is not available (for example, the encoding mode is Inter, etc.), the prediction mode of the position corresponding to the next order may be brought. Alternatively, the prediction mode may have a high frequency (more than 2 times) in the blocks of the position.

Alternatively, when supporting a plurality of color modes, it is possible to set where to obtain the prediction mode according to the priority. Alternatively, a combination of bringing some prediction modes according to the priority and bringing some prediction modes with high frequency in blocks of the position may be possible. In this case, the priority is one example, and various modifications may be possible.

The color mode and the color copy mode may be prediction modes that can support the color difference component. For example, the prediction mode candidate group for the color difference component may be configured including horizontal, vertical, DC, planar, and diagonal modes. Alternatively, the intra prediction mode candidate group may be configured to include a color mode and a color copy mode.

That is, directional + non-directional + color mode or directional + non-directional + color copy mode, directional + non-directional + color mode + color copy mode, and other additional color difference components It may be configured to include a mode for.

Depending on the encoding / decoding setting, it may be determined whether the color mode and the color copy mode are supported, which may be implicit or explicit. Alternatively, processing of explicit + implicit mixed configurations may be possible. This may include detailed settings related to the color mode and the color copy mode (eg, the number of supported modes, etc.) to allow for implicit or explicit processing.

For example, the related information may include related information explicitly in units of a sequence, a picture, a slice, a tile, a block, or the like, or various sub / decoding elements (eg, image type, position of a block, Size, type of block, width / length ratio of the block, etc.). Alternatively, some conditions may be implicit in some conditions or explicitly related information in some conditions.

9 is an exemplary diagram for describing a configuration of a reference pixel used for intra prediction according to an embodiment of the present invention. The size and shape (M x N) of the prediction block may be obtained through the block partitioner.

The block range information defined by the minimum block and the maximum block size for intra prediction may include related information in units of a sequence, a picture, a slice, a tile, and the like. In general, the size information may be set because the horizontal and vertical lengths are specified (for example, 32 x 32, 64 x 64, etc.), but the size information may be set in the form of the product of the horizontal and vertical lengths. For example, the minimum block size may correspond to 4 x 16, 8 x 8, 16 x 4, and the like when the product of the width and the length is 64.

In addition, setting the size information by specifying the horizontal and vertical lengths or setting the size information in the form of a product may be used in combination. For example, if the maximum block size is 4096, the product of width and length is 4096, and the maximum value of one of the lengths is 64, 64 x 64 may correspond to the size of the maximum block.

As in the above example, block division information may be mixed in addition to the minimum block and maximum block size information to finally determine the size and shape of the prediction block. In the present invention, the prediction block requires that the product of the width and length is greater than or equal to s (for example, s is a multiple of 2, such as 16 and 32), and one of the width and length lengths is k (for example, k is 4, Must be greater than or equal to 8). In addition, the width and height of the block may be defined under a setting equal to or less than v and w (for example, v and w are multiples of 2, such as 16, 32, and 64). Not limited to this, various block range settings may be possible.

Intra prediction may be generally performed in units of prediction blocks, but may be performed in units of coding blocks, transform blocks, or the like according to the setting of the block partitioner. After checking the block information, the reference pixel configuration unit may configure a reference pixel used for prediction of the current block. In this case, the reference pixel may be managed through a temporary memory (for example, an array <Array>, a primary or a secondary array, etc.), generated and removed for each intra-prediction process of the block, and the size of the temporary memory is referred to. It may be determined according to the configuration of the pixel.

In the present example, it is assumed that left, top, top left, top right, and bottom left blocks are used for prediction of the current block with respect to the current block. However, the present invention is not limited thereto, and block candidate groups having other configurations may be used for prediction of the current block. have. For example, the candidate group of neighboring blocks for the reference pixel may be an example when following a raster or Z scan, and some of the candidate groups may be removed according to the scanning order, or other block candidate groups (eg, right , Lower, lower right block, etc. may be configured).

In addition, if some prediction modes (color copy modes) are supported, this may also be considered a reference pixel since some regions of other color spaces may be used for prediction of the current block. An existing reference pixel (spatially adjacent area of the current block) and the additional reference pixel can be managed as one or separately (for example, reference pixel A and reference pixel B. That is, the reference pixel is used as if the temporary memory is used separately). Memory can be named separately).

For example, the temporary memory of the base reference pixel may have a size of <2 x blk_width + 2 x blk_height + 1> (based on one reference pixel line), and the temporary memory of the additional reference pixel may be (4: 4: 4 In the case of 4), the size of <blk_width x blk_height> (in case of 4: 2: 0, blk_width / 2 x blk_height / 2 is required). The temporary memory size is one example and is not limited thereto.

In addition, a neighboring area of a block corresponding to the current block to be compared (or referenced) to obtain correlation information may be managed as a reference pixel. That is, additional reference pixels may be managed according to the color copy mode.

In summary, adjacent regions of the current block may be included as reference pixels for intra prediction of the current block, and corresponding blocks of other color spaces and their adjacent regions may be included as reference pixels according to a prediction mode.

10 is a conceptual diagram illustrating a block adjacent to a target block of intra prediction according to an embodiment of the present invention. Specifically, the left side of FIG. 10 represents a block adjacent to the current block of the current color space, and the right side represents a corresponding block of another color space. For convenience of explanation, the following description assumes that a block adjacent to the current block of the current color space has a basic reference pixel configuration.

As illustrated in FIG. 9, the reference pixels used for prediction of the current block may include adjacent pixels (Ref_L, Ref_T, Ref_TL, Ref_TR, and Ref_BL in FIG. 9) of the left, top, left top, right top, and bottom left blocks. In this case, the reference pixel is generally composed of pixels of a neighboring block closest to the current block (a of FIG. 9 is represented by a reference pixel line), but other pixels (b of FIG. 9 and other outer lines of pixels) May also be configured in the reference pixel.

Pixels adjacent to the current block may be classified into at least one reference pixel line, and pixels closest to the current block are ref_0 {for example, pixels having a distance between the boundary pixel of the current block and the pixel 1. p (-1, -1) to p (2m-1, -1), p (-1,0) to p (-1,2n-1)}, and the next adjacent pixel {e.g., The distance between the boundary pixel and the pixel is 2.p (-2, -2) to p (2m, -2), p (-2, -1) to p (-2,2n)} is ref_1, and then the adjacent pixel. {For example, the distance between the boundary pixel and the pixel of the current block is 3.p (-3, -3) to p (2m + 1, -3), p (-3, -2) to p (-3, 2n + 1)} may be divided into ref_2 and the like. That is, it may be classified as a reference pixel line according to a pixel distance adjacent to the boundary pixel of the current block.

Here, the supported reference pixel lines may be N or more, and N may be an integer of 1 or more, such as 1 to 5. In this case, the reference pixel line candidate group is sequentially included in the reference pixel line closest to the current block, but is not limited thereto. For example, when N is 3, the candidate group may be sequentially configured as <ref_0, ref_1, ref_2>, or <ref_0, ref_1, ref_3>, <ref_0, ref_2, ref_3>, <ref_1, ref_2, ref_3 It may also be possible for the candidate group to be configured in such a manner as to exclude non-sequential or closest reference pixel lines such as >

Prediction may be performed using all reference pixel lines in the candidate group, or prediction may be performed using some reference pixel lines (one or more).

For example, one of a plurality of reference pixel lines may be selected according to a negative / decoding setting to perform intra prediction using the reference pixel lines. Alternatively, two or more of the plurality of reference pixel lines may be selected to perform intra prediction by using the reference pixel lines (for example, applying a weighted average to data of each reference pixel line).

Here, the reference pixel line selection may be determined implicitly or explicitly. For example, in the case of implied, it means that it is determined according to an encoding / decoding setting defined by one or a combination of two or more elements such as an image type, a color component, and a size / shape / position of a block. In addition, the explicit case means that reference pixel line selection information may occur in a unit such as a block.

Although the present invention will be described based on the case of performing intra-prediction using the nearest reference pixel line, it should be understood that various embodiments described below may be applied in the same or similar manner even when a plurality of reference pixel lines are used.

The reference pixel component of the intra prediction according to the present invention may include a reference pixel generator, a reference pixel interpolator, a reference pixel filter, and the like, and may include all or part of the above configuration.

The reference pixel configuration unit may check the availability of the reference pixel to classify the available reference pixel and the unavailable reference pixel. Here, the availability of the reference pixel is determined to be unavailable when at least one of the following conditions is satisfied. Of course, although the possibility of using the reference pixel may be determined by additional conditions not mentioned in the examples described below, the present invention will be described on the assumption that it is limited to the conditions in the examples described below.

For example, when located outside the picture boundary, the same division unit as the current block (for example, a unit that cannot be referred to each other such as a slice or a tile, etc., but a unit such as a slice or a tile has characteristics that can be referred to each other) If it does not belong to the exception processing, even if not in the same partition unit, it can be determined that it is unavailable if any one of the cases where encoding / decoding is not completed is satisfied. That is, when none of the above conditions are satisfied, it can be determined that it can be used.

In addition, the use of the reference pixel can be restricted by the negative / decoding settings. For example, even when it is determined to be usable according to the above condition, the use of the reference pixel may be limited according to whether limited intra prediction (eg, constrained_intra_pred_flag) is performed. The limited intra prediction may be performed when an attempt is made to prohibit the use of a block reconstructed by referencing from another image as a reference pixel when attempting to perform error robust encoding / decoding to an external factor such as a communication environment.

If the limited intra prediction is deactivated (e.g., I picture type or constrained_intra_pred_flag is set to 0 in P or B picture type), the above-described conditions such as being located inside the picture boundary are excluded. If satisfied) may be available.

Alternatively, when limited intra prediction is activated (for example, constrained_intra_pred_flag is set to 1 in a P or B picture type), it may be determined whether a reference pixel candidate block is used according to an encoding mode (Intra or Inter). In general, it can be used in Intra mode, but not in Inter mode. In the above example, it is assumed that use is determined according to an encoding mode. However, the use may be determined according to various other encoding / decoding elements.

Since the reference pixel is composed of one or more blocks, when the possibility of the reference pixel is identified and classified, the reference pixel may be classified into three cases, such as <usable for all>, <some for some use>, and <not for all use>. In other cases except when all are available, reference pixels of candidate block positions that are not available may be filled or generated (A). Alternatively, the reference pixel of the unusable candidate block position may not be used in the prediction process, and the prediction mode encoding / decoding may be performed except for the prediction mode in which the prediction is performed from the reference pixel of the corresponding position (B).

When the reference pixel candidate block is available, the pixel at the corresponding position may be included in the reference pixel memory of the current block. In this case, the pixel data may be copied as it is or included in the reference pixel memory through processes such as reference pixel filtering and reference pixel interpolation.

If the reference pixel candidate block is unavailable, it can be processed under the reference pixel processing A or B setting. An example of processing when the reference pixel candidate block is unavailable according to each setting is described below.

(A) When the reference pixel candidate block is unavailable, the pixel of the corresponding position obtained through the reference pixel generation process may be included in the reference pixel memory of the current block.

The following describes a method of generating a reference pixel at a position that cannot be used as an example of a reference pixel generation process.

For example, the reference pixel may be generated using any pixel value. Here, the arbitrary pixel value means one pixel value (for example, the minimum value of the pixel value range) belonging to the pixel value range (for example, the pixel value range based on the bit depth or the pixel value range according to the pixel distribution in the corresponding image). Maximum value, median value, etc.). In more detail, the present invention may be an example in which all of the reference pixel candidate blocks are unavailable, but the present invention is not limited thereto and may be applicable even when only some of the reference pixel candidate blocks are unavailable.

Alternatively, the reference pixel may be generated from an area in which encoding / decoding of the image is completed. In detail, the reference pixel may be generated based on at least one available block (or available reference pixel) adjacent to the unavailable block. In this case, at least one of extrapolation, interpolation, and copying may be used.

(B) When the reference pixel candidate block is unavailable, the prediction mode using the pixel at the corresponding position may limit the use. For example, when the reference pixel at the TR position is not available in FIG. 9, the use of the 51 to 66 modes (FIG. 4) that perform prediction using the pixel at the corresponding position is limited, and the reference pixel at the TR position is limited. In addition, the use of modes 2 to 50 (vertical mode) in which prediction is performed using reference pixels at positions T, TL, L, and BL may be allowed (this example is limited to the directional mode).

As another example, an allowable prediction mode may not exist when reference pixels of all positions are unavailable. In this case, a prediction block may be generated using an arbitrary pixel value as in some configurations of the A setting, and a prediction mode (for example, a DC mode) is set for reference in the prediction mode encoding / decoding process of the next block. Etc.) may be set as the prediction mode of the corresponding block. That is, the prediction mode encoding / decoding process may be implicitly omitted.

The above example may be a case associated with a prediction mode encoding / decoding process. The prediction mode encoder / decoder of the present invention is described below on the assumption that the A setting is supported. If the B setting is supported, some components of the prediction mode encoder / decoder may be changed. Since the case where the reference pixels at all positions are not available has already been described above, the case where the reference pixels at some positions are not available will be described later.

For example, suppose that the MPM candidate group includes the prediction mode of the neighboring block. When the prediction mode of the neighboring block is a prediction mode using a reference pixel of a block position that is not available in the current block, a process of excluding the corresponding mode from the MPM candidate group may be added. That is, the unavailable mode check process may be added to the redundancy check process to be described later in the prediction mode encoder / decoder. Here, the unusable mode may be specified by various definitions, but in the present example, it is assumed that the unusable mode is a prediction mode using a reference pixel of an unusable block position. Therefore, the process of constructing the MPM candidate group in the prediction mode according to the priority may be performed, and whether to include it in the MPM candidate group through the redundancy check process and / or the unavailable mode check process may be determined. If the prediction mode of the corresponding rank does not pass the checking process, the prediction mode of the next priority may be a candidate for the MPM candidate group formation process.

Or, suppose that when reference pixels of the TR, T, and TL positions of FIG. 9 are not available, use of Nos. 19 to 66 for performing prediction using the pixels of the corresponding positions is limited. In this case, modes 2 to 18 in which prediction is performed using reference pixels at L and BL positions may be available prediction modes. In this case, assuming that the number of MPM candidate groups is 6, the non-MPM candidate group may be used. The number included in may be twelve. At this time, maintaining the number of MPM candidate groups to 6 (larger number than the total mode) may be inefficient, such as adjusting the number of MPM candidate groups when the use of the prediction mode is restricted due to the unavailable reference pixels. Changes in entropy portion / decoding settings such as p-> q. p> q) and binarization (e.g., variable length binarization A-> variable length binarization B, etc.) can also occur. That is, it may be a situation in which adaptive prediction mode encoding / decoding is supported, and a detailed description thereof will be omitted.

In addition, since prediction modes in which use is restricted by reference pixels that are not available to non-MPM candidate groups are not likely to occur, it may not be necessary to include the corresponding modes in the candidate groups. This can be achieved by entropy min / decoding settings such as adjusting the number of non-MPM candidates (e.g. s-> t. S> t) and binarization (e.g., fixed length binarization-> variable length binarization, etc.). It may be a situation in which adaptive prediction mode encoding / decoding which can be changed, such as a candidate group, is supported.

Various examples of processing when the reference pixel is not available have been described through the above examples. This may be the case in the case of the color copy mode as well as the general prediction mode.

Next, when the color copy mode is supported, classification is performed into usable reference pixels and unusable reference pixels based on the availability of reference pixels, and examples of various processes thereof will be described.

11 is an exemplary diagram for describing a possibility of using a reference pixel in a color copy mode according to an embodiment of the present invention. 11 illustrates a case in which left and upper blocks of a block (another color space) corresponding to a current block (current color space) mean an area used for acquiring correlation information, and in some color formats (YCbCr 4: 4: 4). Assume

It has already been mentioned that the reference pixel availability can be determined based on the position of the current block (eg, located outside the picture boundary). 11 illustrates various examples of the possibility of using a reference pixel that may have based on a position of a current block.

In FIG. 11, the possibility of using the reference pixel of the current block means the same case as the corresponding block (result). However, it is assumed that the case is divided into the same division unit (tile, slice, etc.) as the current color space in another color space (but it is necessary to consider the component ratio of the color format).

FIG. 11A illustrates a case in which all of the reference pixels are usable. FIG. 11B illustrates a case in which some of the reference pixels are available (the upper and left blocks, respectively). In FIG. 11D, all reference pixels are unavailable. It means the case.

Except for a of FIG. 11, at least one reference pixel belongs to a case where it is not available, and thus processing for this is necessary. According to the reference pixel processing A setting, a process of filling the unusable region through the reference pixel generation process may be performed. In this case, the reference pixel of the current block may be processed through the aforementioned reference pixel generation process (general intra prediction).

The reference pixel of the corresponding block may be the same as or different from the current block. For example, when the reference pixel at the L2 position (see FIG. 10) among the reference pixels of the current block is unavailable, the reference pixel may be generated through an arbitrary pixel value or through an available reference pixel. Specifically, the usable reference pixel may be located in the left / right or up / down direction (L1, L3, etc. in this example) of the unavailable reference pixel, or in the same reference pixel line (R3, etc. in this example). Can be located.

On the other hand, the reference pixel of the corresponding block may also be generated through an arbitrary pixel value or through an available reference pixel. However, the position of the available reference pixel may be the same as or different from the current block. In detail, the usable reference pixel may be located in various directions such as left, right, top, bottom, right and bottom as well as left, right, up, and down directions of the unavailable reference pixel. The pixels of a to p in FIG. 10 do not belong to usable reference pixels because the encoding / decoding is not yet completed in the case of the current block, but the pixels of aa to pp in FIG. It may also belong to the available reference pixel. Therefore, various methods such as interpolation, extrapolation, copying, and filtering of usable reference pixels may be used to generate reference pixels at unusable positions.

Through the above process, reference pixels of unusable positions such as b to d of FIG. 11 may be generated and included in the reference pixel memory, and reference pixels of corresponding positions may be used to obtain correlation information as shown in FIG. have.

The following is a description of the case where the reference pixel processing B setting is followed. The use of the reference pixel at an unusable position can be restricted. In addition, the use of a prediction mode in which prediction is performed from a reference pixel at an unusable position may be restricted (adapted by adaptive prediction mode decoding / decoding) or other processing may be possible.

First, a case of limiting the use of the reference pixel at an unusable position will be described. As shown in a of FIG. 11, each of the left and top blocks of the block corresponding to the current block should be used to obtain correlation information. However, b and c of FIG. 11 correspond to cases in which some reference pixels are unavailable. In this case, the correlation information may be obtained by using the reference pixel that is not used and the usable reference pixel. On the other hand, consideration may be necessary when there is not enough data for obtaining correlation information.

For example, when the left (N) and the upper (M) blocks of the current block (M x N) are the correlation information acquisition region, there are k or more reference pixels available (k is greater than 0 and less than <M + N>). When / is exceeded, the reference pixel may be used in the process of acquiring correlation information. If the number of usable reference pixels is less than / less than k, the reference pixels may not be used in the correlation information acquisition process.

Alternatively, when there are at least p reference pixels (p is greater than 0 or less N) and / or more than the number of reference pixels available in the upper block and q or more (q is greater than or equal to M and less than 0), the corresponding reference is used. The pixel may be used in the process of obtaining correlation information. If the number of reference pixels usable in the left block is less than or equal to p or the number of reference pixels usable in the upper block is less than or equal to q, the corresponding reference pixels cannot be used in the correlation information acquisition process.

The former may be a classification according to the boundary value condition in the entire region for obtaining correlation information, and the latter may be a classification according to the boundary value condition in the partial (partial) region for obtaining correlation information. In the latter case, the adjacent region for acquiring correlation information may be classified into left, top, top left, top right, and bottom left positions, but may be classified based on the classification of various adjacent regions (for example, left and top). In this case, the upper + right block may be classified as upper ^* , and the left + lower block may be classified as left ^* .

In the latter case, the boundary value setting may be the same or different according to each area. For example, in the left block, when all the reference pixels (N) are available, the reference pixel may be used in the correlation information acquisition process, and in the upper block, when at least one or more reference pixels are available (that is, some reference pixels) Is used), the corresponding reference pixel may be used in the correlation information acquisition process.

In addition, the above example may be the case that the same or different settings are supported according to the color copy mode (for example, when a, b, c, etc. of FIG. 8 is supported). Related settings may be defined differently according to other encoding / decoding settings (eg, image type, block size, shape, position, block division type, etc.).

If the correlation information acquisition process is not performed (that is, even one reference pixel is not used in the correlation information acquisition process), the correlation information may be implicitly obtained. For example, in the color copy mode equation, a and b may be set to 1 and 0 (that is, the data of the corresponding block may be used as the prediction value of the current block as it is). Alternatively, in the color copy mode, correlation information or preset correlation information of a block in which encoding / decoding is completed may be used.

Alternatively, the predicted value of the current block may be filled with an arbitrary value (eg, a minimum value, a median value, a maximum value, etc. of a bit depth or a pixel value range). That is, the method may be similar to the method performed when all the reference pixels are not available in the general intra prediction.

When correlation information such as the above example is obtained implicitly, or when the prediction value is filled with an arbitrary value, it may be an example applicable to the case of d of FIG. 11. That is, even one reference pixel is not used in the correlation information acquisition process.

The following describes a case of restricting the use of the prediction mode in which prediction is performed from a reference pixel at an unusable position.

12 is an exemplary diagram for describing a possibility of using a reference pixel in a color copy mode according to an embodiment of the present invention. In the following example, it is assumed that left, top, right and bottom left blocks of a block corresponding to the current block are areas used for obtaining correlation information. In addition, it is assumed that three color copy modes (mode_A, mode_B, mode_C) are supported, and each mode is divided into obtaining correlation information in the left + top, top + top, and left + bottom blocks of each block.

Referring to FIG. 12, FIG. 12A illustrates a case in which a current block position is located inside an image (picture, slice, tile, etc.), and FIG. 12B illustrates a case in which the current block position is positioned on an image left boundary. 12c illustrates a case in which the position of the current block is located at the upper boundary of the image, and d in FIG. 12 illustrates a case in which the position of the current block is located at the upper left boundary of the image. That is, it is assumed that the availability of the reference pixel is determined based on the position of the current block.

Referring to FIG. 12, mode_A, mode_B, and mode_C may be supported in FIG. 12A, mode_B may be supported in FIG. 12B, mode_C may be supported in FIG. 12C, and d of FIG. 12. No mode can be supported in. That is, when at least one reference pixel used for acquiring correlation information is unavailable, the corresponding mode may not be supported.

For example, when configuring the intra prediction mode candidate group for the chrominance component, the directional and non-directional modes, the color mode, and the color copy mode may be configured. In this case, assume that the candidate group includes a total of seven prediction modes including four prediction modes, one color mode, and three color copy modes such as DC, planar, vertical, horizontal, and diagonal modes.

As shown in the above example, the color copy mode in which the correlation information is obtained from the unavailable reference pixel may be excluded. In the situation of FIG. 12C, the candidate group for the color difference component may be configured with the remaining six prediction modes except mode_c. have. That is, it may be possible to adjust the number of m prediction mode candidate groups to n (m> n. N is an integer greater than or equal to 1). This may also be required to change the entropy encoding / decoding such as prediction mode index setting and binarization.

In the situation of d of FIG. 12, mode_A, mode_B, and mode_C may be excluded, and a total of four candidate groups may be configured in the remaining prediction modes, directional and non-directional modes and color modes.

As described above, it may be possible to configure an intra prediction mode candidate group by applying a prediction mode usage restriction setting based on an unavailable reference pixel.

As described above, when the reference pixel processing B is set, various processing methods may be supported. Depending on the encoding / decoding, reference pixel processing and intra prediction may be performed based on one processing method either implicitly or explicitly.

The above example has described a case where it is determined that the availability of reference pixels in adjacent areas is determined based on the position of the current block. That is, the block corresponding to the current block has the same position in the image (picture, slice, tile, maximum coding block, etc.), and thus, if a specific block (current block or corresponding block) is adjacent to the image boundary, the corresponding block There is also a relationship such as being located at the image boundary. Therefore, the same result was obtained when the possibility of using the reference pixel was determined based on the position of each block.

In addition, the above-described intra-picture prediction and the like have been described as a reference for determining the possibility of the reference pixel. This may result in the possibility that the reference pixel possibilities of adjacent areas of the block corresponding to the current block are not equal.

13 is an exemplary diagram for describing a possibility of using a reference pixel in a color copy mode according to an embodiment of the present invention. In the following example, it is assumed that left and top blocks of a block corresponding to the current block are regions used for obtaining correlation information. That is, it may be a description of the mode_A of FIG. 12, and it is assumed that the following descriptions may be similarly or similarly applied in other color copy modes.

Referring to FIG. 13, a case in which all adjacent areas of both blocks (the block corresponding to the current block) is available (i), only a part of them (ii), and all of them (iii) are unavailable. Here, the case of using only a part (ii) may be divided into a case where there is a region that can be used jointly in both blocks (ii-1) and a case where there is no (ii-2). Here, if there is a jointly available area (ii-1), it can be classified into a case where the corresponding area is perfectly matched in both blocks (ii-1-1) and a case where only a part (ii-1-2) is matched. have.

Referring to FIG. 13, i and iii of the above classification correspond to a and f of FIG. 13, ii-2 corresponds to d and e of FIG. 13, and ii-1-1 corresponds to c of FIG. 13. , ii-1-2 corresponds to b of FIG. 13. Here, ii-2 and ii-1-2 may be a new situation to be compared with the case where the reference pixel possibility is determined based on the position of the current block (or corresponding block).

The process may be a component included in the determining of the possibility of the reference pixel in the color copy mode. According to the determination result, the intra prediction setting of the color copy mode including the processing related to the reference pixel may be determined. Next, an example of reference pixel processing and intra prediction based on the availability of each adjacent region of a block corresponding to the current block will be described.

In the case of setting the reference pixel processing A, the reference pixels at positions which are not usable may be filled through the aforementioned various methods. However, the detailed setting may be different according to the classification according to the availability of the adjacent areas of the two blocks. The following description will focus on the case of ii-2 and ii-1-2, and descriptions of other classifications may be overlapped with the above description of the present invention, and thus detailed descriptions thereof will be omitted.

For example, d (ii-2) of FIG. 13 may be a case in which there is no jointly available area in the block corresponding to the current block. In addition, b (ii-1-2) of FIG. 13 may be a case where some adjacent areas available in the block corresponding to the current block overlap. In this case, the data may be used in the process of acquiring correlation information through a process of filling data of an adjacent area of an unusable location (for example, filling from an available area).

Alternatively, in FIG. 13E (ii-2), one of the current block or the corresponding block may be a case where all adjacent areas are unavailable. That is, one of the two blocks is when there is no data to be used for the correlation information acquisition process, and may be filled with various data in the corresponding area.

Here, the corresponding area may be filled using any value, and a minimum value, a median value, or a maximum value based on a pixel value range (or bit depth) of an image may be used.

Herein, the corresponding area may be filled by a method such as copying in an adjacent adjacent area of another color space. In this case, since adjacent areas of both blocks have the same image characteristic (that is, the same data) through the above process, predetermined correlation information may be obtained. For example, in a correlation-related formula, a corresponds to 1 and b corresponds to 0 so that data of the corresponding block is copied as the predicted value of the current block as it is, and various other correlation information may be set.

In the case of setting the reference pixel processing B, it is possible to limit the use of the reference pixel at a location that cannot be used or the use of the corresponding color copy mode through the aforementioned various methods. However, the detailed setting may be changed according to the classification of the availability. The following description will focus on the cases of ii-2 and ii-1-2, and descriptions of other classifications may be duplicated with the above description of the present invention, and thus detailed descriptions thereof will be omitted.

For example, d and e (ii-2) of FIG. 13 may be a case in which there is no jointly available area in the block corresponding to the current block. Since there is no comparable overlapping area for obtaining correlation information in both blocks, the use of the corresponding color copy mode can be restricted. Alternatively, the prediction value of the current block may be filled with an arbitrary value or the like. That is, it may mean that the correlation information acquisition process is not performed.

Alternatively, b (ii-1-2) of FIG. 13 may be a case where some adjacent areas available in the block corresponding to the current block overlap. Therefore, the correlation information acquisition process may be performed even if only limited areas are available.

14 is a flowchart illustrating an intra prediction method of a color copy mode according to an embodiment of the present invention. Referring to FIG. 14, a reference pixel region designated for obtaining correlation information may be checked (S1400). In operation S1410, a process setting regarding the reference pixel may be determined based on the determination of the availability of the designated reference pixel area. Intra-prediction may be performed according to the processing setting for the determined reference pixel (S1420). In this case, the correlation block may be generated based on data of a reference pixel region in which availability is available according to the reference pixel processing setting, and a prediction block according to a color copy mode may be generated, or a prediction block filled with an arbitrary value may be generated. It may be.

In summary, when the color copy mode is supported for the intra prediction of the color difference component, the comparison region for acquiring correlation information designated by the color copy mode may be identified. Unlike the general intra prediction mode, the color copy mode may check not only an adjacent region (particularly, an area used for comparing correlation information) of the current block but also the possibility of using the reference pixel of the corresponding block. Reference pixel processing and intra prediction may be performed according to the aforementioned various examples according to one of a preset reference pixel processing setting or a plurality of reference pixel processing settings and the availability of the reference pixel.

Here, the reference pixel processing setting is compared to the image type, color component, block size / position / shape, block width / vertical length ratio, encoding mode, and intra prediction mode (eg, obtaining correlation information of color copy mode). Range, location, number of pixels, etc.), limited intra prediction prediction settings, or the like, or related information may be explicitly generated in units of a sequence, a picture, a slice, a tile, and the like. In this case, the reference pixel processing setting may be defined limited to the state information of the current block (or the current image), or limited to the state information of the corresponding block (or another color image), or may be defined by combining a plurality of state information. It may be.

Although the reference pixel processing settings A and B have been separately described through the above examples, the two settings may be used alone or in combination, and may also be determined based on the state information or explicit information.

After completing the configuration of the reference pixel in the reference pixel interpolator, the reference pixel may be generated in a decimal unit through linear interpolation of the reference pixel. Alternatively, the reference pixel interpolation process may be performed after the reference pixel filter process described later.

At this time, the horizontal, vertical, some diagonal mode (e.g., a mode of 45 degree difference in the vertical and horizontal, such as Diagonal up right, Diagonal down right, Diagonal down left. Correspondence), non-directional mode, color copy mode, etc., do not perform interpolation process, and other modes (diagonal mode) may perform interpolation process.

The pixel position at which interpolation is performed (ie, which fractional units are interpolated) depending on the prediction mode (e.g. directionality of the prediction mode, dy / dx, etc.) and the position of the reference pixel and the prediction pixel. And the like) can be determined. In this case, one filter (for example, the equation used to determine the filter coefficient or the length of the filter tap is assumed to be the same filter regardless of the precision of the decimal unit, except that the precision of the decimal unit is <1/2, 7/32, 19/32> can be applied, or a filter in which the equations used to determine the length of the filter coefficients or filter taps can be applied. Hypothesis) can be selected and applied according to the decimal unit.

In the former case, an integer pixel may be used as an input for interpolation of fractional pixels, and in the latter case, an input pixel may be changed in stages (for example, an integer pixel in a 1/2 unit). In the case of the / 4 unit, an integer and a 1/2 unit pixel, etc.) may be used, but the present invention is not limited thereto.

Fixed or adaptive filtering may be performed for reference pixel interpolation, which may include sub / decoding settings (e.g., image type, color component, block position / size / shape, block aspect ratio, prediction). Mode or one or more combinations).

Fixed filtering may perform reference pixel interpolation using one filter, and adaptive filtering may perform reference pixel interpolation using one of a plurality of filters.

In this case, in the case of adaptive filtering, one of the plurality of filters may be implicitly determined or explicitly determined according to the encoding / decoding setting. At this time, the type of filter may be composed of a 4-tap DCT-IF filter, a 4-tap cubic filter, a 4-tap Gaussian filter, a 6-tap winner filter, and an 8-tap Kalman filter. It may also be possible for the filter candidate group to be defined differently (eg, some types of filters are the same or different, and short or long filter taps, etc.).

In the reference pixel filter unit, filtering may be performed on the reference pixel for the purpose of improving the accuracy of prediction by reducing the deterioration remaining through the decoding / decoding process. The filter used may be a low-pass filter. However, the present invention is not limited thereto. Whether or not filtering is applied may be determined according to the encoding / decoding setting (derived from the above description). In addition, when filtering is applied, fixed filtering or adaptive filtering may be applied.

Fixed filtering means that reference pixel filtering is not performed or reference pixel filtering is applied using one filter. Adaptive filtering means that filtering is applied according to the encoding / decoding setting, and if more than one filter type is supported, one of them may be selected.

At this time, the type of the filter is various filter coefficients such as 3-tap filter such as [1, 2, 1] / 4, 5-tap filter such as [2, 3, 6, 3, 2] / 16, and filter tap length. A plurality of filters separated by the like may be supported.

The reference pixel interpolator and the reference pixel filter introduced in the reference pixel configuration step may be necessary to improve the accuracy of prediction. The two processes may be independently performed, but a combination of the two processes (ie, one filtering) may be possible.

The prediction block generator may generate a prediction block according to at least one prediction mode, and use a reference pixel based on the prediction mode. In this case, the reference pixel may be used in a method such as extrapolation (directional mode) according to a prediction mode, and may be used in a method such as interpolation or average (DC) or copy (non-directional mode). have.

The following describes the reference pixel used according to the prediction mode.

For directional mode, the mode between horizontal and some diagonal up (modes 2 to 17 in FIG. 4) is the lower left + left block (Ref_BL, Ref_L in FIG. 10), and the horizontal mode is the left block, horizontal and The mode between the verticals (modes 19 to 49 in FIG. 4) is the left + left + up block (Ref_L, Ref_TL, Ref_T in FIG. 10), the vertical mode is the upper block (Ref_L in FIG. 10), the vertical and some diagonal modes. In the mode between (Diagonal down left) (modes 51 to 66 in FIG. 4), reference pixels of the upper and upper right blocks (Ref_T and Ref_TR in FIG. 10) may be used.

In addition, in the non-directional mode, reference pixels positioned in one or more blocks among blocks of the lower left, left, upper left, upper, and upper right blocks (Ref_BL, Ref_L, Ref_TL, Ref_T, and Ref_TR in FIG. 10) may be used. For example, it can be used for intra prediction in the form of various reference pixel combinations such as left, top, left + top, left + top + top left, top + top + top left + top right + bottom, which is used in non-directional mode (DC , Planar, etc.). In the example to be described later, it is assumed that the DC mode is a left + up block, and the planar mode is a left + up + lower left + right block as a reference pixel.

In addition, in the color copy mode, a reconstructed block (Ref_C in FIG. 10) of another color space may be used as a reference pixel. In the following example, a case in which a block corresponding to the current block is used for prediction as a reference pixel will be described.

Here, the reference pixels used for intra prediction may be divided into a plurality of concepts (or units). For example, reference pixels used for intra prediction may be classified into one or more categories, such as a first reference pixel and a second reference pixel. For convenience of description, the first reference pixel and the second reference pixel will be described separately, but it may be understood that other additional reference pixels are also supported.

Here, the first reference pixel may be a pixel used directly to generate the predicted value of the current block, and the second reference pixel may be a pixel used indirectly to generate the predicted value of the current block. Alternatively, the first reference pixel may be a pixel used to generate predicted values of all pixels in the current block, and the second reference pixel may be a pixel used to generate predicted values of some pixels in the current block. Alternatively, the first reference pixel may be a pixel used to generate the first prediction value of the current block, and the second reference pixel may be a pixel used to generate the second prediction value of the current block. Alternatively, the first reference pixel may be a pixel located based on a start point (or origin) of the prediction direction of the current block, and the second reference pixel may be a pixel located regardless of the prediction direction of the current block.

As described above, performing the prediction using the second reference pixel may be referred to as a prediction block (or correction) process. That is, the prediction block generator may be configured to add a prediction block corrector.

In this case, the reference pixels used in the prediction block generator and the prediction block corrector are not limited to the first reference pixel and the second reference pixel in each configuration. That is, the prediction block generator may perform prediction using the first reference pixel or the first reference pixel and the second reference pixel, and the prediction block corrector may use the second reference pixel or the first reference pixel. Prediction (or correction) may be performed using the second reference pixel. In the present invention, it is necessary to understand that the description is divided into a plurality of pixel concepts for convenience of description.

In the intra prediction of the present invention, it may be possible not only to perform the prediction using the first reference pixel but also to perform the prediction using the second reference pixel (i.e., perform the correction). It may be determined according to the decoding setting. First, information about whether the second reference pixel is supported (that is, whether the prediction value correction is supported) may be generated in units of a sequence, a picture, a slice, a tile, and the like. And even if it is determined that the second reference pixel is explicitly or implicitly supported, whether to support all blocks or some blocks, detailed settings related to the second reference pixel in the supported block (refer to the following description). ) May be based on encoding / decoding settings defined according to image type, color component, block size / shape / position, block width / length length ratio, encoding mode, intra picture prediction mode, limited intra picture prediction setting, and the like. . Alternatively, the relevant setting information may be explicitly determined in units of a sequence, a picture, a slice, a tile, and the like.

Here, the number of the first reference pixel and the second reference pixel used for the prediction value of one pixel may be m and n, respectively, where m and n are (1: 1), (1: 1 or more), (2 or more) : 1), (2 or more: 2 or more) may have a pixel number ratio, which may be determined according to the prediction mode, the size / shape / position of the current block, the pixel position, and the like. That is, m may be an integer of 1 or more, such as 1, 2, 3, and the like, and n may be an integer of 1 or more, such as 1, 2, 3, 4, 5, 8, or the like.

When weights applied to the first reference pixel and the second reference pixel (assuming that one pixel is used in this example) are p and q, p may be greater than or equal to q, and p has a positive value. q may have a positive or negative value.

Next, a case in which a prediction block is generated using a first reference pixel and a case in which prediction is performed by using a second reference pixel together will be described.

For example, in the Diagonal up right direction mode (see FIG. 10), prediction may be performed using a <Ref_BL + Ref_L> block or a <Ref_BL + Ref_L + Ref_T + Ref_TR> block. In addition, the horizontal mode may perform prediction using a Ref_L block or a <Ref_L + Ref_T + Ref_TL> block. In addition, the Diagonal down right direction mode may perform prediction using a <Ref_TL + Ref_T + Ref_L> block or a <Ref_TL + Ref_T + Ref_L + Ref_TR + Ref_BL> block. In addition, the vertical mode may perform prediction using a Ref_T block or a <Ref_T + Ref_L + Ref_TL> block. In addition, the diagonal down left direction mode may perform prediction using a <Ref_TR + Ref_T> block or a <Ref_TR + Ref_T + Ref_L + Ref_BL> block.

As another example, the DC mode may perform prediction using a <Ref_T + Ref_L> block or a <Ref_T + Ref_L + Ref_TL + Ref_TR + Ref_BL> block. In addition, the planar mode may perform prediction using a <Ref_T + Ref_L + Ref_TR + Ref_BL> block or a <Ref_T + Ref_L + Ref_TR + Ref_BL + Ref_TL> block.

In addition, the color copy mode may perform prediction using a Ref_C block or a <Ref_C + (Ref_T or Ref_L or Ref_TL or Ref_TR or Ref_BL)> block. Alternatively, prediction may be performed using a block <Ref_C + (Def_T or Def_B or Def_L or Def_R or Def_TL or Def_TR or Def_BL or Def_BR)>. Here, Def is a term used to refer to an adjacent block of Ref_C (the block corresponding to the current block) although not shown in FIG. 10, and Def_T to Def_BR are the top, bottom, left, right, top, left, top, bottom, and bottom of Ref_F. It may be a block adjacent to the lower right direction. That is, in the color copy mode, prediction may be performed using a reference pixel adjacent to the current block or a reference pixel adjacent to Ref_C (corresponding block) in the Ref_C block.

The above example illustrates some examples of performing prediction using the first reference pixel or performing the prediction using the first reference pixel and the second reference pixel, and various modifications may be possible.

The generation or correction of the prediction block using the plurality of reference pixels may be performed for the purpose of compensating for the disadvantages of the existing prediction mode.

For example, the directional mode may be a mode supported for the purpose of improving the accuracy of assuming that there is an edge in a specific direction of the current block, but only a reference pixel located at the start of the prediction direction may change the block in the block. It may not reflect accurately, which may reduce the accuracy of the prediction. Or, in the color copy mode, the prediction is performed by reflecting correlation information from another color image at the same time, but the degradation remaining at the block boundary in the other color image may be reflected, thereby reducing the accuracy of the prediction. . In order to solve the problem, the second reference pixel may be additionally used to increase the accuracy of prediction.

Next, a case of performing prediction using a plurality of reference pixels in the color copy mode will be described. Portions not described in the drawings of the following examples may be derived with reference to FIG. 10. Here, it is assumed that the process of acquiring correlation information of the color copy mode may be performed in a previous process or in a later process, and the following description focuses on a conceptual part. In addition, except for obtaining prediction values from other color spaces, the intra prediction in the color copy mode described later may be equally or similarly applied to other intra prediction modes.

15 is an exemplary diagram for describing prediction in color copy mode according to an embodiment of the present invention. Referring to FIG. 15, a prediction block may be generated using reference pixels of a corresponding block in the color copy mode.

In detail, correlation information may be obtained from an adjacent region (designated by the color copy mode) of each block corresponding to the current block (p1). The prediction block pred_t may be obtained by obtaining data from the corresponding block (p2) and applying the previously obtained correlation information (p3). This may be compensated (p4) by the prediction block pred_f of the current block.

The above example may lead to a related description in the color copy mode described above, and thus the detailed description thereof will be omitted.

16 is an exemplary diagram for describing prediction in color copy mode according to an embodiment of the present invention. Referring to FIG. 16, a prediction block may be generated and corrected by using a corresponding block and an adjacent reference pixel in a color copy mode.

In detail, correlation information may be obtained from an adjacent region of each block corresponding to the current block (p1). The correction may be performed on the data of the corresponding block. In this case, the correction may be defined within the corresponding block (d5) or may be limited to a block boundary adjacent to the corresponding block (d1 to d9 except d5) or may be performed across the corresponding block inner and outer boundaries (d1). To d9). That is, data of the adjacent area of the corresponding block and the corresponding block may be used for the correction. In this case, the outer boundary on which the correction is performed may be one or more (including all of them) of the up, down, left, right, top, left, top, bottom, and bottom directions (d1 to d9, except d5, respectively).

In addition, the prediction block pred_t obtained by applying the previously obtained correlation information (p3) may be obtained by acquiring data (p2) through a correction process of the corresponding block. This may be compensated (p4) by the prediction block pred_f of the current block.

17 is an exemplary diagram for explaining prediction in color copy mode according to an embodiment of the present invention. Referring to FIG. 17, a prediction block may be generated and corrected by using a corresponding block and a neighboring reference pixel of a current block in a color copy mode.

In detail, correlation information may be obtained from an adjacent region of each block corresponding to the current block (p1). The prediction block pred_t may be obtained by obtaining data from the corresponding block (p2) and applying the previously obtained correlation information (p3). This may be compensated (p4) by the first prediction block of the current block, and correction may be performed on the first prediction block (called a prediction block). In this case, the correction may be limited (c5) inside the prediction block, or may be limited to a block boundary adjacent to the prediction block (c1 to c6 except c5) or may be performed across the prediction block internal and outer boundaries (c1 to c6). can do. That is, the prediction block (that is, data based on the corresponding block. Here, the representation of the base line is because correlation information or the like has been applied) and the adjacent data of the prediction block (data of the adjacent region of the current block) may be used for the correction. In this case, the outer boundary on which the correction is performed may be one or more (including all) of the upper, left, upper left, upper right and lower left directions (except c1 to c6, except c5).

The data obtained through the correction process of the prediction block may be compensated (p5) by the (second or final) prediction block pred_f of the current block.

18 is a flowchart of a process of performing correction in a color copy mode according to an embodiment of the present invention. In detail, it illustrates a process of selecting one of the correction processes described with reference to FIGS. 16 and 17 to perform correction.

Referring to FIG. 18, correlation information may be obtained from an adjacent area of a block corresponding to the current block (S1800). In operation S1800, data of a corresponding block may be acquired. The following description assumes the case where it is determined implicitly or explicitly that the correction of the prediction block is performed. It may be determined whether the prediction block correction is performed in the current color space or the prediction block correction in another color space (S1820).

If the prediction block is corrected in the current color space, the prediction block may be generated by applying correlation information to the data of the corresponding block (S1830). This may be the same as the process of the general color copy mode. In operation S1840, the prediction block may be corrected using an adjacent region of the current block. In this case, the acquired prediction block internal data may be used as well as an adjacent region of the current block.

If the prediction block is corrected in another color space, the corresponding block may be corrected using an adjacent region of the corresponding block (S1850). In this case, not only an adjacent area of the corresponding block but also corresponding block internal data may be used. In operation S1860, the prediction block may be generated by applying the correlation information to the corrected corresponding block.

Data obtained through the above process may be compensated with a prediction block of the current block (S1870).

The application examples of FIGS. 16 and 17 are not incompatible due to the division according to the color space in the above example. That is, it may be possible that the application examples of FIGS. 16 and 17 are mixed. For example, the prediction block of the current block may be obtained through a correction process of a corresponding block as shown in FIG. 16, and the final prediction block may be obtained through the correction process of the obtained prediction block as shown in FIG. 17.

Performing the correction in the intra prediction according to the present invention may mean applying filtering to a pixel to be corrected and other pixels (adjacent pixels in this example). In this case, the filtering may be based on one preset filtering setting or by selecting one of the plurality of filtering settings and depending on the corresponding filtering setting. Here, the filtering setting may be content included in the above-described detailed setting related to the second reference pixel.

The filtering setting includes whether to apply filtering, and may include a type of filter, a coefficient of a filter, a pixel position used for the filter, and the like. The unit to which the filtering setting is applied may be a block or a pixel unit. In this case, the image type, color component, color format (that is, the ratio between components), encoding mode (Intra / Inter), block size / shape / position, block width / vertical length ratio, pixel position in the block, screen The encoding / decoding settings may be defined according to the prediction mode, the limited intra prediction, and may be involved in the filtering setting. Here, the block is assumed to be a current block, but may be understood as a concept including an adjacent block of the current block or an adjacent block of a corresponding block in the color copy mode. That is, the state information of the current block and other blocks can act as an input variable in the filtering configuration. In addition, information about a filtering setting may be explicitly included in units of a sequence, a picture, a slice, a tile, a block, and the like.

The following describes the types of filters in the filtering configuration. Filtering may be applied (ie, one-dimensional) to adjacent pixels on one line such as horizontal, vertical, and diagonal lines around the correction target pixel. Alternatively, filtering may be applied (ie, two-dimensional) to spatially adjacent pixels in the left, right, up, down, left, top, right, bottom, and bottom directions with respect to the pixel to be corrected. That is, filtering may be applied to pixels adjacent to M x N around the correction target pixel. In the following example, it is assumed that both M and N are 3 or less, but M or N may have a value greater than 3. In general, adjacent pixels symmetric about the correction target pixel may be applied to the filtering, but an asymmetrical configuration may also be possible.

19 is an exemplary diagram for describing a type of filter applied to a correction target pixel according to an exemplary embodiment of the present invention. In detail, an example of the case where the filter application position is determined symmetrically about the correction target pixel (bold line in the drawing) is shown.

Referring to FIG. 19, a and b in FIG. 19 mean 3-tap filters in a horizontal and vertical direction. In addition, c and d of FIG. 19 refers to a 3-tap filter in a diagonal direction (an angle of inclination of -45 and +45 degrees with respect to the vertical line). Also, e and f of FIG. 19 mean a 5-tap filter having a (+) or (x) form. In addition, g of FIG. 19 means a square 9-tap filter.

As an example of the filtering application, a to g of FIG. 19 may be applied to the inside or the outside (or the boundary) of the block.

As another example, the steps e to g of FIG. 19 may be applied to the inside of the block. Alternatively, a to d of FIG. 19 may be applied to the boundary of the block. In detail, a of FIG. 19 at the left boundary of the block and b of FIG. 19 may be applied at the upper boundary of the block. 19c at the upper left boundary of the block and d of FIG. 19 may be applied at the upper left and lower left boundaries of the block.

The above example is a partial case of filter selection based on the position of the pixel to be corrected, and various applications may be possible, including but not limited to the above.

20 is an exemplary diagram for describing a type of filter applied to a correction target pixel according to an exemplary embodiment of the present invention. In detail, an example of a case where the filter application position is determined asymmetrically about the correction target pixel is shown.

Referring to FIG. 20, a and b in FIG. 20 denote 2-tap filters using only left and upper pixels, respectively, and c in FIG. 20 means 3-tap filters using left and upper pixels. D to f in FIG. 20 denote 4-tap filters in an upper left direction, an upper direction, and a left direction, respectively. G to j in FIG. 20 refer to 6-tap filters in left, right, up, down, left, and right and up and down directions, respectively.

As an example of the filtering application, a to j of FIG. 20 may be applied to the inside or the outside of the block.

As another example, g to j of FIG. 20 may be applied to the inside of the block. Alternatively, a to f of FIG. 20 may be applied to the boundary of the block. Specifically, a and f of FIG. 20 may be applied at the left boundary of the block, and b and e of FIG. 20 may be applied at the upper boundary of the block. 20 c and d may be applied at the upper left boundary of the block.

The above example is a partial case of filter selection based on the position of the pixel to be corrected, and various applications may be possible, including but not limited to the above.

The filter coefficients of FIGS. 19 and 20 may be variously set. For example, for a 2-tap filter, a weight of 1: 1 or 1: 3 ratio (where 1 and 3 are the weights of the pixels to be corrected), and for a 3-tap filter, a weight of 1: 1: 2 ratio (where 2 Is the weight of the pixel to be corrected), the weight is 1: 1: 1: 5 or 1: 1: 2: 4 for the 4-tap filter (where 4 and 5 are the weights of the pixel to be corrected), and Weight ratio of 1: 1: 1: 1: 4 ratio (where 4 is the weight of the pixel to be corrected), and 6-tap filter weight ratio of 1: 1: 1: 1: 2: 2 (where 2 is the weight to be corrected) Weight of the pixel), or 1: 1: 1: 1: 1: 1: 1: 1: 8 or 1: 1: 1: 1: 2: 2: 2: 2: 4 ratio for a 9-tap filter ( Here, 8 and 4 may be applied to the weight of the pixel to be corrected). In the above example, the pixel to which the next highest weight is applied after the weight of the pixel to be corrected may be a short distance from the pixel to be corrected (adjacent to the vertical or horizontal direction) or may be a pixel positioned at the center of the pixel in the symmetrical structure. . The above example is only limited to some cases related to weight setting, and various modification examples may be possible.

The filter used for the correction may be implicitly determined according to the encoding / decoding setting, or may be explicitly included in a unit such as a sequence, a picture, a slice, a tile, and the like. The description of defining the encoding / decoding here can be derived from the various examples described above of the present invention.

In this case, information indicating whether a plurality of filters are supported may occur in the unit. If a plurality of filters is not supported, a predetermined filter may be used. If a plurality of filters are supported, filter selection information may be additionally generated. In this case, the filter may include a filter shown in FIGS. 19 and 20 or other filters in the candidate group.

Next, it is assumed that the correction is performed in another color space as shown in FIG. 16, and for this purpose, reference is made to FIG. In the following example, it is assumed that a filter used in a corresponding block is determined through a previous process, and the corresponding filter is applied to all or most pixels of the corresponding block.

A batch filter may be supported based on state information of adjacent areas of the block to be corrected, but an adaptive filter based on the position of the pixel to be corrected may also be supported.

For example, a 5-tap filter (e of FIG. 19) may be applied to aa to pp of Ref_C around the correction target pixel. In this case, the same filter may be applied regardless of the position of the pixel to be corrected. In addition, an adjacent region of the block where correction is performed may be a case where there is no limitation in applying filtering.

Alternatively, a 5-tap filter can be applied to ff, gg, jj, and kk (that is, the inside of a block) of Ref_C centered on the pixel to be corrected, and other pixels (the boundary of the block) are based on the state of an adjacent block. Filtering can be applied. Specifically, the left block of Ref_C is outside the picture, or the encoding mode is Inter (i.e., when the limited intra picture prediction setting is enabled), so that the availability of the corresponding pixel (in the part of confirming the availability of the reference pixel of the present invention) is related. When it is determined that the description can be induced), a vertical filter (for example, b in FIG. 19) may be applied to aa, ee, ii, and mm of Ref_C. Alternatively, when the upper block of Ref_C is not available, a horizontal filter (for example, a in FIG. 19) may be applied to aa to dd of Ref_C.

A batch filter may be supported based on an intra prediction mode, but an adaptive filter based on a pixel position to be corrected may also be supported.

For example, in the case of obtaining the correlation information from the left and upper blocks in the color copy mode, the 9-tap filter (g of FIG. 19) may be applied to aa to pp of Ref_C. In this case, the same filter may be applied regardless of the position of the pixel to be corrected.

Alternatively, in the case of obtaining the correlation information from the lower left and lower blocks among the color copy modes, the 9-tap filter may be applied to ee to pp of Ref_C, and other pixels (upper boundary) are filtered based on the prediction mode setting. Can be applied. In this example, since correlation information is obtained from the left and the lower left blocks, it can be estimated that the correlation with the upper block is low. For this reason, the left and right filters (for example, i in FIG. 20) may be applied to aa to dd of Ref_C.

In the present invention, the description of the color copy mode has been described based on the case of some color formats (4: 4: 4). Depending on the color format, there may be a difference in the detailed configuration regarding the correction. This example assumes a case of performing correction in another color space.

In the color copy mode, the correlation information and the prediction value acquisition can directly obtain the relevant data from the pixel because in some color formats (4: 4: 4), one pixel in the current color space corresponds to one pixel in the other color space. can do.

On the other hand, in some color formats (4: 2: 0), one pixel in the current color space corresponds to one or more pixels (four in this example) in another color space. If a predetermined pixel is not selected to obtain related data from the corresponding pixel, a down sampling process of acquiring related data from a plurality of corresponding pixels may be required.

The following describes the prediction and correction process of the color copy mode for each color format. In the following example, some formats (4: 2: 0) assume a case where a downsampling process is performed. In addition, blocks corresponding to the current block are referred to as A and B blocks, respectively.

<1> Intra prediction in 4: 4: 4 format

<1-1> Acquisition of pixel value in adjacent area of A block

Acquisition of pixel value corresponding to pixel of <1-1> in adjacent area of <1-2> B block

<1-3> Acquisition of correlation information based on pixel values of adjacent areas in each color space

<1-4> Extracting Pixels of B Blocks and Pixels Adjacent to It

<1-5> B block correction by applying filtering to <1-4> pixels

<1-6> Acquisition of pixel value of B block (M x N) corresponding to pixel of A block (M x N)

<1-7> Prediction pixel generation by applying correlation information to pixel values of <1-6>

<2> Intra prediction in 4: 2: 0 format (1)

<2-1> Acquisition of pixel value in adjacent area of A block

Extracting the pixel corresponding to the pixel of <2-1> and the pixel adjacent thereto in the adjacent region of the <2-2> B block

<2-3> Applying downsampling to pixels of <2-2> to obtain pixel values corresponding to pixels of <2-1> in adjacent areas of B block

<2-4> Obtaining Correlation Information Based on Pixel Values of Adjacent Regions in Each Color Space

<2-5> Extracting Pixels of B Blocks and Pixels Adjacent to It

<2-6> B block correction by applying filtering to <2-5> pixels

<2-7> Extracting Pixels of B Blocks (2M x 2N) Corresponding to Pixels of A Blocks (M x N) and Pixels Adjacent thereto

<2-8> Acquisition of pixel value of B block by applying down sampling to pixels of <2-7>

<2-9> Prediction pixel generation by applying correlation information to pixel values of <2-8>

Looking at the process of <1> and <2>, <1> has one filtering in <1-5>, whereas <2> has multiple filtering in <2-6> and <2-8>. You can see that it applies. In the case of <2-6>, it may be a process for correcting data used to acquire a predictive pixel, and in the case of <2-8>, it may be a down sampling process for obtaining a predictive pixel, and a filter in each process may be configured differently. Can be. Of course, the encoding performance may be improved by each process, but a redundant filtering effect may appear. In addition, an increase in complexity due to an increase in the number of filtering may be a configuration that is not suitable for some profiles. This may require some color formats to support filtering that incorporates them.

<3> Intra-picture prediction in 4: 2: 0 format (2)

<3-1> Acquisition of pixel value in adjacent area of A block

Extracting the pixel corresponding to the pixel of <3-1> and the pixel adjacent thereto in the adjacent region of the <3-2> B block

<3-3> Applying downsampling to pixels of <3-2> to obtain pixel values corresponding to pixels of <3-1> in adjacent areas of B block

<3-4> Obtaining Correlation Information Based on Pixel Values of Adjacent Regions in Each Color Space

<3-5> Pixel Extraction of B Block and Pixels Adjacent to It

<3-6> B block correction by applying filtering to pixels of <3-5>

<3-7> Acquisition of pixel value of B block (2M x 2N) corresponding to pixel of A block (M x N)

<3-8> Prediction pixel generation by applying correlation information to pixel values of <3-7>

<4> Intra-picture prediction in 4: 2: 0 format (3)

<4-1> Acquisition of pixel value in adjacent area of A block

Extracting the pixel corresponding to the pixel of <4-1> and the pixel adjacent thereto in the adjacent region of the <4-2> B block

<4-3> Applying downsampling to pixels of <4-2> to obtain pixel values corresponding to pixels of <4-1> in adjacent areas of B block

<4-4> Obtaining Correlation Information Based on Pixel Values of Adjacent Regions in Each Color Space

<4-5> Pixels of B blocks (2M x 2N) corresponding to pixels of A block (M x N) and pixels adjacent thereto

<4-6> Acquire pixel value of B block by applying down sampling to pixels of <4-5>

<4-7> Prediction pixel generation by applying correlation information to pixel values of <4-6>

Referring to the process of <3>, the downsampling process of the B block is omitted and the correlation information is applied to one pixel at a preset position. Instead, the calibration process is performed, which can eliminate the disadvantages of the downsampling process.

On the other hand, the process of <4> is a case where the correction process is omitted and the down sampling process of the B block is performed. Instead, a filter can be used to effect the correction during downsampling. Deformation may occur to the configuration in which the correction process is included in the downsampling, but the above-described settings in the correction process may be applied as it is. That is, the filter selection information for the downsampling may be explicitly determined in an upper unit or implicitly determined according to the encoding / decoding. In addition, down sampling is performed using only the internal data of the B block, or down sampling is performed using the B block and external data in one or more directions in the left, right, up, down, top, right, left, and bottom directions together. You can decide whether to do so, either implicitly or explicitly.

The prediction mode determiner performs a process for selecting an optimal mode among a plurality of prediction mode candidate groups. In general, block distortion (eg, Distortion of current and reconstructed blocks). A mode that is optimal in terms of coding cost may be determined using a sum-of-absolute difference (SAD), a sum of square difference (SSD), etc., and a rate-distortion technique that takes into account the amount of bits generated according to a corresponding mode. The prediction block generated based on the prediction mode determined through the above process may be transmitted to the subtractor and the adder.

In order to determine the optimal prediction mode, all prediction modes present in the prediction mode candidate group may be searched, or the optimal prediction mode may be selected through other decision processes for the purpose of reducing the amount of computation / complexity. For example, the first stage selects some modes that perform well in terms of image quality deterioration for all intra prediction prediction candidates, and the second stage considers not only the image quality degradation but also the amount of bits generated for the mode selected in the first stage. The optimal prediction mode can be selected. In addition to the above method, various methods of reducing the amount of computation / complexity may be applied.

In addition, the prediction mode determiner may generally be included only in the encoder, but may also be included in the decoder according to a sub / decoding setting. For example, when a template matching is included as a prediction method or a method of deriving an in-picture prediction mode in an adjacent region of the current block. In the latter case, it can be understood that the method of implicitly obtaining the prediction mode in the decoder is used.

The prediction mode encoder may encode the prediction mode selected by the prediction mode determiner. In the prediction mode candidate group, index information corresponding to the prediction mode may be encoded, or the information about the prediction mode may be predicted. The former may be a method applied to the luminance component and the latter may be a method applied to the color difference component, but is not limited thereto.

When the prediction mode is predicted and encoded, the prediction value (or prediction information) of the prediction mode may be referred to as Most Probable Mode (MPM). The MPM may be configured in one prediction mode or in a plurality of prediction modes. The number of MPMs (k. K is an integer of 1 or more, such as 1, 2, 3, 6, etc.) according to the number of prediction mode candidates. Can be determined. When the MPM is composed of a plurality of prediction modes, it may be referred to as an MPM candidate group.

MPM is a concept supported for efficiently encoding a prediction mode. In fact, the candidate group may be configured as a prediction mode that is more likely to occur in the prediction mode of the current block.

For example, the MPM candidate group may be a preset prediction mode (or a statistically frequently occurring prediction mode such as DC, Plaanr, vertical, horizontal, and some diagonal modes), adjacent blocks (left, top, left top, right top, bottom left blocks, etc.). Can be configured as a prediction mode. In this case, the prediction modes of the adjacent blocks are obtained at L0 to L3 (left block), T0 to T3 (upper block), TL (upper left block), R0 to R3 (upper right block), and B0 to B3 (lower left block) in FIG. 10. can do.

Predefined priority (e.g., L0-L1-L2) when MPM candidates can be constructed from two or more subblock positions (e.g., L0, L2, etc.) in adjacent blocks (e.g., left block). Etc.), the prediction mode of the block may be configured in the candidate group. Alternatively, when the MPM candidate group cannot be formed from two or more subblock positions, the prediction mode of the subblock corresponding to the predefined position (eg, L0) may be configured in the candidate group. In detail, a prediction mode of positions L3, T3, TL, R0, and B0 among adjacent blocks may be selected as a prediction mode of the adjacent block and included in the MPM candidate group. The above description is in some cases configuring the prediction mode of the adjacent block in the candidate group, but is not limited thereto. In the example described below, it is assumed that a candidate mode is configured with a prediction mode of a predefined position.

In addition, when one or more prediction modes are configured with the MPM candidate group, a mode derived from one or more included prediction modes may be further configured as the MPM candidate group. In detail, when mode k (directional mode) is included in the MPM candidate group, a mode that can be derived from the mode (a mode having an interval of + a and -b on the basis of k. A and 1 are 1, 2, 3, etc.). Above integer) may be further included in the MPM candidate group.

There may be a priority for configuring the MPM candidate group, and the MPM candidate group may be configured in the order of the prediction mode of the adjacent block, the preset prediction mode, the derived prediction mode, and the like. The process of configuring the MPM candidate group may be completed by filling the maximum number of MPM candidates according to the priority. If the prediction mode coincides with the previously included prediction mode, the prediction mode may include a redundancy check process in which an order is passed to a candidate of the next priority without configuring the candidate group.

The following presupposes that the MPM candidate group consists of six prediction modes.

For example, candidate groups may be configured in the order of L-T-TL-TR-BL-Planar-DC-Vertical-Horizontal-Diagonal mode. It may be the case that a prediction mode of an adjacent block is preferentially configured in a candidate group and additional configuration of a preset prediction mode is performed.

Alternatively, candidate groups may be configured in the order of L-T-Planar-DC-<L + 1>-<L-1>-<T + 1>-<T-1>-Vertical-Horizontal-Diagonal mode. . First of all, a prediction mode of some adjacent blocks and a part of a preset prediction mode are configured first, and a mode and a part of a preset prediction mode derived based on the assumption that a prediction mode in a direction similar to that of an adjacent block will occur will be configured. It may be the case.

The above example is part of the MPM candidate group configuration, and various modifications may be possible without being limited thereto.

The MPM candidate group may be binarized such as unary binarization or truncated rice binarization based on the index in the candidate group. That is, mode bits can be represented by allocating short bits to candidates having a small index and assigning long bits to candidates having a large index.

Modes not included in the MPM candidate group may be classified as non-MPM candidate groups. In addition, the non-MPM candidate group may be classified into two or more candidate groups according to the encoding / decoding setting.

The following presupposes that there are 67 modes including a directional mode and a non-directional mode in the prediction mode candidate group, and 6 MPM candidates are supported to include 61 prediction modes in the non-MPM candidate group.

If the non-MPM candidate group consists of one, the prediction mode that is not included in the MPM candidate group configuration process is left, so no additional candidate group formation process is required. Therefore, binarization such as fixed length binarization and truncated unary binarization may be used based on indexes in the non-MPM candidate group.

On the assumption that the non-MPM candidate group is composed of two or more candidate groups, the non-MPM candidate group is classified into non-MPM_A (ideal A candidate group) and non-MPM_B (ideal B candidate group). It is assumed that a candidate group (p. MPM candidate groups or more) constitutes the candidate group in a prediction mode that is more likely to occur in the prediction mode of the current block than the B candidate group (q. Or more than the A candidate group numbers). At this time, the A candidate group configuration process may be added.

For example, some prediction modes having even intervals (eg, 2nd, 4th, 6th, etc.) among the directional modes may be configured in the candidate A group or included in the preset prediction mode (eg, the MPM candidate group). Mode derived from the predicted mode). The prediction mode remaining through the MPM candidate group configuration and the A candidate group configuration may be configured as a B candidate group, and an additional candidate group configuration process is not required. Binarization, such as fixed length binarization and truncated unary binarization, may be used based on indices in candidate A and candidate B groups.

The above example is a case in which the non-MPM candidate group is composed of two or more, but is not limited thereto, and various modification examples may be possible.

The following is a process for the case of predicting and encoding a prediction mode.

Information (mpm_flag) about whether the prediction mode of the current block matches the MPM (or some modes in the MPM candidate group) may be checked.

In case of matching with the MPM, the MPM index information (mpm_idx) may be additionally checked according to the configuration of the MPM (one or two or more). After that, the encoding process of the current block is completed.

If the MPM does not match the non-MPM candidate group consists of one, non-MPM index information (remaining_idx) can be confirmed. After that, the encoding process of the current block is completed.

If the non-MPM candidate group includes a plurality (two in this example), information (non_mpm_flag) about whether the prediction mode of the current block matches some prediction mode in the A candidate group may be checked.

If the A candidate group matches, the A candidate group index information (non_mpm_A_idx) may be checked. If the A candidate group does not match, the B candidate group index information (remaining_idx) may be checked. After that, the encoding process of the current block is completed.

When the prediction mode candidate group configuration is fixed, the same prediction number index may be used for the prediction mode supported in the current block, the prediction mode supported in the adjacent block, and the preset prediction mode.

Meanwhile, when the prediction mode candidate group configuration is adaptive, the prediction mode supported by the current block, the prediction mode supported by the adjacent block, and the preset prediction mode may use the same prediction number index or different prediction number indexes. Reference is made to FIG. 4 for the following description.

In the prediction mode encoding process, a prediction mode candidate group unification (or adjustment) process for configuring an MPM candidate group or the like may be performed. For example, the prediction mode of the current block may be one of prediction mode candidate groups of modes -5 to 61, and the prediction mode of an adjacent block may be one of prediction mode candidate groups of modes 2 to 66. In this case, since a part of the prediction mode (mode 66) of the adjacent block may be a mode that is not supported by the prediction mode of the current block, a process of unifying it in the prediction mode encoding process may be performed. That is, the process may not be required when the fixed intra prediction mode candidate group configuration is supported, or the process may be required when the adaptive intra screen prediction mode candidate group configuration is supported, and a detailed description thereof will be omitted. .

Unlike the method using the MPM, encoding may be performed by assigning an index to a prediction mode belonging to a prediction mode candidate group.

For example, when an index is assigned to a prediction mode according to a predefined priority and a prediction mode of a current block is selected, the method of encoding a corresponding index corresponds to this. This means that the prediction mode candidate group is fixed and a fixed index is assigned to the prediction mode.

Or, if the prediction mode candidate group is adaptively configured, the fixed index allocation method may not be suitable. To this end, an index may be allocated to a prediction mode according to an adaptive priority, and a method of encoding a corresponding index may be applied when the prediction mode of the current block is selected. Because of the adaptive configuration of the prediction mode candidate group, it is possible to efficiently encode the prediction mode by changing the indices assigned to the prediction modes. That is, the adaptive priority may be to assign a candidate having a high probability of being selected as the prediction mode of the current block to an index in which a short mode bit occurs.

The following presupposes that eight prediction modes including a prediction mode (directional mode and non-directional mode), a color copy mode, and a color mode preset in the prediction mode candidate group are supported (color difference component).

For example, four preset modes are supported: Planar, DC, horizontal, vertical, and diagonal modes (Diagonal down left in this example), one color mode (C), and three color copy modes (CP1, CP2). Assume that CP3) is supported. The basic order of the indexes allocated to the prediction modes may be given in a preset prediction mode-color copy mode-color mode.

In this case, the preset prediction modes such as the directional mode, the non-directional mode, and the color copy mode may be easily classified into prediction modes in which the prediction methods are divided. However, the color mode may be a directional mode or a non-directional mode, and there may be a possibility of overlapping with the preset prediction mode. For example, when the color mode is a vertical mode, a case in which the color mode overlaps with the vertical mode which is one of the preset prediction modes may occur.

In the case of adaptively adjusting the number of prediction mode candidate groups according to the encoding / decoding setting, if the overlapping case occurs, the number of candidate groups may be adjusted (8-> 7). Alternatively, in the case where the number of prediction mode candidate groups is fixed, an index may be allocated by adding and considering other candidates when the overlapping case occurs, which will be described below on the assumption of this setting. In addition, the adaptive prediction mode candidate group may be a supported configuration even when a variable mode such as a color mode is included. Therefore, when adaptive index allocation is performed, it can be regarded as an example of adaptive prediction mode candidate group configuration.

The following describes a case where adaptive index allocation is performed according to the color mode. The basic index is assumed to be allocated in the order of Planar (0)-vertical (1)-horizontal (2)-DC (3)-CP1 (4)-CP2 (5)-CP3 (6)-C (7). In addition, when the color mode does not match the preset prediction mode, it is assumed that index allocation is performed in the above order.

For example, when the color mode matches one of the preset prediction modes (Planar, vertical, horizontal, and DC modes), the prediction mode corresponding to the index 7 of the color mode is filled. The index (one of 0 to 3) of the matching prediction mode is filled with a preset prediction mode (Diagoanl down left). Specifically, when the color mode is horizontal mode, Planar (0)-Vertical (1)-Diagoanal down left (2)-DC (3)-CP1 (4)-CP2 (5)-CP3 (6)-Horizontal ( Index allocation as shown in 7) may be performed.

Alternatively, when the color mode matches one of the preset prediction modes, the prediction mode corresponding to the index 0 is filled. The preset prediction mode (Diagoanal down left) is filled in the index 7 of the color mode. In this case, if the filled prediction mode is not the existing index 0 (that is, not the planar mode), the existing index configuration may be adjusted. Specifically, when the color mode is DC mode, DC (0)-Planar (1)-Vertical (2)-Horizontal (3)-CP1 (4)-CP2 (5)-CP3 (6)-Diagonal down left ( Index allocation as shown in 7) may be performed.

The above examples are some cases related to adaptive index allocation, and various modifications may be possible without being limited thereto. In addition, binarization such as fixed length binarization, unary binarization, truncated unary binarization, truncated Rice binarization, or the like may be used based on the index within the candidate group.

Next, another example of performing encoding by assigning an index to a prediction mode belonging to the prediction mode candidate group will be described.

For example, a method of dividing into a prediction mode, a prediction method, and the like into a plurality of prediction mode candidate groups, and assigning and encoding an index to a prediction mode belonging to the candidate group corresponds thereto. In this case, candidate group selection information encoding may precede the index encoding. For example, the directional mode, the non-directional mode, and the color mode, which are prediction modes for performing prediction in the same color space, may belong to one candidate group (or more S candidate groups), and the color copy, which is a prediction mode for performing prediction in another color space. The mode may belong to one candidate group (or more than D candidate group).

The following presupposes that nine prediction modes including a prediction mode, a color copying mode, and a color mode that are preset in the prediction mode candidate group are supported (color difference component).

For example, four preset prediction modes are supported among planar, DC, horizontal, vertical, and diagonal modes, and one color mode (C) and four color copy modes (CP1, CP2, CP3, CP4) are supported. Assume the case. The S candidate group may have five candidates configured with a preset prediction mode and a color mode, and the D candidate group may have four candidates configured with the color copy mode.

The S candidate group is an example of an adaptive mode candidate group that is adaptively configured, and an example of adaptive index allocation has been described above, and thus a detailed description thereof will be omitted. Since the D candidate group is an example of a fixed prediction mode candidate group, a fixed index allocation method may be used. For example, an index allocation such as CP1 (0)-CP2 (1)-CP3 (2)-CP4 (3) may be performed.

Binarization, such as fixed length binarization, unary binarization, truncated unary binarization, truncated Rice binarization, or the like, may be used based on the index within the candidate group. In addition, various modifications may be possible without being limited to the above examples.

The prediction related information generated by the prediction mode encoder may be transmitted to the encoder and may be included in the bitstream.

In the image decoding method according to an embodiment of the present invention, the intra prediction may be configured as follows. The intra prediction of the prediction unit may include a prediction mode decoding step, a reference pixel construction step, and a prediction block generation step. Also, the image decoding apparatus may be configured to include a prediction mode decoding unit, a reference pixel construction unit, and a prediction block generation unit that implement the prediction mode decoding step, the reference pixel construction step, and the prediction block generation step. Some of the processes described above may be omitted or other processes may be added, and may be changed in an order other than the order described above.

Since the reference pixel component and the prediction block generator of the image decoding apparatus play the same role as the corresponding components of the image encoding apparatus, a detailed description thereof will be omitted, and the prediction mode decoder may be performed by using the inverse method used by the prediction mode encoder. Can be.

The methods according to the invention can be implemented in the form of program instructions that can be executed by various computer means and recorded on a computer readable medium. Computer-readable media may include, alone or in combination with the program instructions, data files, data structures, and the like. The program instructions recorded on the computer readable medium may be those specially designed and constructed for the present invention, or may be known and available to those skilled in computer software.

Examples of computer readable media may include hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions may include high-level language code that can be executed by a computer using an interpreter, as well as machine code such as produced by a compiler. The hardware device described above may be configured to operate with at least one software module to perform the operations of the present invention, and vice versa.

In addition, the above-described method or apparatus may be implemented by combining all or part of the configuration or function, or may be implemented separately.

Although described above with reference to a preferred embodiment of the present invention, those skilled in the art will be variously modified and changed within the scope of the invention without departing from the spirit and scope of the invention described in the claims below I can understand that you can.

Claims (1)

Identifying a reference pixel region designated for obtaining correlation information;
Determining a reference pixel processing setting based on the availability determination of the reference pixel area; And
And performing intra prediction on the basis of the reference pixel processing setting.

Images