KR20240015700A - Image encoding/decoding method and image decoding apparatus using motion vector precision - Google Patents

Abstract

A motion vector candidate selection method for selecting a prediction candidate from reference blocks of a reference picture including the current picture in order to derive motion information for a current block to be encoded or decoded in an image, and an image encoding and decoding method using the same are disclosed. The motion vector candidate selection method includes the steps of configuring a spatial motion vector candidate (first candidate), determining whether a reference picture of the current block exists in the current picture, and if the determination result of the determining step is yes, the current block It includes adding a spatial motion vector candidate (second candidate) from another block of the current encoded picture.

Description

Image encoding and decoding method and image decoding device using motion vector differential values {IMAGE ENCODING/DECODING METHOD AND IMAGE DECODING APPARATUS USING MOTION VECTOR PRECISION}

The present invention relates to video encoding and decoding technology, and more specifically, to a video encoding method using motion vector differences, a video decoding method, and a video decoding device.

With the spread of the Internet and mobile terminals and the development of information and communication technology, the use of multimedia data is rapidly increasing. Accordingly, the need to improve the performance and efficiency of image processing systems is increasing significantly in order to perform various services or tasks through image prediction in various systems.

Meanwhile, in existing video encoding and decoding technologies, motion information about neighboring blocks of the current block is predicted from at least one reference picture before or after the current picture according to an inter-picture prediction method, or the current picture is predicted according to an intra-picture prediction method. By obtaining motion information from my reference block, I am estimating the motion vector for the current block.

However, existing inter-screen prediction has the disadvantage of having high computational complexity because it generates prediction blocks using a temporal prediction mode between pictures, and intra-screen prediction has the disadvantage of having large coding complexity.

As such, performance improvement in image encoding or image decoding is still required in the image encoding and decoding methods of the prior art.

The purpose of the present invention to solve the above problems is to use a motion vector difference value when selecting a prediction candidate from reference blocks of a reference picture including the current picture in order to derive motion information for the current block in video encoding and decoding. The goal is to provide a video encoding and decoding method using the video encoding and decoding method.

Another object of the present invention is to provide a video encoding and decoding device using motion vector differential values.

In one aspect of the present invention for achieving the above object, an image encoding method for generating a predicted image by predicting motion information in an original image includes the steps of configuring a motion information prediction candidate group; changing the motion vector of a candidate block belonging to the candidate group to match the precision unit of the motion vector of the current block; and obtaining a difference value by subtracting the motion vector of the candidate block to which the precision unit is matched from the motion vector of the current block.

In another aspect of the present invention for achieving the above object, in an image decoding method for entropy decoding an encoded image and generating a restored image through inverse quantization and inverse transformation, the image decoding method is based on header information of the image obtained through the entropy decoding. Constructing a motion information prediction candidate group of the restored image; changing the motion vector of a candidate block belonging to the candidate group to match the precision unit of the motion vector of the current block; and obtaining a difference value by subtracting the motion vector of the candidate block to which the precision unit is aligned from the motion vector of the current block.

Here, in the changing step, the motion vector may be scaled according to the first distance between the current picture where the current block is located and the reference picture and the second distance between the picture of the candidate block and the reference picture of the candidate block. .

Here, the image encoding method may further include, after the calculating step, determining the interpolation precision of each reference picture based on the average distance of the first distance and the second distance.

Here, the changing step can be omitted when the reference picture of the current block and the reference picture of the candidate block are the same.

Here, in the changing step, the motion vector of a neighboring block or neighboring block may be changed in units of motion vector precision of the current block according to the motion vector precision of the current block.

Here, the neighboring block may be positioned between the current block and another block. Additionally, the neighboring block may be a block whose motion vector has been searched for through inter-screen prediction prior to the current block.

In another aspect of the present invention for achieving the above object, a memory for storing a program or program code for entropy decoding an encoded image and generating a restored image through inverse quantization and inverse transformation; and a processor connected to the memory and executing the program, wherein the processor configures, by the program, a motion information prediction candidate group of the reconstructed image based on header information of the image obtained through the entropy decoding, An image decoding device is provided that changes the motion vector of a candidate block belonging to a candidate group to match the precision unit of the motion vector of the current block, and obtains a difference value by subtracting the motion vector of the candidate block whose precision unit is matched from the motion vector of the current block. do.

Here, when the processor wants to change the motion vector of the candidate block, the first distance between the current picture where the current block is located and the reference picture and the second distance between the picture of the candidate block and the reference picture of the candidate block You can scale the motion vector depending on the distance.

Here, when calculating the difference value, the processor may determine the interpolation precision of each reference picture based on the average distance of the first distance and the second distance.

Here, when the processor wants to change the motion vector of the candidate block, the change process may be omitted if the reference picture of the current block and the reference picture of the candidate block are the same.

Here, when the processor wants to change the motion vector of the candidate block, it can change the motion vector of a neighboring block or neighboring block in units of the motion vector precision of the current block according to the motion vector precision of the current block.

Here, the neighboring block is located between the current block and another block, and may be a block for which a motion vector is searched for through inter-screen prediction prior to the current block.

In another aspect of the present invention for achieving the above object, there is provided an image encoding method regarding reference pixel configuration in intra-prediction, comprising: obtaining a reference pixel of the current block from a neighboring block in intra-prediction for the current block; A step of adaptively filtering a reference pixel, generating a prediction block of the current block by using the reference pixel to which adaptive filtering has been applied as an input value according to the prediction mode of the current block, and a post-processing filter adaptive to the prediction block. An image encoding method is provided, including the step of applying .

Here, in the acquiring step, the reference pixel of the current block may be acquired from the neighboring block.

Here, the acquiring step may be determined according to the availability of neighboring blocks.

Here, the availability of the neighboring block may be determined by the location of the neighboring block and/or a specific flag (constrained_intra_pred_flag). As an example, a particular flag may have a value of 1 when a neighboring block is available. It may mean that when the prediction mode of a neighboring block is inter-screen mode, the reference pixel of that block cannot be used for prediction of the current block.

Here, a specific flag (constrained_intra_pred_flag) is determined according to the prediction mode of the neighboring block, and the prediction mode may be either intra-prediction or inter-prediction.

Here, when the specific flag (constrained_intra_pred_flag) is 0, the availability of the neighboring block becomes 'true' regardless of the prediction mode of the neighboring block, and when it is 1, the availability of the neighboring block becomes 'true' if the prediction mode of the neighboring block is intra-screen prediction. ', and in case of inter-screen prediction, the availability of neighboring blocks may be 'false'.

Here, inter-screen prediction can generate a prediction block by referring to one or more reference pictures.

Here, the reference picture is managed through reference picture list 0 (List 0) and reference picture list 1 (List 1), and one or more past pictures, future pictures, and current pictures may be included in List 0 and List 1.

Here, List 0 and List1 may adaptively determine whether to include the current picture in the reference picture list.

Here, information for determining whether to include the current picture in the reference picture list may be included in a sequence, reference picture parameter set, etc.

In another aspect of the present invention for achieving the above object, there is provided an image decoding method performed on a computing device, comprising: obtaining a flag regarding reference pixel availability of a neighboring block from an input bitstream on a sequence or picture basis; determining reference pixel availability of a neighboring block when performing intra-prediction according to the flag; When the flag is 0, the reference pixel of the neighboring block is used for prediction of the current block regardless of the prediction mode of the neighboring block, and when the flag is 1, the reference pixel of the neighboring block is used if the prediction mode of the neighboring block is intra-screen prediction. An image decoding method is provided in which reference pixels of a neighboring block are used for prediction of the current block, and when the prediction mode of the neighboring block is inter-screen prediction, reference pixels of the neighboring block are not used for prediction of the current block.

Here, inter-screen prediction can generate a prediction block based on block matching in a reference picture.

Here, the reference picture can be managed through List 0 in the P picture and List 0 and List 1 in the B picture.

Here, the current picture can be included in List 0 in inter-screen prediction.

Here, the current picture can be included in List 1 in inter-screen prediction.

Here, inclusion of the current picture in List 0 and List 1 can be determined based on the flag transmitted in the sequence parameter.

Here, inclusion of the current picture in List 0 and List 1 can be determined based on the flag transmitted in the picture parameter.

When using the image encoding and decoding method and image decoding device using motion vector difference values according to the embodiment of the present invention as described above, the performance of the image prediction system can be improved.

In addition, there is an advantage in that the performance and efficiency of the image prediction device can be improved while relatively simplifying the system configuration by using the difference value of the motion vector.

In addition, according to the present invention, scaling or precision correction can be applied in various forms depending on the precision of the block or picture, and the optimal candidate is selected from among the applicable candidates, provided with a difference value with the motion vector of the current block, and encoded. There is an advantage in that encoding and decoding performance and efficiency can be improved by using it.

1 is a diagram for explaining a system using the video encoding device and/or video decoding device of the present invention.
Figure 2 is a block diagram of a video encoding device according to an embodiment of the present invention.
Figure 3 is a block diagram of an image decoding device according to an embodiment of the present invention.
Figure 4 is an example diagram showing inter-screen prediction of a P slice in an image encoding and decoding method according to an embodiment of the present invention.
Figure 5 is an example diagram showing inter-screen prediction of a B slice in an image encoding and decoding method according to an embodiment of the present invention.
Figure 6 is an example diagram to explain the case of generating a prediction block in one direction in the video encoding and decoding method according to an embodiment of the present invention.
Figure 7 is an example diagram of configuring a reference picture list in the video encoding and decoding method according to an embodiment of the present invention.
Figure 8 is an exemplary diagram showing another example of performing inter-screen prediction from a reference picture list in the video encoding and decoding method according to an embodiment of the present invention.
Figure 9 is an example diagram for explaining intra-screen prediction in the video encoding method according to an embodiment of the present invention.
Figure 10 is an example diagram for explaining the prediction principle in a P slice or B slice in the image encoding method according to an embodiment of the present invention.
FIG. 11 is an example diagram illustrating a case of performing interpolation in the image encoding method of FIG. 10.
FIG. 12 is a diagram for explaining the main process of the video encoding method according to an embodiment of the present invention using syntax in a coding unit.
Figure 13 is for illustrating an example of supporting symmetric type partitioning or asymmetric type partitioning as in inter-screen prediction when generating a prediction block through block matching in the current picture used in Figure 12. This is an example diagram.
Figure 14 is an example diagram to explain that 2Nx2N and NxN can be supported in inter-screen prediction (Inter), like intra-prediction (Intra) in Figure 9.
Figure 15 is a diagram for explaining the process of performing a horizontal 1D filter on pixels at positions a, b, and c of the image (assumed to be x) in the image encoding method according to an embodiment of the present invention.
Figure 16 is an example diagram of a current block and neighboring blocks that can be employed in an image encoding method according to an embodiment of the present invention.
FIG. 17 is a diagram for explaining a case in which motion vector precision is different for each block in FIG. 15.
Figure 18 is a diagram for explaining a case in which the motion vector precision of a block is determined according to the interpolation precision of a reference picture in the video encoding and decoding method according to an embodiment of the present invention.
Figure 19 is a flowchart of an image encoding and decoding method using motion vector difference according to an embodiment of the present invention.
Figures 20 to 25 are diagrams for explaining the motion vector difference calculation process for various cases when interpolation precision is determined on a block basis in the video encoding and decoding method according to an embodiment of the present invention.
Figures 26 to 29 are diagrams for explaining the process of expressing the precision of motion vector difference values in the video encoding and decoding method according to an embodiment of the present invention.

Since the present invention can make various changes and have various embodiments, specific embodiments will be illustrated in the drawings and described in detail. However, this is not intended to limit the present invention to specific embodiments, and should be understood to include all changes, equivalents, and substitutes included in the spirit and technical scope of the present invention.

Terms such as first, second, A, and B may be used to describe various components, but the components should not be limited by the terms. The above terms are used only for the purpose of distinguishing one component from another. For example, a first component may be named a second component, and similarly, the second component may also be named a first component without departing from the scope of the present invention. The term and/or includes any of a plurality of related stated items or a combination of a plurality of related stated items.

When a component is said to be "connected" or "connected" to another component, it is understood that it may be directly connected to or connected to the other component, but that other components may exist in between. It should be. On the other hand, when it is mentioned that a component is “directly connected” or “directly connected” to another component, it should be understood that there are no other components in between.

The terms used in this application are only used to describe specific embodiments and are not intended to limit the invention. Singular expressions include plural expressions unless the context clearly dictates otherwise. In this application, terms such as “comprises” or “has” are intended to designate the presence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, but are not intended to indicate the presence of one or more other features. It should be understood that this does not exclude in advance the possibility of the existence or addition of elements, numbers, steps, operations, components, parts, or combinations thereof.

Unless otherwise defined, all terms used herein, including technical or scientific terms, mean the same as commonly understood by a person of ordinary skill in the technical field to which the present invention pertains. Terms such as those defined in commonly used dictionaries should be interpreted as consistent with the meaning they have in the context of the related technology, and unless clearly defined in the present application, should not be interpreted in an ideal or excessively formal sense.

Typically, a video may be composed of a series of pictures, and each picture may be divided into predetermined areas such as frames or blocks. In addition, the divided area is not only a block, but also a Coding Tree Unit (CTU), a Coding Unit (CU), a Prediction Unit (PU), and a Transform Unit (TU). It can be referred to by various sizes or terms, such as . Each unit may be composed of one luminance block and two chrominance blocks, which may be configured differently depending on the color format. Additionally, the sizes of the luminance block and the chrominance block may be determined depending on the color format. For example, in the case of 4:2:0, the size of the chrominance block may have a length that is 1/2 the width and height of the luminance block. For this unit and term, reference may be made to existing terms such as HEVC (high efficiency video coding) or H.264/AVC (advanced video coding).

Additionally, a picture, block, or pixel that is referenced for encoding or decoding the current block or current pixel is called a reference picture, reference block, or reference pixel. In addition, those skilled in the art will understand that the term "picture" described below may be used in place of other terms with equivalent meaning, such as image, frame, etc. When you grow up you will understand

Hereinafter, preferred embodiments of the present invention will be described in more detail with reference to the attached drawings. In order to facilitate overall understanding when describing the present invention, the same reference numerals are used for the same components in the drawings, and duplicate descriptions for the same components are omitted.

1 is a diagram for explaining a system using the video encoding device and/or video decoding device of the present invention.

Referring to FIG. 1, a system using an image encoding device and/or an image decoding device includes a personal computer (PC), a laptop computer, a personal digital assistant (PDA), and a portable multimedia player. It is a user terminal (11) such as PMP, PlayStation portable (PSP), wireless communication terminal, smart phone, television, etc., or a server terminal (11) such as application server and service server. 12) It can be. These systems may be referred to as computing devices.

In addition, computing devices include communication devices such as communication modems for communicating with various devices or wired and wireless communication networks, encoding or decoding images, or making predictions between screens (inter) or within screens (intra) for encoding and decoding. It may include various devices including a memory 18 for storing various programs and data, a processor 14 for executing programs to perform operations and control.

In addition, the computing device transmits images encoded into a bitstream by an image encoding device in real time or non-real time through wired and wireless communication networks such as the Internet, short-range wireless communication networks, wireless LAN networks, WiBro networks, and mobile communication networks, or through cables or general-purpose networks. It can be transmitted to a video decoding device through various communication interfaces such as a serial bus (USB: Universal Serial Bus), decoded by the video decoding device, and played back as a restored video. Additionally, an image encoded into a bitstream by an image encoding device may be transmitted from the encoding device to the decoding device through a computer-readable recording medium.

Figure 2 is a block diagram of a video encoding device according to an embodiment of the present invention. Figure 3 is a block diagram of an image decoding device according to an embodiment of the present invention.

As shown in FIG. 2, the image encoding device 20 according to this embodiment includes a prediction unit 200, a subtraction unit 205, a transform unit 210, a quantization unit 215, and an inverse quantization unit 220. , may include an inverse transform unit 225, an adder 230, a filter unit 235, a decoded picture buffer (DPB) 240, and an entropy encoding unit 245. Additionally, the image encoding device 20 may further include a division unit 190.

In addition, as shown in FIG. 3, the image decoding device 30 according to this embodiment includes an entropy decoding unit 305, a prediction unit 310, an inverse quantization unit 315, an inverse transform unit 320, and an adder. 325, a filter unit 330, and a decoded picture buffer 335 may be included.

The above-described video encoding device 20 and video decoding device 30 may be separate devices, but depending on implementation, they may be made into one video encoding and decoding device. In that case, the prediction unit 200, inverse quantization unit 220, inverse transform unit 225, addition unit 230, filter unit 235, and decoding picture buffer 240 of the video encoding device 20 are performed in the listed order. As a technical element that is substantially the same as the prediction unit 310, inverse quantization unit 315, inverse transform unit 320, addition unit 325, filter unit 330, and memory 335 of the image decoding device 30, It may be implemented to include at least the same structure or at least perform the same function. Additionally, the entropy encoding unit 245 may correspond to the entropy decoding unit 305 when performing its function in reverse. Therefore, in the detailed description of the technical elements and their operating principles below, redundant description of the corresponding technical elements will be omitted.

Since the video decoding device corresponds to a computing device that applies the video encoding method performed in the video encoding device to decoding, the following description will focus on the video encoding device.

A computing device may include a memory that stores a program or software module that implements an image encoding method and/or an image decoding method, and a processor that is connected to the memory and performs the program. Additionally, the video encoding device may be referred to as an encoder, and the video decoding device may be referred to as a decoder.

Each component of the video encoding device of this embodiment will be described in more detail as follows.

The division unit 190 divides the input image into blocks (M×N) of a predetermined size. Here, M or N is any natural number greater than or equal to 1.

In detail, the division unit 190 may be composed of a picture division unit and a block division unit. The size or shape of the block may be determined depending on the characteristics and resolution of the image, and the size or shape of the block supported through the picture division unit is an M × N square shape (256 ×256, 128×128, 64×64, 32×32, 16×16, 8×8, 4×4, etc.) or M×N rectangular shape. For example, the input video can be divided into sizes such as 256×256 for high-resolution 8k UHD video, 128×128 for 1080p HD video, and 16×16 for WVGA video.

Information about the size or shape of these blocks can be set in units such as sequence, picture, and slice, and related information can be transmitted to the decoder. That is, it can be set in a sequence parameter set, picture parameter set, slice header, or a combination thereof.

Here, a sequence refers to a structural unit composed by gathering several related scenes. And picture is a term that refers to a series of luminance (Y) components or the entire luminance and chrominance (Y, Cb, Cr) components in one scene or picture, and the range of one picture is one frame or one field depending on the case. It can be.

A slice may refer to one independent slice segment and multiple dependent slice segments that exist within the same access unit. An access unit refers to a set of NAL (network abstraction layer) units related to one coded picture. The NAL unit is a syntax structure that configures the video compression bitstream in a network-friendly format in the H.264/AVC and HEVC standards. It is common to configure one slice unit as one NAL unit, and system standards generally consider a NAL or a set of NALs constituting one frame as one access unit.

Going back to the description of the picture division unit, information about the block size or shape (M×N) may be comprised of an explicit flag, specifically block shape information, length information if the block is square, and In this case, it may include information on each length, or information on the difference between the horizontal and vertical lengths. For example, if M and N are composed of the power of k (assuming k is 2) (M=2 ^m , N=2 ⁿ ), the information about m and n can be unary binarized or truncated unary binarized. The relevant information can be transmitted to the decoding device by encoding in various ways, such as.

In addition, the minimum division size ( ^{Minblksize )} supported by the picture division unit is I You can. As another example, if M and N are different, the difference between m and n (|mn|) can be transmitted. Alternatively, if the maximum division size ( ^{Maxblksize )} supported by the picture division unit is I You can.

In an implicit situation, for example, when syntax for related information exists but cannot be confirmed by the encoder/decoder, the encoder or decoder may follow default settings prepared in advance. For example, if the relevant syntax cannot be confirmed in the step of checking the block shape information, the block shape can be left at the default square shape. In addition, the step of checking the block size information, more specifically, as in the above example, the syntax related to the difference value can be checked in the step of checking the block size information through the difference value from the minimum division allowable size (Minblksize), but the division If the syntax related to the minimum allowable size (Minblksize) cannot be confirmed, it can be obtained from the default setting value related to the minimum allowable split size (Minblksize) prepared in advance.

In this way, the size or shape of the block in the picture division unit may be explicitly transmitted by the encoder and/or decoder or may be implicitly determined depending on the characteristics and resolution of the image.

The block divided and determined through the picture division unit as described above can be used as a basic coding unit. Additionally, the block divided and determined through the picture division unit may be the minimum unit constituting higher level units such as a picture, slice, or tile, and may be a coding block, prediction block, or transform block. It may be the largest unit such as a transform block, quantization block, entropy block, or inloop filtering block, but some blocks are not limited to these and exceptions are possible. Some, for example in-loop filtering blocks, can be applied in units larger than the block size described above.

The block division unit performs division of blocks such as encoding, prediction, transformation, quantization, entropy, and in-loop filter. The division unit 190 is included in each component and performs a function. For example, the transform unit 210 may include a transform block division unit, and the quantization unit 215 may include a quantization block division unit. The size or shape of the initial block of the block division unit may be determined by the division result of the previous level or higher level block.

For example, in the case of a coding block, a block obtained through the picture division unit, which is the previous step, can be set as the initial block. Alternatively, in the case of a prediction block, a block obtained through the division process of a coding block that is a higher level of the prediction block can be set as the initial block. Alternatively, in the case of a transform block, a block obtained through the division process of a coding block, which is a higher level of the transform block, can be set as the initial block.

The conditions that determine the size or shape of the initial block are not always fixed, and some may change or exceptions may exist. In addition, the division status of the previous level or higher level block (e.g., size of coding block, type of coding block, etc.) and setting conditions of the current level (e.g., size of supported transform block, type of transform block, etc.) ) Depending on the combination of at least one factor, the division behavior of the current level (whether division is possible, type of block that can be divided, etc.) may be affected.

The block division unit may support a quad tree-based division method. In other words, the block before division can be divided into four blocks each with a width and height of 1/2 the length. This is based on the first block (dep_0), and the division allowed depth limit (dep_k, k means the number of divisions allowed, and the block size when the division allowed depth limit (dep_k) is (M >> k, N >> k)) Division can be repeated repeatedly.

Additionally, a binary tree-based partitioning method can be supported. This indicates that the block can be divided into two blocks whose length, either horizontal or vertical, is 1/2 that of the block before division. In the case of the quad tree partition and the binary tree partition, there may be symmetric partition or asymmetric partition, and which partition method to follow can be determined depending on the settings of the encoder/decoder. The video encoding method of the present invention will be explained mainly in terms of the symmetric division method.

The division flag (div_flag) can be used to indicate whether each block is divided. If the value is 1, division is performed, and if the value is 0, division is not performed. Alternatively, if the value is 1, division may be performed and additional division may be possible, and if the value is 0, division may not be performed and further division may not be permitted. Depending on conditions such as the minimum size allowed for division and the limit on the allowable division depth, the flag may only consider whether to split or not and may not consider whether to split additionally.

The split flag can be used in quad-tree splitting, and can also be used in binary tree splitting. In binary tree division, the division direction can be one of the division depth of the block, encoding mode, prediction mode, size, shape, type (encoding, prediction, transformation, quantization, entropy, in-loop filter, etc.), or luminance or chrominance. may be one) and may be determined based on at least one or a combination of factors such as slice type, division allowable depth limit, and division allowable minimum/maximum size. Additionally, depending on the split flag and/or the corresponding split direction, that is, the block may be split into 1/2 only horizontally or 1/2 only vertically.

For example, assuming that the block supports horizontal splitting with M The split flag is assigned as 1 bit. If the value is 1, horizontal split is performed, and if the value is 0, no further split can be performed. The split depth can be one split depth for quad tree and binary tree splits, or each split depth can be set for quad tree and binary tree splits. Additionally, the allowable splitting depth limit may be set to one splitting depth limit for quad tree and binary tree splits, or may be set to a separate allowable splitting depth limit for quad tree and binary tree splits.

As another example, if the block is M , if it is 0, division is not performed.

Additionally, flags (div_h_flag, div_h_flag) for horizontal division or vertical division can be supported, respectively, and binary division can be supported according to the flags. You can indicate whether each block is split horizontally or vertically through the horizontal split flag (div_h_flag) or vertical split flag (div_v_flag). If the horizontal split flag (div_h_flag) or vertical split flag (div_v_flag) is 1, horizontal or vertical split is performed. If it is 0, horizontal or vertical division is not performed.

In addition, if each flag is 1, horizontal or vertical division is performed and additional horizontal or vertical division is possible, and if the value is 0, horizontal or vertical division is not performed and further horizontal or vertical division may not be allowed. . Depending on conditions such as the minimum size allowed for division and the limit on the allowable division depth, the flag may consider whether to split or not and may not consider whether to split additionally.

Additionally, flags (div_flag/h_v_flag) for horizontal or vertical division may be supported, and binary division may be supported according to the flags. The split flag (div_flag) can indicate whether to split horizontally or vertically, and the split direction flag (h_v_flag) can indicate the horizontal or vertical split direction.

If the division flag (div_flag) is 1, division is performed and horizontal or vertical division is performed according to the division direction flag (h_v_flag). If the division flag (div_flag) is 0, horizontal or vertical division is not performed. Additionally, if the value is 1, horizontal or vertical division is performed according to the division direction flag (h_v_flag) and additional horizontal or vertical division is possible. If the value is 0, horizontal or vertical division is not performed and no further horizontal or vertical division is possible. may be considered not permitted.

Depending on conditions such as the minimum size allowed for division and the limit on the allowable division depth, the flag may consider whether to split or not and may not consider whether to split additionally.

These splitting flags can also be supported for horizontal and vertical splitting, and binary tree splitting can be supported according to the flags. Additionally, when the division direction is predetermined, as in the above example, only one of the two division flags may be used, or both division flags may be used.

For example, if all of the above flags are allowed, the possible block types can be divided into any of M×N, M/2×N, M×N/2, and M/2×N/2. In this case, the flag may be encoded as 00, 10, 01, 11 in the order of horizontal split flag or vertical split flag (div_h_flag/div_v_flag).

The above case is an example of a setting in which split flags can be overlapped and used, and a setting in which split flags cannot be overlapped and used is also possible. For example, the split block type can be split into M Alternatively, it may be encoded as 0, 10, and 11 in the order of the division flag (div_flag) and the horizontal-vertical flag (h_v_flag, which indicates the division direction is horizontal or vertical). Here, the meaning of overlapping may mean performing horizontal division and vertical division simultaneously.

The above-described quad-tree division and binary tree division may be used alone or in combination depending on the settings of the encoder and/or decoder. For example, quad tree or binary tree division may be determined depending on the size or shape of the block. That is, if the block type is M is M×N, and in the case where N and M are the same, quadtree partitioning can be supported.

As another example, binary tree partitioning may be supported when the size of the block (M

As another example, if M or N of the block (M Binary tree partitioning may be supported when M or N of (M

If the first division support range and the second division support range, which can be defined as the maximum division allowable size and the minimum division allowable size, overlap, the first or second division method takes priority depending on the setting of the encoder/decoder. Rankings may be given. In this embodiment, the first splitting method is quad-tree splitting, and the second splitting method is binary tree splitting. For example, if the minimum size allowed for the first division (Minblksize1) is 16, the maximum size allowed for the second division (Maxblksize2) is 64, and the block before division is 64×64, the first division support range and the second division support range are Since they all belong, quad tree partitioning and binary tree partitioning are possible.

If priority is given to the first division method (quad tree division in this embodiment) according to the preset, if the division flag (div_flag) is 1, quad tree division is performed and additional quad tree division is possible, and 0 days In this case, quad-tree partitioning is not performed and it can be considered that quad-tree partitioning is no longer performed. Depending on conditions such as the minimum size allowed for division and the limit on the allowable division depth, the flag may only consider whether to split or not and may not consider whether to split additionally.

When the division flag (div_flag) is 1, it is divided into 4 blocks with a size of 32×32 and is larger than the minimum size allowed for the first division (Minblksize1), so quad tree division can continue to be performed. If the split flag is 0, additional quad tree split is not performed, and since the current block size (64×64) falls within the second split support range, binary tree split can be performed. If the split flag (in the order of div_flag/h_v_flag) is 0, no further splitting is performed, and if it is 10 or 11, horizontal splitting or vertical splitting can be performed.

If the block before division is 32 2 Since it does not fall within the scope of division support, further division may not be supported. In the above description, the priority of the segmentation method may be determined based on at least one factor or a combination of slice type, encoding mode, luminance/chrominance component, etc.

As another example, various settings may be supported depending on luminance and chrominance components. For example, the quad tree or binary tree division structure determined in the luminance component can be used as is in the chrominance component without additional information encoding/decoding. Alternatively, when supporting independent division of the luminance component and the chrominance component, both quad tree and binary tree may be supported for the luminance component, and quad tree division may be supported for the chrominance component.

In addition, quad tree division and binary tree division are supported in luminance and chrominance components, but the division support range may or may not be the same or proportional to luminance and chrominance components. For example, in the case where the color format is 4:2:0, the division support range of the chrominance component may be N/2 of the division support range of the luminance component.

As another example, different settings can be set depending on the slice type. For example, an I slice can support quad tree partitioning, a P slice can support binary tree partitioning, and a B slice can support both quad tree partitioning and binary tree partitioning.

As in the example above, quad tree partitioning and binary tree partitioning can be set and supported according to various conditions. The above examples are not specific only to the above-described cases and may include cases where the conditions are reversed, and may also include cases where one or more factors mentioned in the above examples or a combination thereof are used, and may also be modified as examples of other cases. possible. The above allowable division depth limit may be determined based on at least one or a combination of factors such as division method (quad tree, binary tree), slice type, luminance/chrominance component, and encoding mode.

In addition, the division support range may be determined based on at least one or a combination of factors such as division method (quad tree, binary tree), slice type, luminance/chrominance component, and encoding mode, and related information is provided in the division support range. It can be expressed as the maximum or minimum value. When this information is configured as an explicit flag, information on the length of each maximum/minimum value, or information on the difference between the minimum and maximum values, etc. can be expressed.

For example, if the maximum and minimum values are composed of the exponents of k (assuming k to be 2), the exponent information of the maximum and minimum values can be encoded through various binarizations and transmitted to the decoding device. Alternatively, the difference between the exponents of the maximum and minimum values can be transmitted. At this time, the information transmitted may be index information of the minimum value and information of the difference value of the index.

According to the above description, information related to the flag can be generated and transmitted in units such as sequence, picture, slice, tile, and block.

The split flags presented in the example above can represent block split information through a quad tree or binary tree, or a mixture of the two tree methods, and the split flag is encoded in various methods such as unary binarization and truncated unary binarization to decode the related information. Can be transmitted to the device. The bitstream structure of the split flag for expressing the split information of the block can be selected from one or more scan methods.

For example, the bitstream of the split flags can be configured based on the split depth order (from dep0 to dep_k), and the bitstream of the split flags can also be configured based on whether or not the split flags are split. In the division depth order-based method, division information at the current level depth is obtained based on the first block and then division information at the next level depth is obtained. In the division method based on the first block, division information is obtained in the divided block based on the first block. It refers to a method of preferentially acquiring additional segmentation information, and other scan methods not presented in the above example may be included and selected.

Additionally, depending on implementation, the block division unit may generate and express index information for a predefined type of block candidate group rather than the above-described division flag. The form of the block candidate group is, for example, the form of the split block that can be had in the block before splitting, such as M×N, M/2×N, M×N/2, M/4×N, 3M/4×N, It may include M×N/4, M×3N/4, M/2×N/2, etc.

Once the candidate group for the split block is determined as above, the index information for the split block type can be encoded through various methods such as fixed-length binarization, truncated binarization, truncated binarization, etc. As with the split flag described above, at least one or a combination of factors such as the block's split depth, coding mode, prediction mode, size, shape, type, slice type, split allowable depth limit, and split allowable minimum/maximum size, etc. Accordingly, the split block candidate group may be determined.

For the following explanation, (M×N, M×N/2) is candidate list 1 (list1), and (M×N, M/2×N, M×N/2, M/2×N/2) is candidate list 1. List 2 (list2), (M×N, M/2×N, M×N/2) is converted into candidate list 3 (list3), (M×N, M/2×N, M×N/2, M/ 4×N, 3M/4×N, M×N/4, M×3N/4, M/2×N/2) is assumed to be candidate list 4 (list4). For example, when explaining based on M×N, if (M=N), the split block candidate of candidate list2 can be supported, and if (M≠N), the split block candidate of candidate list3 can be supported.

As another example, if M or N of M In addition, if M or N is greater than or equal to the first boundary value (blk_th_1), the split block candidate of candidate list1 is selected, and if M or N is less than the first boundary value (blk_th_1) but greater than or equal to the second boundary value (blk_th_2), the candidate If the split block candidate in list2 is smaller than the second boundary value (blk_th_2), the split block candidate in candidate list4 can be supported.

As another example, if the encoding mode is intra-screen prediction, the split block candidate of candidate list2 can be supported, and if the encoding mode is inter-screen prediction, the split block candidate of candidate list4 can be supported.

Even if the above split block candidates are supported, the bit configuration according to binarization in each block may be the same or different. For example, if the supported split block candidates are limited depending on the block size or shape as applied in the split flag above, the bit configuration may vary depending on the binarization of the block candidate. For example, in the case of (M>N), block types according to horizontal division, that is, M The binary bits of the index according to M×N/2 in (2×N, M×N/2, M/2×N/2) and M×N/2 of the current condition may be different.

Depending on the type of block used, for example, for encoding, prediction, transformation, quantization, entropy, in-loop filtering, etc., information about the division and shape of the block can be expressed using either a split flag or a split index method. Additionally, depending on each block type, the block size limit and allowable split depth limit for splitting and block type support may be different.

In the block-level encoding and decoding process, a coding block is first determined, and then encoding and decoding can be performed according to processes such as prediction block determination, transformation block determination, quantization block determination, entropy block determination, and in-loop filter determination. The order of the above encoding and decoding processes is not always fixed, and some orders may be changed or excluded. The size and shape of each block are determined according to the encoding cost for each candidate for the size and shape of the block, and segmentation-related information such as image data of each determined block and the size and shape of each determined block can be encoded.

The prediction unit 200 may be implemented using a prediction module, which is a software module, and may generate a prediction block for a block to be encoded using an intra-screen prediction method or an inter-screen prediction method. Here, the prediction block is a block that is understood to closely match the block to be encoded in terms of pixel difference, and can be determined by various methods including sum of absolute difference (SAD) and sum of square difference (SSD). Additionally, at this time, various syntaxes that can be used when decoding image blocks can be generated. Prediction blocks can be classified into intra-screen blocks and inter-screen blocks depending on the encoding mode.

Intra prediction is a prediction technique that uses spatial correlation and refers to a method of predicting the current block using reference pixels of previously encoded, decoded, and restored blocks within the current picture. In other words, the brightness value reconstructed through intra-screen prediction and restoration can be used as a reference pixel in the encoder and decoder. In-screen prediction can be effective for flat areas with continuity and areas with a certain direction, and because it uses spatial correlation, it can be used to ensure random access and prevent error spread.

Inter prediction uses a compression technique that removes data duplication using temporal correlation by referring to images encoded in one or more past or future pictures. That is, inter-picture prediction can generate a prediction signal with high similarity by referring to one or more past or future pictures. In an encoder using inter-screen prediction, a block with a high correlation with the block to be currently encoded can be searched in the reference picture, and the position information and residual signal of the selected block can be transmitted to the decoder, and the decoder can provide selection information of the transmitted image. Using , the same prediction block as the encoder can be generated and the transmitted residual signal can be compensated to construct a restored image.

Figure 4 is an example diagram showing inter-screen prediction of a P slice in an image encoding and decoding method according to an embodiment of the present invention. Figure 5 is an example diagram showing inter-screen prediction of a B slice in an image encoding and decoding method according to an embodiment of the present invention.

In the video encoding method of this embodiment, inter-screen prediction can increase encoding efficiency because prediction blocks are generated from previously encoded pictures with high temporal correlation. Current(t) may refer to the current picture to be encoded, and based on the temporal flow of the video picture or POC (picture order count), the first temporal distance (t-1) before the POC of the current picture. It may include a first reference picture having and a second reference picture having a second temporal distance (t-2) before the first temporal distance.

That is, as shown in FIG. 4, inter-screen prediction that can be employed in the video encoding method of this embodiment is the current block of the current picture (current(t)) and the reference pictures (t-1, t-2). Through block matching of reference blocks, motion estimation can be performed to find the optimal prediction block from reference pictures (t-1, t-2) for which blocks with high correlation have previously been encoded. For precise estimation, if necessary, an interpolation process based on a structure in which at least one or more subpixels are arranged between two adjacent pixels is performed, then the optimal prediction block is found, and motion compensation is performed to create the final prediction block. can be found.

In addition, as shown in FIG. 5, inter-screen prediction that can be employed in the video encoding method of this embodiment is based on already encoded reference pictures (current(t)) that exist in both directions temporally based on the current picture (current(t)). A prediction block can be generated from (t-1, t+1). Additionally, two prediction blocks can be generated from one or more reference pictures.

When encoding an image through inter-screen prediction, motion vector information about the optimal prediction block and information about the reference picture are encoded. In this embodiment, when generating a prediction block in one direction or two directions, the reference picture list can be configured differently and the prediction block can be generated from the corresponding reference picture list. Basically, reference pictures that exist temporally before the current picture can be managed by being assigned to list 0 (L0), and reference pictures that exist after the current picture can be managed by being assigned to list 1 (L1).

When constructing reference picture list 0, if the allowable number of reference pictures in reference picture list 0 is not filled, a reference picture that exists after the current picture can be assigned. Similarly, when constructing reference picture list 1, if the allowable number of reference pictures in reference picture list 1 is not filled, a reference picture that exists before the current picture can be assigned.

Figure 6 is an example diagram illustrating the case of generating a prediction block unidirectionally in the video encoding and decoding method according to an embodiment of the present invention.

Referring to FIG. 6, in the video encoding and decoding method according to this embodiment, a prediction block can be found from previously encoded reference pictures (t-1, t-2) as before, and in addition, the current picture (current( A prediction block can be found from an area that has already been encoded in t)).

That is, in the image encoding and decoding method according to this embodiment, not only a prediction block is generated from previously encoded pictures (t-1, t-2) with high temporal correlation, but also a prediction block with high spatial correlation is generated. It can be implemented to find together. Finding such spatially highly correlated prediction blocks can correspond to finding prediction blocks by way of intra-picture prediction. In order to perform block matching from a fully encoded area in the current picture, the video encoding method of this embodiment can be combined with an intra-prediction mode to configure a syntax for information related to the prediction candidate.

For example, if n (n is a random natural number) intra-screen prediction modes are supported, n+1 modes are supported by adding one mode to the intra-screen prediction candidate group, and 2 ^M-1 ≤n+1 The prediction mode can be encoded using M fixed bits that satisfy <2 ^M. Additionally, it can be implemented to select from a group of candidates for a highly probable prediction mode, such as HEVC's most probable mode (MPM). Additionally, encoding may be preferentially performed at a higher level of prediction mode encoding.

When generating a prediction block through block matching in the current picture, the video encoding method of this embodiment may be combined with the inter-screen prediction mode to configure syntax for related information. Information related to motion or displacement may be used as additional related prediction mode information. Motion or movement-related information may include optimal candidate information among several vector candidates, difference values between the optimal candidate vector and the actual vector, reference direction, reference picture information, etc.

Figure 7 is an example diagram of configuring a reference picture list in the video encoding and decoding method according to an embodiment of the present invention. Figure 8 is an exemplary diagram showing another example of performing inter-screen prediction from a reference picture list in the video encoding and decoding method according to an embodiment of the present invention.

Referring to FIG. 7, the video encoding method according to this embodiment includes a first reference picture list (reference list 0, L0) and a second reference picture list (reference list) for the current block of the current picture (current(t)). 1, L1), inter-screen prediction can be performed.

Referring to Figures 7 and 8, reference picture list 0 can be composed of reference pictures before the current picture (t), where t-1 and t-2 are the first picture before the POC of the current picture (t), respectively. Indicates reference pictures having a temporal distance (t-1) and a second temporal distance (t-2). In addition, reference picture list 1 can be composed of reference pictures after the current picture (t), where t+1 and t+2 are the first temporal distance (t+1) after the POC of the current picture (t), respectively. ), indicates reference pictures having a second temporal distance (t+2).

The above-described examples of reference picture list construction show examples of configuring the reference picture list with reference pictures whose temporal distance difference (in this example, POC standard) is 1, but the temporal distance difference between reference pictures is constructed differently. You may. That is, this means that the index difference between reference pictures and the temporal distance difference between reference pictures may not be proportional. Additionally, the order of list composition may not be based on temporal distance. Details about this can be found in the reference picture list configuration example that will be described later.

Prediction can be performed from reference pictures in the list depending on the slice type (I, P, or B). And when generating a prediction block through block matching in the current picture (current(t)), encoding can be performed using inter-screen prediction by adding the current picture to the reference picture list (reference list 0 and/or reference list 1). You can.

As shown in FIG. 8, the current picture (t) can be added to reference picture list 0 (reference list 0) or the current picture (current(t)) can be added to reference picture list 1 (reference list 1). In other words, reference picture list 0 can be constructed by adding a reference picture with a temporal distance (t) to the reference picture before the current picture (t), and reference picture list 1 can be constructed by adding a reference picture with a temporal distance (t) to the reference picture before the current picture (t). It can also be configured by adding a reference picture with an appropriate distance (t).

For example, when constructing reference picture list 0, the reference picture before the current picture can be assigned to reference picture list 0, followed by the current picture (t), and when constructing reference picture list 1, the reference picture after the current picture can be assigned. A reference picture can be assigned to reference picture list 1 and then the current picture (t) can be assigned. Alternatively, when constructing reference picture list 0, you can allocate the current picture (t) and then allocate the reference picture before the current picture, and when constructing reference picture list 1, you can allocate the current picture (t) and then assign the current picture. Subsequent reference pictures can be assigned.

Additionally, when constructing reference picture list 0, a reference picture before the current picture can be allocated, then a reference picture after the current picture can be allocated, and then the current picture (t) can be allocated. Similarly, when constructing reference picture list 1, a reference picture after the current picture can be allocated, then a reference picture before the current picture can be allocated, and then the current picture (t) can be allocated. The above examples are not specific to the above-described cases and may include cases where the conditions are reversed, and can also be modified to examples of other cases.

Whether to include the current picture in each reference picture list (e.g., not adding it to any list, or adding it only to list 0, or only adding it to list 1, or adding it to both lists 0 and 1) has the same setting in the encoder/decoder. This is possible, and information about this can be transmitted in units such as sequence, picture, and slice. This information can be encoded through methods such as fixed-length binarization, short-cut binarization, and truncated binarization.

Unlike the method of FIG. 7, the video encoding and decoding method of this embodiment selects a prediction block by performing block matching in the current picture (t), and constructs a reference picture list containing related information about this prediction block. , there is a difference in using this reference picture list for video encoding and decoding.

When constructing a reference picture list, the order and rules for constructing each list and the allowable number of reference pictures in each list can be set differently, which depends on whether or not the current picture is included in the list (whether the current picture is included as a reference picture in inter-screen prediction). various parameters such as slice type, list reconstruction parameters (may be applied to lists 0 and 1 respectively, or may be applied to lists 0 and 1 together), location within GOP (Group of Picture), temporal layer information (temporal id), etc. It may be determined based on at least one of the factors or a combination thereof, and related information may be explicitly transmitted in units such as sequences and pictures.

For example, in the case of a P slice, reference picture list 0 can follow list construction rule A regardless of whether the current picture is included in the list, and in the case of a B slice, reference picture list 0 that includes the current picture in the list can be configured in the list. Rule B, list construction rule C can be followed for reference picture list 1, list construction rule D can be followed for reference picture list 0, which does not include the current picture, and list construction rule E can be followed for reference picture list 1, list construction rule Among them, B and D, C and E may be the same.

The list construction rule may be configured in the same or modified manner as described in the reference picture list construction example. As another example, when the current picture is included in the list, the allowable number of first reference pictures can be set, and when the current picture is not included in the list, the allowable number of second reference pictures can be set. The allowed number of first reference pictures and the allowed number of second reference pictures may be the same or different, and the default setting may be that the difference between the allowed number of first reference pictures and the allowed number of second reference pictures is 1.

As another example, when the current picture is included in the list and the list reconstruction parameters are applied, all reference pictures may be included in the list reconstruction candidates in slice A, and only some reference pictures may be included in the list reconstruction candidates in slice B. At this time, slice A or B can be distinguished by whether or not it is included in the list of the current picture, temporal layer information, slice type, position within the GOP, etc., and the POC or reference picture index of the reference picture, reference prediction, etc. are factors for determining inclusion in the candidate group. It can be determined by the direction (before/after the current picture), whether it is the current picture, etc.

According to the above-described configuration, a reference block encoded by inter-prediction can be used in the current picture, so inter-prediction can be allowed or used even in motion prediction of an I slice.

Additionally, when constructing a reference picture list, the order of index allocation or list construction can be varied depending on the slice type. In the case of an I slice, the priority is set high in the current picture (current(t)) as in the reference picture list configuration example, and fewer indices (e.g., idx=0, 1, 2) are used, and the corresponding The amount of bits in video encoding can be reduced through binarization (fixed-length binarization, truncated binarization, truncated binarization, etc.) that sets the allowable number (C) of reference pictures in the reference picture list to the maximum.

Additionally, in the case of a P or B slice, if block matching is performed on the current picture and the probability of selecting the reference picture of the current block as a prediction candidate is determined to be lower than the probability of selecting the prediction candidate through another reference picture, the current picture Various methods of binarization that set the priority for block matching backwards and use a higher index (e.g., idx=C, C-1) to maximize the number of reference pictures allowed in the corresponding reference picture list. Through this, the amount of bits in video encoding can be reduced.

In the above example, the priority setting of the current picture may be configured in the same or modified manner as described in the reference picture list configuration example. Additionally, it is possible to omit information about reference pictures by not constructing a reference picture list according to slice type (eg, I slice). For example, a prediction block can be generated through existing inter-prediction, but the inter-prediction information can be expressed by excluding the reference picture information from the motion information in the inter-prediction mode.

The method of performing block matching in the current picture can be supported or not depending on the slice type. For example, block matching in the current block can be set to be supported in the I slice but not in the P slice or B slice, and variations to other examples are also possible. In addition, the method of supporting block matching in the current picture may be determined on a per picture, slice, or tile basis, or may be determined based on location within the GOP, temporal layer information (temporal ID), etc. This setting information can be transmitted during the video encoding process or from the encoder to the decoder in units such as sequences, pictures, and slices.

In addition, even in a situation where setting information or syntax related to the above exists in a higher level unit and a setting-related operation is turned on, when the same setting information or syntax as above exists in a lower level unit, the setting information in the lower level unit is transmitted to the higher level unit. Setting information in units can be prioritized. For example, if the same or similar setting information is processed in a sequence, picture, or slice, the picture unit may have priority over the sequence unit, and the slice unit may have priority over the picture unit.

Figure 9 is an example diagram for explaining intra-screen prediction in the video encoding method according to an embodiment of the present invention.

Referring to FIG. 9, the intra-screen prediction method according to this embodiment includes reference pixel padding, reference sample filtering, intra prediciton, and boundary filtering. It may involve a series of steps.

The reference pixel filling step may be an example of the reference pixel configuration step, the reference pixel filtering step may be executed by the reference pixel filter unit, the intra-screen prediction may include a prediction block generation step and a prediction mode encoding step, and the boundary Filtering may be an example of one embodiment of a post-processing filter step.

That is, intra-picture prediction performed in the video encoding method of this embodiment may include a reference pixel configuration step, a reference pixel filtering step, a prediction block generation step, a prediction mode encoding step, and a post-processing filtering step. Depending on various environmental factors, such as block size, block type, block location, prediction mode, prediction method, quantization parameter, etc., one or part of the above-described processes may be omitted, and other processes may be added, in the order described above. It may be changed to a different order than .

The above-described reference pixel configuration step, reference pixel filtering step, prediction block generation step, prediction mode encoding step, and post-processing filtering step may be implemented in a form in which a processor connected to the memory executes software modules stored in the memory. Therefore, in the following description, for convenience of explanation, the reference pixel component and the reference pixel filter are referred to as functional units created by a combination of software modules implementing each step and processors executing them, and/or components that perform the functions of these functional units, respectively. The unit, prediction block generation unit, prediction mode encoding unit, and post-processing filter unit will be referred to as the executor of each step.

To describe each component in more detail, the reference pixel component configures reference pixels to be used for prediction of the current block through reference pixel filling. If a reference pixel does not exist or is unavailable, reference pixel filling can be used for the reference pixel, such as by copying values from a nearby available pixel. A restored picture buffer or a decoded picture buffer (DPB) can be used to copy values.

That is, intra-picture prediction is performed using reference pixels of previously encoded blocks of the current picture. To this end, in the reference pixel configuration step, neighboring pixels of the current block, i.e., left, upper left, lower left, upper, upper right blocks, etc., are mainly used as reference pixels.

However, the candidate group of neighboring blocks for the reference pixel is only an example when the coding order of the block follows raster scan or z-scan, and scan such as inverse z-scan. When this coding order scan method is used, in addition to the above blocks, adjacent pixels such as the right, bottom right, and bottom blocks can also be used as reference pixels.

In addition, depending on the implementation, depending on the step-by-step configuration of intra-screen prediction, additional pixels other than immediately adjacent pixels may be used as replacements or mixed with existing reference pixels.

Additionally, when prediction is made in a directional mode among intra-prediction modes, reference pixels in decimal units can be generated through linear interpolation of reference pixels in integer units. The mode of performing prediction through reference pixels existing at integer unit positions includes some modes having vertical, horizontal, 45 degrees, and 135 degrees, and for the above prediction modes, the process of generating reference pixels in decimal units is It may not be necessary.

Reference pixels interpolated in prediction modes with different directions other than the above prediction mode are multiplied by an exponent of 1/2, such as 1/2, 1/4, 1/8, 1/16, 1/32, and 1/64. It may have interpolation precision, or it may have precision of a multiple of 1/2.

This is because the interpolation precision can be determined depending on the number of prediction modes supported or the prediction direction of the prediction mode. Fixed interpolation precision may always be supported in pictures, slices, tiles, blocks, etc., or adaptive interpolation precision may be supported depending on the size of the block, the shape of the block, the prediction direction of the supported mode, etc. At this time, the predicted direction of the mode can be expressed as tilt information or angle information in the direction indicated by the mode with a specific line reference (for example, the positive x-axis on the coordinate plane).

As an interpolation method, linear interpolation is performed using immediately adjacent integer pixels, but other interpolation methods can be supported. For interpolation, one or more filter types and number of taps, for example, 6-tap Wiener filter, 8-tap Kalman filter, etc. can be supported, and the type of interpolation to be performed can be determined depending on the size of the block, prediction direction, etc. there is. Additionally, related information may be transmitted in units such as sequence, picture, slice, and block.

The reference pixel filter unit may perform filtering on the reference pixel for the purpose of increasing prediction efficiency by reducing deterioration remaining during the encoding process after configuring the reference pixel. The reference pixel filter unit may implicitly or explicitly determine the type of filter and whether or not to apply filtering depending on the size, shape, and prediction mode of the block. In other words, even for filters of the same tap, filter coefficients can be determined differently depending on the filter type. For example, you could use a 3-tap filter like [1,2,1]/4, [1,6,1]/8.

Additionally, the reference pixel filter unit may determine whether to apply filtering by determining whether to send additional bits or not. For example, in the implicit case, the reference pixel filter unit may determine whether to apply filtering according to the characteristics (variance, standard deviation, etc.) of pixels in the surrounding reference blocks.

Additionally, the reference pixel filter unit may determine whether to apply filtering when the relevant flag satisfies preset hiding conditions such as residual coefficients and intra-screen prediction mode. The number of taps in the filter is, for example, 3-tap such as [1,2,1]/4 in small block (blk), and [2,3,6,3,2]/16 in large block (blk). It can be set to the same 5-tap, and the number of applications can be determined by whether to not perform filtering, to filter once, or to filter twice.

Additionally, the reference pixel filter unit can basically apply filtering to the closest reference pixel of the current block. In addition to the nearest reference pixel, additional reference pixels may also be considered in the filtering process. For example, filtering may be applied to additional reference pixels by replacing the most adjacent reference pixel, or filtering may be applied by mixing additional reference pixels with the most adjacent reference pixel.

The above filtering can be applied fixedly or adaptively, which includes the size of the current block or the size of the neighboring block, the coding mode of the current block or the neighboring block, and the block boundary characteristics of the current block and the neighboring block (e.g., the size of the coding unit). It can be determined based on at least one or a combination of factors such as (whether it is a boundary or a boundary of a transformation unit, etc.), prediction mode or direction of the current block or neighboring blocks, prediction method of the current block or neighboring blocks, quantization parameters, etc. there is. This decision may be made by having the same settings in the encoder/decoder (implicit) or taking into account coding costs, etc. (explicit). Basically, the applied filter is a low pass filter, and the number of filter taps, filter coefficients, whether the filter flag is encoded, the number of filter applications, etc. can be determined depending on the various factors specified above, and information about this can be stored in the sequence, picture, Settings can be made in units such as slices and blocks, and related information can be transmitted to the decoder.

In intra-screen prediction, the prediction block generator uses extrapolation or extrapolation through reference pixels, an interpolation method such as the average value (DC) of the reference pixel or planar mode, or a copy of the reference pixel. ) method, you can create a prediction block.

*In the case of copying a reference pixel, one or more predicted pixels can be created by copying one reference pixel, or one or more predicted pixels can be created by copying one or more reference pixels. The number of copied reference pixels is It may be equal to or less than the number of predicted pixels.

In addition, depending on the prediction method, it can be classified into a directional prediction method and a non-directional prediction method. In detail, the directional prediction method can be classified into a linear directional method and a curved directional method. The straight-line directional method uses an extrapolation or extrapolation method, but the pixels of the prediction block are generated through reference pixels located on the prediction direction line. The curved directionality method adopts an extrapolation or extrapolation method, but the pixels of the prediction block are generated through reference pixels located on the prediction direction line. This refers to a method in which partial prediction direction changes on a pixel basis are allowed while being generated through pixels, taking into account the detailed direction of the block (for example, edge <Edge>).

In the case of the directional prediction mode in the video encoding and decoding method of this embodiment, the description will focus on the linear directional method.

Additionally, in the case of the directional prediction method, the spacing between adjacent prediction modes may be equal or uneven, and this may be determined depending on the size or shape of the block. For example, when the current block obtains a block with the size and shape of M The intervals between prediction modes may be non-uniform.

As another example, when M is greater than N, modes with vertical directionality can be assigned finer spacing between prediction modes that are close to the vertical mode (90 degrees), and wider spacing can be assigned to prediction modes that are far from the vertical mode. . When N is greater than M, modes with horizontal orientation can be assigned finer intervals between prediction modes that are close to the horizontal mode (180 degrees), and wider intervals can be assigned to prediction modes that are far from the horizontal mode.

The above examples are not specific to the above-described cases and may include cases where the conditions are reversed, and can also be modified to examples of other cases. At this time, the interval between prediction modes can be calculated based on a numerical value indicating the directionality of each mode, and the directionality of the prediction mode can be quantified by direction slope information or angle information.

Additionally, in addition to the above method, a prediction block can be generated including other methods that use spatial correlation. For example, using the current picture as a reference picture, a reference block using inter prediction methods such as motion detection and compensation can be generated as a prediction block.

In the prediction block generation step, a prediction block may be generated using reference pixels according to the prediction method. That is, depending on the prediction method, a prediction block can be generated through directional or non-directional prediction methods such as extrapolation, interpolation, copy, and average of the existing intra-screen prediction method, and the prediction block can be generated using the inter-screen prediction method. It can be created, and other additional methods can also be used.

The intra-picture prediction method can be supported under the same settings of the encoder/decoder and can be determined depending on the slice type, block size, block shape, etc. The intra-screen prediction method may be supported according to at least one of the above-mentioned prediction methods or a combination thereof. The intra-screen prediction mode can be configured according to the supported prediction method. The number of supported intra-screen prediction modes may be determined depending on the prediction method, slice type, block size, block shape, etc. The related information can be set and transmitted in units such as sequence, picture, slice, and block.

The prediction mode encoding step of executing prediction mode encoding may determine the mode with the optimal encoding cost for each prediction mode as the prediction mode of the current block in terms of encoding cost.

For example, the prediction mode encoder may use the mode of one or more neighboring blocks for current block mode prediction for the purpose of reducing prediction mode bits. A mode with a high probability of being the same as the mode of the current block (most_probable_mode, MPM) can be included as a candidate group, and modes of neighboring blocks can be included in the above candidate group. For example, the prediction mode of the block, such as top left, bottom left, top, and top right of the current block, can be included in the above candidate group.

Candidates for the prediction mode are selected from factors such as the location of the neighboring block, the priority of the neighboring block, the priority in the split block, the size or shape of the neighboring block, a preset specific mode, and the prediction mode of the luminance block (in the case of a chrominance block). It can be configured based on at least one factor or a combination thereof, and related information can be transmitted in units such as sequence, picture, slice, and block.

For example, if the current block and the neighboring block are divided into two or more blocks, which block's mode among the divided blocks is to be included as a mode prediction candidate for the current block can be determined under the same settings of the encoder/decoder. there is. In addition, for example, among the neighboring blocks of the current block (M When including blocks of M/4×M/4 and M/4×M/4, the prediction mode of the M/2×M/2 block can be included as a mode prediction candidate for the current block based on the block size.

As another example, among the neighboring blocks of the current block (N When including blocks of It can be included as .

As another example, if the prediction mode of the current block and the neighboring block is a directional prediction mode, the prediction direction of the corresponding mode and the adjacent prediction mode (in terms of slope information or angle information of the direction of the mode) are added to the mode prediction candidate group of the current block. It can be included. Additionally, preset modes (planar, DC, vertical, horizontal, etc.) may be preferentially included depending on the configuration or combination of prediction modes of neighboring blocks.

In addition, among the prediction modes of neighboring blocks, the prediction mode with a high occurrence frequency may be preferentially included. The priority may mean not only the possibility of being included in the mode prediction candidate group of the current block, but also the possibility of being assigned a higher priority or index (i.e., a higher probability of being assigned fewer bits during the binarization process) in the candidate group configuration. there is.

As another example, the maximum number of mode prediction candidates for the current block is k, the left block consists of m blocks whose length is smaller than the vertical length of the current block, and the upper block consists of n blocks whose length is smaller than the horizontal length of the current block. When the split block sum (m+n) of neighboring blocks is greater than k, the candidate group can be filled according to a preset order (left to right, top to bottom), and the split block sum (m+ If n) is greater than the maximum value (k) of the candidate group, the prediction mode of the neighboring block (left block, upper block) is set to other neighboring blocks (e.g., lower left, upper left, upper right, etc.) than the neighboring block location. The prediction mode of a block such as may also be included in the mode prediction candidate group of the current block. The above examples are not specific to the above-described cases and may include cases where the conditions are reversed, and can also be modified to examples of other cases.

In this way, the candidate block for predicting the mode of the current block is not limited to a specific block location, and prediction mode information can be utilized from at least one of the blocks located on the left, top left, bottom left, top, and top right. As in the above example, the prediction mode of the current block can be configured as a candidate group by considering various factors.

In the prediction mode encoding unit, the mode (MPM) candidates that have a high probability of being the same as the mode of the current block (referred to as candidate group 1 in this example) can be classified into a mode candidate group that is not (referred to as candidate group 2 in this example), and the current The prediction mode encoding process may vary depending on which candidate group the prediction mode of the block belongs to among the candidate groups.

The overall prediction mode can be composed of the sum of the prediction modes of candidate group 1 and the prediction mode of candidate group 2, and the number of prediction modes in candidate group 1 and the number of prediction modes in candidate group 2 are the number of total prediction modes, slice type, size of block, It may be determined based on one or more factors, such as the shape of the block, or a combination of these factors. Depending on the candidate group, the same binarization or different binarization can be applied.

For example, fixed-length binarization may be applied to candidate group 1, and single-cut binarization may be applied to candidate group 2. In the above explanation, the number of candidate groups is given as two, but the first mode candidate group has a high probability of being the same as the mode of the current block, the second mode candidate group has a high probability of being the same as the mode of the current block, the mode candidate group that does not, etc. Expansion is possible, and modifications thereof are also possible.

The post-processing filtering step executed by the post-processing filter unit selects some predicted pixels from the prediction blocks generated in the previous process, taking into account the characteristic of high correlation between reference pixels adjacent to the boundary between the current block and neighboring blocks and pixels in the adjacent current block. may be replaced with a value generated by filtering one or more reference pixels adjacent to the boundary and one or more prediction pixels, and a value that quantifies the characteristics between reference pixels adjacent to the boundary of the block (e.g., difference in pixel values, The prediction pixel can be replaced with a value generated by applying gradient information, etc.) to the filtering process, and in addition to the above method, other methods with a similar purpose (correcting some prediction pixels of the prediction block through reference pixels) have been added. It can be.

In the post-processing filter unit, the type of filter and whether or not filtering is applied can be implicitly or explicitly determined, and the location and number of reference pixels and current pixels used in the post-processing filter unit, and the type of prediction mode applied, etc. It can be set in the encoder/decoder, and related information can be transmitted in units such as sequence, picture, and slice.

Additionally, in the post-processing filtering step, additional post-processing processes may be performed after generating the prediction block, such as block boundary filtering. In addition, after acquiring the residual signal, the current block restored by combining the residual signal and the prediction signal obtained through the conversion/quantization process and the reverse process is post-processed by taking into account the characteristics of the pixels of the adjacent reference block similar to the boundary filtering above. It can also be done.

Finally, the prediction block is selected or acquired through the above-described process, and the information resulting from this process may include prediction mode-related information, and after obtaining the prediction block, it is transmitted to the transformer 210 for encoding of the residual signal. You can.

Figure 10 is an example diagram for explaining the prediction principle in a P slice or B slice in the image encoding method according to an embodiment of the present invention. FIG. 11 is an example diagram illustrating a case of performing interpolation in the image encoding method of FIG. 10.

Referring to FIG. 10, the image encoding method according to this embodiment may include motion estimation (motion estimation module) and interpolation steps. Information about the motion vector, reference picture index, and reference direction generated in the motion prediction step may be transmitted to the interpolation step. In the motion prediction step and the interpolation step, values stored in the restored picture buffer (decoded picture buffer, DPB) can be used.

That is, the video encoding device may perform motion estimation to find a block similar to the current block in previously encoded pictures. In addition, the video encoding device can perform interpolation of the reference picture for prediction more precise than decimal precision. Finally, the video encoding device obtains a prediction block through a predictor, and the information that comes out of this process is a motion vector, a reference picture index (reference picture index or reference index), and a reference direction. ), etc., and then residual signal coding can be performed.

In this embodiment, since intra-screen prediction is also performed on the P slice or B slice, it is possible to implement a combination method as shown in Figure 11 that supports inter-screen prediction and intra-screen prediction.

As shown in FIG. 11, the image encoding method according to this embodiment includes reference pixel padding, reference sample filtering, intra prediction, boundary filtering, It may include steps of motion estimation and interpolation.

When the video encoding device supports block matching in the current picture, the prediction method in the I slice can be implemented using the configuration shown in FIG. 11 rather than the configuration shown in FIG. 9. That is, the video encoding device can use information such as motion vectors, reference picture index, and reference direction that occur only in the P slice or B slice as well as the prediction mode in the I slice to generate a prediction block. However, due to the nature of the reference picture being current, there may be information that can be partially omitted. For example, if the reference picture is the current picture, the reference picture index and reference direction can be omitted.

In addition, when applying interpolation, the image encoding device may not require block matching up to decimal units due to the nature of artificial images such as computer graphics, so whether or not to perform this may be determined by the encoder. It is also possible to set units such as sequence, picture, and slice.

For example, the video encoding device may not perform interpolation of reference pictures used for inter-screen prediction depending on the settings of the encoder, and may make various settings such as not performing interpolation only when block matching is performed in the current picture. You can. That is, the video encoding device of this embodiment can set whether to perform interpolation of reference pictures. At this time, it can be determined whether to perform interpolation on all or some reference pictures constituting the reference picture list.

For example, the image encoding device does not perform interpolation when there is no need to perform block matching in decimal units because the characteristics of the image in which the reference block exists in a certain current block is an artificial image, but matches blocks in decimal units because it is a natural image. When it is necessary to do so, it can be operated to perform interpolation.

Additionally, the video encoding device can set whether to apply block matching to a reference picture on which interpolation was performed on a block-by-block basis. For example, if natural and artificial images are mixed, interpolation is performed on the reference picture, but if the optimal motion vector can be obtained by exploring parts of the artificial image, it is performed in certain units (assuming integer units here). The motion vector can be expressed, and if the optimal motion vector can be obtained by selectively searching a part of the natural image, the motion vector can be expressed in another constant unit (here, assumed to be a 1/4 unit).

FIG. 12 is a diagram for explaining the main process of the video encoding method according to an embodiment of the present invention using syntax in a coding unit.

Referring to FIG. 12, curr_pic_BM_enabled_flag means a flag that allows block matching in the current picture, and can be defined and transmitted in sequence and picture units. At this time, the process of generating a prediction block by performing block matching in the current picture is screen This may mean a case where it operates through inter-prediction. And cu_skip_flag, an inter-screen technology that does not encode residual signals, can be assumed to be a flag supported only in P slices or B slices, excluding I slices. In that case, when curr_pic_BM_enabled_flag is on, block matching (BM) can be supported in the inter-screen prediction mode even in the I slice.

The video encoding method according to this embodiment can support skip when generating a prediction block through block matching to the current picture, and can also support skip in the case of intra-screen technology other than block matching. And depending on conditions, skip may not be supported in the I slice. Whether to skip this can be determined depending on the encoder settings.

For example, when skip is supported in the I slice, the prediction block can be directly restored to the restoration block through block matching without encoding the residual signal by connecting to prediction_unit(), which is a prediction unit, through a specific flag, if(cu_skip_flag). . In addition, the video encoding device can classify the method of using a prediction block through block matching in the current picture as an inter-screen prediction technology, and process such classification through a specific flag, pred_mode_flag.

In addition, the video encoding device according to this embodiment can set the prediction mode to the inter-screen prediction mode (MODE_INTER) if pred_mode_flag is 0, and can set it to the intra-prediction mode (MODE_INTRA) if pred_mode_flag is 1. This is a similar intra-screen technology to the existing one, but in order to distinguish it from the existing structure, it can be classified as an inter-screen technology or an intra-screen technology in I-slice. That is, the video encoding device of this embodiment does not use temporal correlation in the I slice, but can use a temporal correlation structure. part_mode refers to information about the size and shape of blocks divided in the coding unit.

Some examples of syntax used in connection with this embodiment are as follows.

The syntax sps_curr_pic_ref_enabled_flag in the sequence parameter may be a flag for whether to use IBC.

In the picture parameter, the syntax pps_curr_pic_ref_enabled_flag may be a flag for whether to use IBC on a picture basis.

Additionally, it is possible to determine whether to increase NumPicTotalCurr for the current picture depending on the on/off of the syntax called pps_curr_pic_ref_enabled_flag.

In the process of creating reference picture list 0, it is possible to decide whether to include the current picture in the reference picture according to pps_curr_pic_ref_enabled_flag.

In the process of creating reference picture list 1, it can be decided whether to add the current picture to the reference picture list according to pps_curr_pic_ref_enabled_flag.

According to the above configuration, it can be determined whether the current picture will be included in the reference picture list in inter prediction according to the above flags.

Figure 13 is for illustrating an example of supporting symmetric type partitioning or asymmetric type partitioning as in inter-screen prediction when generating a prediction block through block matching in the current picture used in Figure 12. This is an example diagram.

Referring to FIG. 13, in the video encoding method according to this embodiment, when generating a prediction block through block matching in the current picture, 2N×2N, 2N×N, N×2N, N× It can support symmetric partitions such as N, or asymmetric partitions such as nL × 2N, nR × 2N, 2N × nU, and 2N × nD.

FIG. 14 is an example diagram to explain that 2N×2N and N×N can be supported in inter-screen prediction (Inter), like intra prediction (Intra) in FIG. 9. Various block sizes and shapes can be determined depending on the division method of the block division unit.

Referring to FIG. 14, the image encoding method according to this embodiment can support 2N×2N and N×N prediction block types used in existing intra-prediction. This is an example of supporting a square shape in the block division through a quad-tree division method or a division method according to a predefined block candidate group, and in the intra-screen prediction, a binary tree division method or a rectangular shape according to a predefined block candidate group is used. Other block types can also be supported by adding shapes, and settings for this can be set in the encoder.

In addition, when performing block matching to the current picture during intra-screen prediction, whether to apply skip only when matching the current picture (ref_idx = curr), whether to apply it to existing intra-picture prediction, and other (else) partition types (Figure 13 It is possible to set in the encoder whether to apply to new intra-picture prediction even for prediction blocks with (reference). Information about this may be transmitted in units such as sequence, picture, slice, etc.

The subtractor 205 (see FIG. 2) may generate a residual block by subtracting the pixel values of the prediction block generated by the prediction unit 200 from the pixel value of the current block to be encoded to derive a pixel difference value.

The conversion unit 210 (see FIG. 2) receives the residual block, which is a difference value between the current block and the prediction block generated through intra-screen prediction or inter-screen prediction, from the subtractor 205 and converts it into the frequency domain. Through the transformation process, each pixel in the residual block corresponds to the transformation coefficient of the transformation block. The size and shape of the transform block may be the same as or smaller than the coding unit. Additionally, the size and shape of the transform block may be the same as or smaller than the prediction unit. A video encoding device can perform conversion processing by grouping multiple prediction units.

The size or shape of the conversion block can be determined through the block division unit, and conversion in a square or rectangular shape can be supported depending on the block division. The block division operation may be affected depending on the transformation-related settings (size, shape, etc. of supported transformation blocks) supported by the encoder/decoder.

The size and shape of each transform block are determined according to the coding cost for each candidate for the size and shape of the transform block, and segmentation information such as image data of each determined transform block and the size and shape of each determined transform block can be encoded. .

The transformation can be transformed by a one-dimensional transformation matrix. For example, each transformation matrix may be used adaptively in discrete cosine transform (DCT), discrete cosine transform (DST), horizontal, and vertical units. Adaptive use may include making decisions based on several factors, such as block size, block shape, block type (luminance/chrominance), coding mode, prediction mode information, quantization parameters, and coding information of neighboring blocks. there is.

For example, in the case of intra-screen prediction, when the prediction mode is horizontal, a DCT-based transformation matrix can be used in the vertical direction and a DST-based transformation matrix can be used in the horizontal direction. Additionally, when the prediction mode is vertical, a DCT-based transformation matrix can be used in the horizontal direction, and a DST-based transformation matrix can be used in the vertical direction.

The transformation matrix is not limited to that shown in the above description. Information about this may be determined using an implicit or explicit method, and may be determined by one or more of factors such as block size, block shape, encoding mode, prediction mode, quantization parameter, and encoding information of neighboring blocks, or a combination thereof. It can be determined according to , and the related information can be transmitted in units such as sequence, picture, slice, and block.

Here, considering the case of using an explicit method, if there are two or more transformation matrices for the horizontal and vertical directions as candidates, information about which transformation matrix was used for each direction can be sent, or horizontal, Information on which transformation matrices were used in the horizontal and vertical directions can be transmitted by grouping each pair into one pair and having two or more pairs as candidates.

Additionally, partial or full conversion can be omitted considering the characteristics of the image. For example, either or both the horizontal and vertical components can be omitted. This is because if intra-screen or inter-screen prediction is not performed well and there is a large difference between the current block and the predicted block, that is, when the residual component is large, the resulting encoding loss may increase when converting it. This can be determined based on at least one factor or a combination of factors such as coding mode, prediction mode, block size, block shape, block type (luminance/chrominance), quantization parameter, and encoding information of neighboring blocks. . Depending on the above conditions, this can be expressed using an implicit or explicit method, and information about this can be transmitted in units such as sequences, pictures, and slices.

The quantization unit 215 (see FIG. 2) performs quantization of the residual component converted in the conversion unit 210. Quantization parameters are determined on a block basis, and quantization parameters can be set in units such as sequence, picture, slice, and block.

For example, the quantization unit 215 may predict the current quantization parameter using one or two or more quantization parameters derived from neighboring blocks such as the left, top left, top, top right, and bottom left of the current block.

In addition, when the quantization parameter predicted from a neighboring block does not exist, that is, when the block is at the boundary of a picture, slice, etc., the quantization unit 215 performs the difference with the basic parameters transmitted in units of sequence, picture, slice, etc. Values can be output or transmitted. If there is a quantization parameter predicted from a neighboring block, the difference value may be transmitted using the quantization parameter of the corresponding block.

The priority of the block from which the quantization parameter is to be derived can be set in advance, and can be transmitted in units such as sequence, picture, and slice. The residual block can be quantized using Dead Zone Uniform Threshold Quantization (DZUTQ), a quantization weighted matrix, or an improved technique thereof. This may have one or more quantization techniques as candidates and may be determined by encoding mode, prediction mode information, etc.

For example, the quantization unit 215 may set a quantization weight matrix to apply to inter-screen coding, intra-screen coding units, etc., and may also set a different weight matrix depending on the intra-prediction mode. The quantization weight matrix has a size of M Additionally, the quantization unit 215 may select one of several existing quantization methods or may be used under the same settings of the encoder/decoder. Information about this can be transmitted in units such as sequence, picture, and slice.

Meanwhile, the inverse quantization units 220 and 315 and the inverse transform units 225 and 320 shown in FIGS. 2 and 3 may be implemented by reversing the processes of the above transform unit 210 and quantization unit 215. That is, the inverse quantization unit 220 can inversely quantize the quantized transform coefficient generated in the quantization unit 215, and the inverse transform unit 225 can inversely transform the inverse quantized transform coefficient to generate a restored residual block. there is.

The adders 230 and 324 shown in FIGS. 2 and 3 may add the pixel value of the prediction block generated from the prediction unit to the pixel value of the restored residual block to generate a restored block. The restored block may be stored in the encoded and decoded picture buffers 240 and 335 and provided to the prediction unit and filter unit.

The filter unit 235 (see FIG. 2) may apply an in-loop filter such as a deblocking filter, sample adaptive offset (SAO), or adaptive loop filter (ALP) to the restoration block. The deblocking filter can filter the restored block to remove distortion between block boundaries that occurs during the encoding and decoding process. SAO is a filter process that restores the difference between the original image and the restored image as an offset on a pixel basis for the residual block. ALF can perform filtering to minimize the difference between the prediction block and the restored block. ALF can perform filtering based on the comparison value of the block restored through the deblocking filter and the current block.

The entropy encoding unit 245 (see FIG. 2) may entropy encode the quantized transform coefficients through the quantization unit 215. For example, Context Adaptive Variable Length Coding (CAVLC), Context Adaptive Binary Arithmetic Coding (CABAC), Syntax-Based Context Adaptive Binary Arithmetic Coding (SBAC), and Stochastic Interval Partitioning Entropy Coding may be implemented using other encoding schemes ( PIPE) coding techniques can be performed.

The entropy encoder 245 may include a bit stream in which the quantization coefficient is encoded and various information necessary to decode the encoded bit stream in the encoded data. The encoded data may include the encoded block type, quantization coefficient, bit string in which the quantized block is encoded, and information necessary for prediction. In the case of quantization coefficients, two-dimensional quantization coefficients can be scanned in one dimension. The distribution of quantization coefficients may vary depending on the characteristics of the image. In particular, in the case of intra-screen prediction, the distribution of coefficients may have a specific distribution depending on the prediction mode, so the scan method can be set differently.

Additionally, the entropy encoding unit 245 may be set differently depending on the size of the block to be encoded. The scan pattern can be preset or set as a candidate as at least one of various patterns such as zigzag, diagonal, and raster, and can be determined by encoding mode, prediction mode information, etc., under the same settings of the encoder and decoder. can be used Information about this can be transmitted in units such as sequence, picture, and slice.

The size of the quantized block (hereinafter referred to as quantization block) input to the entropy encoding unit 245 may be equal to or smaller than the size of the transform block. Additionally, the quantization block may be divided into two or more sub-blocks, and when divided, the scan pattern in the divided block may be set the same as that of the existing quantization block or may be set differently.

For example, if the scan pattern of an existing quantization block is zigzag, the zigzag pattern can be applied to all subblocks, or the zigzag pattern can be applied to the subblock located in the upper left corner of the block containing the average value (DC) component. and a diagonal pattern can be applied to other blocks. This can also be determined depending on the encoding mode, prediction mode information, etc.

In addition, the starting position of the scan pattern in the entropy encoding unit 245 basically starts from the upper left, but can start from the upper right, lower right, or lower left depending on the characteristics of the image, and selects one of two or more candidates. Information about what has been done can be transmitted in units such as sequence, picture, and slice. As an encoding technology, entropy encoding technology may be used, but is not limited to this.

Meanwhile, the inverse quantization of the inverse quantization unit 220 and the inverse transformation of the inverse transform unit 225 shown in FIGS. 2 and 3 are configured to reverse the positive quantization of the quantization unit 215 and the transformation structure of the transformation unit 210. It can be implemented by combining the basic filter units 235 and 330.

Next, interpolation that can be employed in the video encoding device of the present invention will be briefly described as follows.

In order to increase the accuracy of prediction through block matching, interpolation is performed with decimal resolution, which is more precise than integer units. Such interpolation methods include technologies such as DCT-IF (discrete cosine transform based interpolation filter). DCT-IF technology is used as an interpolation method in HEVC (high efficiency video coding). For example, pixels are generated in units of 1/2 or 1/4 between integers to interpolate a reference picture, and then block A prediction block can be created by performing matching.

Table 1 and Table 2 show the filter coefficients used in the luminance component and chrominance component, respectively. An 8-tap DCT-IF filter is used for the luminance component and a 4-tap DCT-IF filter is used for the chrominance component. Different filters can be applied to the color difference component depending on the color format. In the case of 4:2:0 of YCbCr, the same filter as in Table 2 can be applied, and in 4:4:4, the same filter as in Table 1 or other filters rather than Table 2 can be applied, and 4:2 :2, a horizontal 1D 4-tap filter as shown in Table 2 and a vertical 1D 8-tap filter as shown in Table 1 can be applied.

Figure 15 is a diagram for explaining the process of performing a horizontal 1D filter on pixels at positions a, b, and c of the image (assumed to be x) in the image encoding method according to an embodiment of the present invention.

As shown in FIG. 15, a horizontal 1D filter can be applied to the subpixels at positions a, b, and c (assuming x) between the first pixel (G) and the adjacent second pixel (H). there is. This can be expressed in a formula as follows:

x = ( f1*E + f2*F + f3*G + f4*H + f5*I + f6*J + 32 ) / 64

Next, a vertical 1D filter can be applied to the subpixels at positions d, h, and n (assuming y). This can be expressed in a formula as follows:

y = ( f1*A + f2*C + f3*G + f4*M + f5*R + f6*T + 32 ) / 64

And a 2D separable filter can be applied to the subpixels e, f, g, i, j, k, p, q, and r in the center. Taking subpixel e as an example, the pixels in the direction perpendicular to a are first interpolated and then interpolated using those pixels. Then, the e value can be obtained by performing a horizontal 1D filter as if interpolating a between G and H, and performing a vertical 1D filter on the resulting subpixels. And a similar operation can be performed for color difference signals.

The above explanation is only a partial explanation of interpolation. Filters other than DCT-IF can also be used, and the type of filter and number of tabs applied to each decimal unit can be varied. For example, an 8-tap Kalman filter for 1/2, a 6-tap Wiener filter for 1/4, and a 2-tap linear filter for 1/8. Filter coefficients can be calculated by calculating fixed coefficients or filter coefficients like DCT-IF. It can also be encoded. As above, you can use one interpolation filter for a picture, or you can use different interpolation filters for each area depending on the characteristics of the image, or you can create two or more reference pictures with multiple interpolation filters applied and select one of them. there is.

Different filters may be applied depending on encoding information such as the type of reference picture, temporal layer, and status of the reference picture (eg, whether it is the current picture or not). The above-mentioned information can be set in units such as sequence, picture, and slice, and can be transmitted in that unit.

Next, before explaining in detail the improved technologies for motion estimation, motion compensation, and motion prediction that can be employed in the video encoding method according to this embodiment, the basic meaning of these terms is defined as follows.

Motion estimation refers to the process of predicting the motion of the current block to be encoded by dividing the video frame into small blocks and estimating which block on the temporally previous or subsequent non-encoded frame (reference frame) it has moved from. In other words, motion estimation can be said to be a process of finding the block most similar to the target block when encoding the current block to be compressed. Block-based motion estimation can refer to the process of estimating where a video object or screen processing unit block (macro block, etc.) has moved in time.

Motion compensation retrieves at least some areas of a previously encoded reference image to encode the current image and provides motion information (motion vector, reference picture index) about the optimal prediction block found in the motion estimation process to predict the current image. This means generating a prediction block of the current block based on it. In other words, motion compensation can refer to the process of creating an error block based on the difference between the current block to be encoded and a reference block found to be the most similar block.

Motion prediction means finding a motion vector during encoding for motion compensation. The main technologies for motion prediction include skip, temporal prediction, and spatial prediction. In skip, the motion of the screen is constant, so the size of the motion vector predicted by the video encoding device is zero (0) or the residual is sufficiently small. If it can be ignored, it means omitting the encoding of the corresponding video block and moving on. Temporal prediction can be mainly used for inter-screen prediction, and spatial prediction or inter-view prediction can be mainly used for intra-screen prediction.

Information that comes out through inter-screen prediction may include information that distinguishes the direction of the reference picture list (unidirectional (L0, L1), bidirectional), an index that distinguishes reference pictures in the reference picture list, a motion vector, etc. Because temporal correlation is used, motion information can be encoded efficiently by utilizing the characteristic that the motion vectors of the current block and neighboring blocks are the same or similar.

Figure 16 is an example diagram of a current block and neighboring blocks that can be employed in an image encoding method according to an embodiment of the present invention.

As shown in FIG. 16, the setting of whether to refer to a candidate group for neighboring blocks of the current block may be determined depending on information such as the type of the current picture and temporal identifier, and this information may be used fixedly or It can be transmitted in units such as sequence, picture, slice, etc.

Here, the setting for whether to refer to the candidate group is, for example, using only spatially adjacent blocks (A, B, C, D, E) or temporally adjacent to spatially adjacent blocks (A, B, C, D, E). Settings may include using blocks (H, I, J), or using spatially adjacent blocks (A, B, C, D, E) and spatially distant blocks (F, G).

The following explains the setting “Block matching can be applied to the current picture.”

First, to explain the case of an I picture, for example, spatially adjacent blocks can be given priority first and other blocks can be set as candidates. For example, the availability of reference blocks can be checked in the listed order: E → D → C → B → A → H → I → J. Availability is used to determine whether something is available and can be compared to a preset standard value or compared to other values that have been checked for availability. Availability can be determined based on the coding mode of the candidate block, motion information, location of the candidate block, etc. Motion information may include a motion vector, reference direction, reference picture index, etc.

Since the current picture is an I picture, motion information exists only when the encoding mode is set to intra prediction (hereinafter referred to as INTER) in this embodiment. Therefore, in order of priority, check whether it is INTER first. For example, if n is 3 and E is encoded as INTER, E is excluded from the candidate group and the next D is checked. If D is encoded as INTER, it has motion information because block matching is performed in the current picture, and D is added to the candidate group based on this motion information. Then there are 2 n left. Then, the video encoding device can check the priority again. When the final three candidates are filled in this way, the search for candidates is stopped.

Availability is not only used in coding mode, but can also be used in the case of boundaries such as pictures, slices, and tiles. If borderline, availability is checked as not available. And if the determination result is that the block is the same or similar to an already filled candidate, the corresponding block is excluded from the candidate, and the reference pixel component checks the availability of the next candidate.

Here, INTER is different from the existing inter-screen prediction (inter). That is, the INTER mode of this embodiment only utilizes the inter prediction (inter) structure and generates a prediction block from the current picture, so it is different from inter prediction that generates a prediction block from a reference picture. That is, in the coding mode of this embodiment, the block matching method in the current picture can be classified into INTER mode and intra (same as existing intra) mode and applied.

Hereinafter, motion vector copy (MVC) and motion vector prediction (MVP) will be separately described. The reason is that whether or not the scaling process is included is different.

Motion vector prediction (MVP) will first be explained as follows.

For P picture or B picture

In addition to the candidates mentioned above (A, B, C, D, E, F, G, H, I, J), the explanation will also include temporal candidates (F, G). In this embodiment, it is assumed that candidates are searched spatially, searched temporally, a mixed list is formed and searched, and constant candidates are searched in the described order.

First, prioritize the candidates and check their availability accordingly. The number of motion vector candidates (n) is set to 2, and the priority is assumed to be as stated in parentheses.

For example, when searching spatially, you can classify into the following groups:

Group 1_1 = {A, B, C, I, J}, (C → B → A → I → J)

Group 1_2 = {D, E, H}, (D → E → H)

In this embodiment, among the two groups, group 1_1 includes blocks immediately above, above left, and above right based on the current block, and group 1_2 includes blocks immediately adjacent to the left and not immediately adjacent to the current block. , and includes the blocks in the lower left.

As another embodiment, candidate blocks of motion vectors can be spatially searched by dividing them into three groups. The three groups can be classified as follows.

Group 1_1 = {A, B, C}, (C → B → A)

Group 1_2 = {D, E}, (D → E)

Group 1_3 = {H, I, J}, (J → I →H)

In this embodiment, among the three groups, group 1_1 includes blocks immediately adjacent to the top, adjacent to the upper left, and adjacent to the upper right of the current block, and group 1_2 includes blocks immediately adjacent to the left and immediately adjacent to the current block. Contains the blocks in the lower left, and group 1_3 includes non-adjacent blocks that are one or more blocks apart from the current block.

As another embodiment, candidate blocks of motion vectors can be spatially searched by dividing them into three groups in another way. The three groups can be classified as follows.

Group 1_1 = {B}

Group 1_2 = {D}

Group 1_3 = {A, C, E}, (E → C → A)

In this embodiment, among the three groups, group 1_1 includes blocks located vertically relative to the current block, group 1_2 includes adjacent blocks located horizontally relative to the current block, and group 1_3 includes blocks located in the horizontal direction relative to the current block. Based on the block, the remaining adjacent blocks are included.

As seen above, in a P picture or B picture, there is a lot of reference information such as reference direction or reference picture, so the candidate group can be set accordingly. Candidate blocks whose reference pictures are different from the current block based on the current block can be included in the candidate group, or, conversely, by considering the temporal distance (picture of count, POC) between the reference picture of the current block and the reference picture of the candidate block. The vector of the corresponding block can also be scaled and added to the candidate group. Additionally, a scaled candidate group can be added depending on which picture the reference picture of the current block is. Additionally, if the temporal distance between the reference picture of the current block and the reference picture of the candidate block exceeds a certain distance, it may be excluded from the candidate group, and if it is less than this distance, the scaled block may be included in the candidate group.

The similarity check described above is a process of comparing and determining how similar the motion vector already included in the prediction candidate group is to the motion vector to be newly added. Depending on the definition, it can be set to be true when the x and y components perfectly match, or it can be set to be true when the difference is less than a certain threshold value.

In this embodiment, the condition that the reference picture points to the current picture is used as an example, but this can be extended to the case where the reference picture is not the current picture. For example, you can use a setting such as 'Exclude blocks that use pictures farther than the picture pointed to by the current block.' In this embodiment, even if the current block and the reference picture are different, they can be included in the candidate group through scaling.

mixed list

When bidirectional prediction of the current block is performed and each reference picture in the reference picture list (L0, L1) contains motion information. Availability is checked according to the priority of preset candidates. In this case, the priority is first checked for those encoded with bidirectional prediction.

If each reference picture is different, scaling is performed. When searching spatially and temporally, only bidirectionally predicted blocks were put into the candidate group, and when the maximum number of candidates was not exceeded, blocks encoded by unidirectional prediction among previously performed candidate blocks were put into the preliminary candidate group, and then a combination of these candidates was used. Candidates for bidirectional prediction can be created.

First, let's assume that the motion information for bidirectional prediction of the current block is referenced from reference picture number 1 in L0 and number 0 in L1. In Table 5(a), the motion information of the first candidate block is (mvA ₁ , ref1) and (mvA ₂ , ref0), and the motion information of the second candidate block is (mvB ₁ ', ref1 ) and (mvB ₂ ', ref0). Here, the meaning of the apostrophe (') is a scaled vector. If the number of candidates after completing spatial and temporal search is 2, assuming n is 5, blocks predicted one way in the previous step can be included as preliminary candidates according to the preset priority.

In Table 3(a), since the maximum number of candidates has not yet been filled, new candidates can be added by combining the scaled unidirectional candidates using the remaining motion vectors mvC, mvD, and mvE.

In Table 3(b), the motion information of each unidirectional predicted block is scaled according to the reference picture of the current block. Here, an example of creating a new combination of unidirectional candidates was shown, but a new candidate combination may be possible using the motion information of each of the already added bidirectional reference pictures (L0, L1). This part is not performed in situations such as one-way prediction. Additionally, it is not performed even if the reference picture of the current block is the current picture.

constant candidate

If it is not possible to configure candidate blocks with the maximum number of candidates n (assumed to be 2 in this embodiment) through the above process, constant candidates with preset fixed coordinates can be added. Fixed candidates with fixed coordinates such as (0,0), (-a,0), (-2*a,0), (0,-b) can be used, and the number of fixed candidates is adjusted to the maximum number of candidates. can be set.

You can set fixed coordinates as above, or you can add them as fixed candidates through the process above, such as the average, weighted average, and median of at least two motion vectors included in the candidate group so far. If n is 5 and so far 3 candidates have been registered as candidates {(mvA_x,mvA_y), (mvB_x, mvB_y), (mvC_x, mvC_y)}, fixed candidates with predetermined priorities are selected to fill the remaining 2 candidates. You can have a group of candidates included and add fixed candidates according to their priority. Fixed candidates are, for example, {(mvA_x + mvB_x)/2, (mvA_y + mvB_y)/2), ((mvA_x + mvB_x + mvC_x)/3, (mvA_y + mvB_y + mvC_y)/3), (median(mvA_x) , mvB_x, mvC_x), median(mvA_y, mvB_y, mvC_y)), etc.

Additionally, fixation candidates can be set differently depending on the reference picture of the current block. For example, when the current picture is a reference picture, you can set fixed candidates such as (-a,0), (0,-b), (-2*a,0), and when the current picture is not a reference picture, It can also be set as (0,0), (-a,0), (average(mvA_x, …), average(mvA_y, …)). The corresponding information can be set in advance in the encoder or decoder, or can be transmitted in units such as sequences, pictures, and slices.

Below, motion vector copy (MVC) will be described in more detail.

Description of P picture or B picture

In this embodiment, it is assumed that temporal candidates (F, G) are also included. Candidates include A, B, C, D, E, F, G, H, I, and J. Although the search order is not set, it is assumed here that MVC candidates are searched spatially, searched temporally, searched by constructing a mixed list, and constant candidates are added.

In other words, the part described above also arbitrarily determines the search order, and does not use a predetermined order. Set priorities and check availability accordingly. Assume n is 5 and the priority is the same as in parentheses.

In the following description, only the differences in motion vector prediction (MVP) described above will be described. The MVP part can be written by attaching the following content, excluding the part about scaling from the previous part. For spatial candidates, availability can be checked without the scaling process. However, similar to MVC, the type of reference picture, the distance to the current picture or the reference picture of the current block, etc. can be excluded from the candidate group.

If a mixed list exists, a candidate for bidirectional prediction can be created by combining the candidates added so far, as shown in Table 4 below.

As shown in Table 4(a), a new candidate can be added to the motion vector candidate group by combining a candidate using the reference list LO and a candidate using the reference list L1. If the predetermined number of motion vectors of 5 is not met, a new candidate can be added by combining the next candidate in L0 and the candidate using L1, as shown in Table 4(b).

As above, encoding can be done according to modes such as MVP and MVC that find candidates for optimal motion information.

In skip mode, encoding can be done using MVC. That is, after skip flag processing, information about the optimal motion vector candidate can be encoded. If there is only one candidate, this part can be omitted. Instead of separately encoding the motion vector difference, etc., the residual component, which is the difference between the current block and the prediction block, can be encoded through processes such as transformation and quantization.

If it is not a skip, it first goes through a confirmation process as to whether to process the motion information through MVC as a priority, and if it is correct, information about the candidate group of the optimal motion vector can be encoded. If you do not want to process motion information through MVC, you can process motion information through MVP. In the case of MVP, information about optimal motion vector candidates can be encoded. Here, if there is only one candidate, motion information processing can be omitted. In addition, information such as the difference value with the motion vector of the current block, reference direction, and reference picture index can be encoded, residual components can be obtained, and then encoded through processes such as transformation and quantization.

Details about the codec, such as entropy and post-processing filtering, are omitted to avoid overlap with the above description.

FIG. 17 is a diagram for explaining a case in which motion vector precision is different for each block in FIG. 15. FIG. 18 is a diagram illustrating a case in which the motion vector precision of a block is determined according to the interpolation precision of a reference picture in the video encoding and decoding method according to an embodiment of the present invention.

In Figure 17, it is assumed that the number (2) in parentheses written below each picture is interpolation precision depth information. That is, in Figure 17, the interpolation precision of each picture including the current picture (t) and the reference pictures (t-1, t-2, t-3) is fixed, but the motion vector precision is adaptively adjusted on a block basis. This is a case where the motion vector precision of a block is determined according to the interpolation precision of the reference picture. In Figure 18, the depth information of the interpolation precision is

In each of the two cases described above, a motion vector prediction value can be generated based on the above-described 'selection of motion information prediction candidate' or 'selection of motion vector candidate'.

In this embodiment, the maximum number of candidates used for motion vector prediction can be set to 3. Here, candidate blocks may be limited to blocks to the left, above, and above right of the current block. Candidate blocks can be set as nearby blocks that are not adjacent spatially as well as temporally and spatially.

Figure 19 is a flowchart of an image encoding and decoding method using motion vector difference according to an embodiment of the present invention.

Referring to FIG. 19, the image decoding method according to this embodiment can configure a motion information prediction candidate group in the prediction unit and then obtain a difference value with the motion vector of the current block.

To explain in more detail, first, the motion vector of a block belonging to the candidate group can be changed to match the precision unit of the motion vector of the current block (S192).

Next, a motion vector scaling process can be performed according to the distance between the motion vector of the current block and the reference picture, that is, the distance between the current picture and the reference picture, and the distance between the picture of the block belonging to the candidate group and the reference picture of the block ( S194).

Next, the prediction unit can obtain the motion vector difference between the current block and the corresponding block based on the motion vectors scaled into one picture (S196).

The process of obtaining motion vector differential values through the above-described steps will be explained in more detail through the example below.

Figures 20 to 25 are diagrams for explaining the process of calculating motion vector difference values for various cases when interpolation precision is determined on a block-by-block basis in the video encoding and decoding method according to an embodiment of the present invention.

When interpolation precision is determined on a per-picture basis

As shown in FIG. 20, it is assumed that three blocks surrounding the current block are candidate blocks for motion information encoding. Here, the interpolation precision of the reference picture is an integer (integer, Int) for the current picture (t), 1/4 for the first reference picture (t-1), 1/4 for the second reference picture (t-2), and 1/4 for the second reference picture (t-2). 3 Assume that the reference picture (t-3) is 1/2.

In FIG. 20, the blocks of (A1) can be expressed like the blocks of (A2) according to each motion vector precision.

If blocks pointing to reference pictures different from the reference picture of the current block are set to be used as candidates through scaling considering the distance between reference pictures, the block in (A2) is used as the reference picture of the candidate blocks in (A3) and the current block. Scaling can be performed considering the distance of the reference picture. Blocks of the same reference picture are not scaled. When performing scaling, if the distance between the current picture and the reference picture is smaller, you can select one of rounding, rounding up, or down to apply. Then, like the block in (A4), the motion vector of the candidate block can be adjusted considering the motion vector precision of the current block. In this embodiment, it is assumed that the upper right block is selected as the optimal candidate.

In Figure 20, based on the current block located in the bottom center, only the upper right block is 1/2 units, so to change it to 1/4 units, the denominator and numerator were multiplied by 2, but if the opposite situation occurs, for example, 1/8 When adjusting from a unit to a 1/4 unit, the optimal candidate (MVcan) can be selected from among the candidates, then the difference value (MVD) with the motion vector (MVx) of the current block is obtained and encoded. This can be expressed in a formula as follows:

MVD = MVx - MVcan → (2/4, 1/4)

In the above-described embodiment, scaling is performed first and precision is adjusted later, but the present invention is not limited to this, and precision may be performed first and then scaling.

When interpolation precision is determined on a block basis, it is assumed that the reference pictures of the current block and candidate blocks are the same.

As shown in FIG. 21, three blocks around the current block are candidate blocks for motion information encoding. Here, it is assumed that the interpolation precision of the reference pictures is all the same at 1/8. The blocks in (B1) can be expressed like the blocks in (B2) according to each motion vector precision. Since the reference picture of the current block and the reference picture of the candidate block are the same, the scaling process is omitted.

Next, the motion vector of each candidate block in (B2) can be adjusted considering the motion vector precision of the current block like the blocks in (B3).

Then, after selecting the optimal candidate (MVcan) among the candidates, the difference value (MVD) with the motion vector (MVx) of the current block is obtained and encoded. In Figure 21, it is assumed that based on the current block located in the center of the line below, the block above it is selected as the optimal candidate. This can be expressed in a formula as follows:

MVD = MVx - MVcan → (1/2, -1/2)

When interpolation precision is determined on a block-by-block basis, the reference pictures of the current block and candidate blocks are different.

In this embodiment, it is assumed that the interpolation precision of all reference pictures is the same. For example, the interpolation precision of the reference picture may all be 1/8.

Referring to FIG. 22, the blocks of (C1) can be expressed like the blocks of (C2) according to each motion vector precision. And since the reference picture of the current block and the reference picture of the candidate block are different, the blocks of (C3) can be obtained by performing scaling.

Next, considering the motion vector precision of the current block among the blocks in (C3), the motion vector of the candidate block can be adjusted like the blocks in (C4). Afterwards, the optimal candidate (MVcan) can be selected from among the candidates, then the difference value (MVD) with the motion vector (MVx) of the current block is obtained and encoded. In Figure 22, it is assumed that the upper right block is selected as the optimal candidate based on the current block located in the lower center. This can be expressed in a formula as follows:

MVD = MVx - MVcan → (1/4, 3/4)

When interpolation precision is determined per picture, temporally located candidate blocks are also included.

In this embodiment, it is assumed that the interpolation precision of all reference pictures is the same. For example, assume that the interpolation precision of the reference picture is 1/2. As shown in FIG. 23, the blocks of (D1) can be expressed like the blocks of (D2) according to each motion vector precision. In the case of a block that uses the current picture as a reference picture, the block can be excluded (invalid) from the candidate group because it is not a general motion prediction.

Next, since the reference picture of the current block and the reference picture of the candidate block are different, scaling can be performed to obtain the blocks of (D3). At this time, it is assumed that the picture of the co-located block is a specific reference picture (t-1).

Afterwards, the motion vector of the candidate block can be adjusted like the block in (D4) by considering the motion vector precision of the current block located in the bottom center among the blocks in (D3). Then, after selecting the optimal candidate (MVcan) from the candidate group, the difference value (MVD) with the motion vector (MVx) of the current block can be obtained and encoded. In this embodiment, it is assumed that the co-located block is selected as the optimal candidate. The above-described configuration can be expressed in a formula as follows.

MVD = MVx - MVcan → (2/4, 2/4)

When interpolation precision is determined on a block-by-block basis, when the current block refers to the current picture

In this embodiment, it is assumed that the interpolation precision of all reference pictures is the same. For example, the interpolation precision of the reference picture may be 1/4.

As shown in FIG. 24, the blocks of (E1) can be expressed like the blocks of (E2) according to each motion vector precision. In the case of a block that uses the current picture located in the bottom center as a reference picture, since block matching is performed on the current picture rather than general motion prediction, only the upper block that performs the same motion will remain as a candidate, and the rest will be excluded (invalid) from the candidate group. You can.

Because the reference picture of the current block and the reference picture of the candidate block are the same, scaling is not performed. Then, considering the motion vector precision of the current block, the motion vector of the candidate block can be adjusted like the blocks in (E3).

Then, after selecting the optimal candidate (MVcan) from the candidate group, the difference value (MVD) with the motion vector (MVx) of the current block can be obtained and encoded. In this embodiment, it is assumed that the upper block is selected as the optimal candidate. The result of the above-mentioned process is expressed in a formula as follows.

MVD = MVx - MVcan → (-5/2, -1/2)

Next, the image encoding and decoding method according to this embodiment can utilize information on neighboring reference blocks to encode the current block. In this regard, as shown in FIG. 25, the encoding mode and corresponding information of neighboring blocks can be used, focusing on the current block.

In FIG. 25, the upper blocks E5 represent a case where the current picture is an I picture, and represent a case where a prediction block is generated and encoded through block matching in the current picture.

The intra-screen prediction of this embodiment applies the method of this embodiment in addition to the existing extrapolation-based method, and can be expressed as INTRA, and may include Inter when block matching is performed in the current picture.

To be more specific, when using Inter, information such as motion vectors and reference pictures can be utilized. If the neighboring block encoding modes are the same, the information of the corresponding blocks (E5, E6, and E7) can be used to encode the current block in the order described, and blocks that are not immediately adjacent, for example, blocks of (E5), can be used to encode the current block. When checking the coding mode of a block and adding it as a candidate block, the coding mode is inter and a reference picture (ref) is additionally checked. Blocks with ref indicated as t can also be used for coding of the current block. When the precision of the motion vector is determined according to the interpolation precision of the reference picture, for example, when the interpolation precision of the current picture is in integer units, the motion vectors of blocks encoded as Inter can be expressed in integer units.

(E6) represents the case when the current picture is P or B. Information on neighboring reference blocks can be used to encode the current block. If the precision of the motion vector is determined according to the interpolation precision of the reference picture, the motion vector precision of each block can be set according to the interpolation precision of each reference picture.

(E7) is a case where the current picture is P or B, and the current picture is included as a reference picture. The precision of the motion vector of each block may be determined depending on the interpolation precision of the reference block.

(E8) is a case where information from a co-located block is used to encode the current block. The precision of the motion vector of each block can be determined according to the interpolation precision of each reference picture.

In subsequent encoding and decoding processes, a process of encoding motion information may be applied. However, in this embodiment, the differential value of the motion vector is encoded with adaptive precision. That is, this may include cases where the interpolation precision of the reference picture is constant and the motion vector precision between blocks is constant. That is, motion vector precision can be determined according to the interpolation precision of the reference picture.

Figures 26 to 29 are diagrams for explaining the process of expressing the precision of motion vector difference values in the video encoding and decoding method according to an embodiment of the present invention.

In this embodiment, the motion vector of the optimal candidate block among several candidate blocks is (a, b) and the motion vector of the current block is (c, d). For convenience of explanation, it is assumed that the reference pictures are the same.

As shown in (F1) of FIG. 26, it is assumed that the interpolation precision of the reference picture is 1/4. The motion vector difference values (c-a, d-b) of the current block and the optimal candidate block must be obtained and sent to the decoder. To this end, in this embodiment, the blocks of (F1) can be expressed like the blocks of (F2) according to each motion vector precision. Since the reference picture is the same and the motion vector precision of each block is the same, the blocks in (F2) can be used for motion vector coding.

That is, as shown in (F2), (c,d), the motion vector of the current block, is (21/4, 10/4), and if the optimal candidate block is the left block, the motion vector of the left block is ( Since a,b) are (13/4, 6/4), (c-d, d-b), which represents their difference, becomes (8/4, 4/4). Since the motion vector precision is 1/4 units, the binary index can be processed using 8 and 4 as shown in Table 5 below. However, if information about the precision of the difference value of the motion vector is sent, this precision can be processed by allocating a shorter bit index.

If information is transmitted that the motion vector differential value of the current block has a precision of integer units, a shorter bin index can be used by moving the existing bin index of 8 and 4 to integer units and using 2 and 1. When using various binarization methods, for example, when unary binarization is used, 111111110 + 11110 bits must be transmitted for the above-mentioned difference value (8/4, 4/4), but information about the precision of the difference value is required. If you transmit 110 + 10 bits corresponding to (2/1, 1/1), you can use shorter bits and increase encoding efficiency.

If the precision of the motion vector difference value is expressed in the manner shown in Figure 27, in the above case, information about 11 (integer) and information about (2/1, 1/1) for the difference value can be sent.

In the above-described embodiment, motion vector difference precision was applied to the x and y components together, but it is also possible to apply them individually.

*In Figure 28, (G1) can be expressed as (G2) according to the precision of each motion vector. Since the reference picture is the same and the motion vector precision of each block is the same, (G2) can be used directly for motion vector coding.

Assuming that the optimal candidate block is the block to the left of the current block, the difference value for this can be expressed as (8/4, 5/4). If the precision of the motion vector difference value is applied to x and y of the optimal candidate block expressed in this way, x can be expressed as 2/1 in integer units, and y can be expressed as 5/4 in 1/4 units.

If the precision of the motion vector difference value is expressed as a tree below the table in Figure 27, information about 11 (integer) for x and information about 0 (1/4) for y, and each difference value information Just send 2/1 and 5/4 (see Figure 29).

That is, in the above case, as shown in FIG. 29, the range for difference value precision can be included in the candidate group from the maximum precision (1/4) to the minimum precision (integer). Of course, the present invention is not limited to such a configuration, and the candidate group can be configured in various ways. It may be configured to include at least two units of precision from the maximum precision (for example, 1/8) to the minimum precision.

In the above-described embodiment, the video encoding method may be used instead of the video decoding method when using the motion vector difference described above to decode the encoded video. In addition, of course, the image encoding/decoding method can be performed by an image processing device or an image encoding and decoding device including at least one means for encoding and decoding or a component that performs a function corresponding to such means.

According to the above-described embodiment, international codecs such as MPEG-2, MPEG-4, H.264 or other codecs in which intra-screen prediction technology is used, media using these codecs, and high performance usable overall in the video industry Highly efficient video encoding and decoding technology can be provided. In addition, in the future, it is expected to be applied to the image processing field using standard codecs such as the current high-efficiency video coding technology (HEVC) and H.264/AVC and intra-screen prediction.

Although the present invention has been described above with reference to preferred embodiments, those skilled in the art may make various modifications and changes to the present invention without departing from the spirit and scope of the present invention as set forth in the claims below. You will understand that you can do it.

Claims (11)

determining motion vector precision of the current block based on motion vector precision information of the current block;
determining a motion vector of the current block from motion information of the current block, based on the motion vector precision of the current block; and
Predicting the current block based on the motion vector of the current block,
The motion vector precision of the current block is determined depending on whether the reference picture of the current block is the current picture,
An image decoding method wherein the motion vector precision of each block of the current picture including the current block is adaptively determined on a block-by-block basis.
According to paragraph 1,
The video decoding method is,
In the parameter set, obtaining a parameter set flag indicating whether the current picture can be a reference picture of the current block,
An image decoding method, characterized in that whether the reference picture of the current block is the current picture is determined based on the parameter set flag.
According to paragraph 2,
A video decoding method, characterized in that the parameter set flag is obtained from a sequence parameter set.
According to paragraph 1,
The video decoding method is,
determining whether a reference picture of the current block is the current picture; and
If the reference picture of the current block is not the current picture, obtaining reference picture index information indicating the reference picture from a reference picture list, and determining the reference picture of the current block based on the reference picture index information. Contains,
When the reference picture of the current block is the current picture, acquisition of the reference picture index information is omitted.
According to paragraph 1,
When the reference picture of the current block is the current picture, the motion vector precision of the current block is determined with integer precision.
determining a motion vector precision of a current block and a motion vector of the current block;
predicting the current block based on the motion vector of the current block; and
Encoding motion vector precision information of the current block indicating the motion vector precision of the current block,
The motion vector precision of the current block is determined depending on whether the reference picture of the current block is the current picture,
An image encoding method wherein the motion vector precision of each block of the current picture including the current block is adaptively determined on a block basis.
According to clause 6,
The video encoding method is,
An image encoding method further comprising encoding a parameter set flag indicating whether the current picture can be a reference picture of the current block into a parameter set.
In clause 7,
A video encoding method, characterized in that the parameter set flag is encoded in a sequence parameter set.
According to clause 6,
The video encoding method is,
determining whether a reference picture of the current block is the current picture; and
When the reference picture of the current block is not the current picture, encoding reference picture index information indicating the reference picture in a reference picture list,
An image encoding method, wherein when the reference picture of the current block is the current picture, encoding of the reference picture index information is omitted.
According to clause 6,
When the reference picture of the current block is the current picture, the motion vector precision of the current block is determined with integer precision.
In the bitstream storage method,
Generating a bitstream including motion information of the current block and motion vector precision information of the current block indicating motion vector precision of the current block; and
Including storing the bitstream,
The motion vector precision of the current block is determined based on the motion vector precision information of the current block,
The motion vector of the current block is determined from the motion information of the current block based on the motion vector precision of the current block,
The current block is predicted based on the motion vector of the current block,
The motion vector precision of the current block is determined depending on whether the reference picture of the current block is the current picture,
A bitstream storage method, wherein the motion vector precision of each block of the current picture including the current block is adaptively determined on a block basis.