WO2020060342A1 - Method and device for processing image signal by using inter prediction - Google Patents

Abstract

Embodiments of the present specification provide a method and a device for processing a video signal by using inter prediction. A method for processing an image signal, according to an embodiment of the present specification, comprises the steps of: acquiring a motion vector difference (MVD) for a current block; determining the resolution of the MVD on the basis of information about an adaptive motion vector resolution (AMVR) and information about a reference picture; scaling the MVD according to the resolution; determining a motion vector for the current block on the basis of the scaled MVD and a motion vector predictor; and generating a prediction sample for the current block on the basis of the motion vector.

Description

Method and apparatus for processing video signal using inter prediction

The present specification relates to a method and apparatus for processing an image signal, and more particularly, to a method and apparatus for encoding or decoding an image signal by performing inter prediction.

Compression coding refers to a series of signal processing techniques for transmitting digitized information through a communication line or storing it in a form suitable for a storage medium. Media such as video, image, and audio may be the subject of compression encoding, and a technique for performing compression encoding on an image is referred to as video image compression.

Next-generation video content will have the characteristics of high spatial resolution, high frame rate and high dimensionality of scene representation. In order to process such content, a huge increase in terms of memory storage, memory access rate and processing power will be produced.

Therefore, it is necessary to design a coding tool to process next-generation video content more efficiently. In particular, the video codec standard after the high efficiency video coding (HEVC) standard requires a prediction technique capable of generating prediction samples accurately while using resources more efficiently.

Embodiments of the present specification are to provide an image signal processing method and apparatus that can efficiently use data resources in an inter prediction process.

The technical problems to be achieved in the present specification are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by a person having ordinary knowledge in the technical field to which this specification belongs from the following description. Will be able to.

A method for decoding an image signal according to an embodiment of the present specification includes: obtaining a motion vector difference (MVD) for a current block, and adaptive motion vector resolution (AMVR) ) Based on information about the reference picture and information about the reference picture, scaling the MVD according to the resolution, and scaling the MVD and a motion vector predictor. And determining a motion vector for the current block based on the result, and generating a prediction sample for the current block based on the motion vector.

Further, the information on the AMVR includes an AVMR flag indicating whether to use a predefined native resolution and AMVR accuracy information indicating a unit of the AVMR, and the information on the reference picture includes: a picture of POC (picture of the reference picture) order count).

In addition, the step of determining the resolution of the MVD may include comparing the POC of the current block with the POC of the reference picture after parsing the AMVR flag or AMVR accuracy information.

In addition, the determining of the resolution of the MVD may include determining whether the AMVR is applied using the AMVR flag, and when the AMVR is not applied, the POC of the current block and the POC of the reference picture are the same. It may include the step of determining the resolution of the MVD based on whether or not.

In addition, when the AMVR flag is 1 and the AMVR accuracy information indicates a 4-pel unit, the resolution of the MVD is determined to be 4, the AVMR flag is 1, and the AMVR accuracy information is an integer unit. In case of indicating, the resolution of the MVD is determined to be 1, if the AVMR flag is 0, and the POC of the current block is different from the POC of the reference picture, the resolution of the MVD is determined to be 1/4, and the If the AVMR flag is 0 and the POC of the current block is the same as the POC of the reference picture, the resolution of the MVD may be determined to be 1.

In addition, when the AMVR flag is 1 and the AMVR accuracy information indicates a 4-pel unit, the resolution of the MVD is determined to be 4, the AVMR flag is 1, and the AMVR accuracy information is an integer unit. When the POC of the current block and the POC of the reference picture are different while indicating, the resolution of the MVD is determined to be 1, the AVMR flag is 1, and the AMVR accuracy information indicates an integer unit, and the POC of the current block is When the POC of the reference picture is the same, the resolution of the MVD is determined to be 4, and when the AVMR flag is 0 and the POC of the current block and the POC of the reference picture are the same, the resolution of the MVD is determined to be 1, When the AVMR flag is 0 and the POC of the current block is different from the POC of the reference picture, the resolution of the MVD may be determined to be 1/4.

In addition, when the AMVR flag is 1 and the AMVR accuracy information indicates a 4-pel unit and the POC of the current block and the POC of the reference picture are the same, the resolution of the MVD is determined to be 16, When the AMVR flag is 1 and the AMVR accuracy information indicates a 4-pel unit and the POC of the current block is different from the POC of the reference picture, the resolution of the MVD is determined to be 4, and the AMVR When the flag is 1 and the AMVR accuracy information indicates an integer unit, and the POC of the current block and the POC of the reference picture are the same, the resolution of the MVD is determined to be 4, the AMVR flag is 1 and the AMVR accuracy information is When the POC of the current block and the POC of the reference picture are different while indicating an integer unit, the resolution of the MVD is determined to be 1, the AMVR flag is 0, the POC of the current block and the true When the POC of the crude picture is the same, the resolution of the MVD is determined to be 1, and when the AMVR flag is 0 and the POC of the current block is different from the POC of the reference picture, the resolution of the MVD may be determined to be 1/4. have.

An apparatus for decoding an image signal according to another embodiment of the present invention includes a memory for storing the image signal, a processor coupled with the memory, and the processor includes motion vector difference values for the current block (motion Vector difference (MVD) is obtained, the resolution of the MVD is determined based on information on adaptive motion vector resolution (AMVR) and information on a reference picture, and the MVD is scaled according to the resolution And, it is set to determine a motion vector for the current block based on the scaled MVD and a motion vector predictor, and generate a prediction sample for the current block based on the motion vector.

According to an embodiment of the present specification, by scaling a motion vector in consideration of current picture referencing (CPR), data resources can be efficiently used in inter prediction.

The effects obtained in the present specification are not limited to the above-mentioned effects, and other effects not mentioned will be clearly understood by those skilled in the art from the following description. .

BRIEF DESCRIPTION OF THE DRAWINGS The accompanying drawings, which are included as part of the detailed description to aid understanding of the present specification, provide embodiments of the present specification, and describe the technical features of the present specification together with the detailed description.

1 shows an example of a video coding system according to an embodiment of the present specification.

2 is an embodiment to which the present specification is applied, and shows a schematic block diagram of an encoding device in which encoding of a video / image signal is performed.

3 is an embodiment to which the present specification is applied, and shows a schematic block diagram of a decoding apparatus in which decoding of a video signal is performed.

4 is an embodiment to which the present specification is applied, and is a structural diagram of a content streaming system.

5 shows an example of a picture divided into coding tree units (CTUs).

6 shows an example of multi-type tree splitting modes according to an embodiment of the present specification.

7 shows an example of a signaling mechanism of partition partition information of a quadtree with nested multi-type tree structure with a multitype tree.

8 exemplarily shows that a CTU is divided into multiple coding units (CUs) based on a quadtree and nested multi-type tree structure.

9 shows an example of a case where TT (ternary tree) partitioning is restricted for a 128x128 coding block.

10 exemplarily shows redundant partition patterns that may occur in binary tree partitions and ternary tree partitions.

11 and 12 illustrate a video / video encoding procedure based on inter prediction and an inter prediction unit in an encoding device.

13 and 14 illustrate a video / video decoding procedure based on inter prediction and an inter prediction unit in a decoding apparatus.

15 shows an example of a spatial merge candidate configuration for the current block.

16 shows an example of a flowchart for configuring a merge candidate list according to an embodiment of the present specification.

17 shows an example of a flowchart for constructing a prediction candidate list (MVP candidate list).

18 shows another example of a decoding apparatus according to an embodiment of the present specification.

19 shows a usable area of motion compensation when using motion compensation in the current image.

20 shows an example of a decoding flowchart according to a history-based motion vector prediction (HMVP) method.

21 and 22 show an example of a process of updating a table in the HMVP method.

23 shows an example of a flowchart for adaptively adjusting the resolution of a motion vector.

24 illustrates an example of a process of scaling a motion vector difference (MVD) using adaptive motion vector resolution (AMVR) according to an embodiment of the present specification.

25 illustrates another example of a process of scaling an MVD using an AMVR according to an embodiment of the present specification.

26 illustrates another example of a process of scaling an MVD using an AMVR according to an embodiment of the present specification.

27 shows another example of a process of scaling an MVD using an AMVR according to an embodiment of the present specification.

28 illustrates an example of a method of constructing a merge candidate list according to an embodiment of the present specification.

29 illustrates an example of a method of constructing a motion vector prediction list according to an embodiment of the present specification.

30 shows an example of a flowchart for performing HMVP according to an embodiment of the present specification.

31 shows another example of a flowchart for performing HMVP according to an embodiment of the present specification.

32 shows an example of a flowchart for performing prediction according to an embodiment of the present specification.

33 shows an example of a block diagram of an apparatus for processing an image signal according to an embodiment of the present specification.

34 is a diagram schematically showing an example of a service system including a digital device.

35 is a block diagram illustrating a digital device according to an embodiment.

36 is a configuration block diagram illustrating another embodiment of a digital device.

37 is a configuration block diagram illustrating another embodiment of a digital device.

38 is a block diagram illustrating a detailed configuration of the control unit of FIGS. 35 to 37 to illustrate one embodiment.

39 is a diagram illustrating an example in which a screen of a digital device displays a main image and a sub image simultaneously, according to an embodiment.

Hereinafter, preferred embodiments according to the present specification will be described in detail with reference to the accompanying drawings. DETAILED DESCRIPTION The following detailed description, together with the accompanying drawings, is intended to describe exemplary embodiments of the present specification, and is not intended to represent the only embodiments in which the present specification may be practiced. The following detailed description includes specific details to provide a thorough understanding of the specification. However, one of ordinary skill in the art knows that the present specification may be practiced without these specific details.

In some cases, in order to avoid obscuring the concept of the present specification, well-known structures and devices may be omitted, or block diagrams centered on core functions of each structure and device may be illustrated.

In addition, the terms used in the present specification have been selected as general terms that are currently widely used as much as possible, but in a specific case, description will be made using terms arbitrarily selected by the applicant. In such a case, since the meaning is clearly described in the detailed description of the relevant part, it should not be interpreted simply by the name of the term used in the description of this specification, and it is to be clarified that the meaning of the term should be understood and interpreted .

Certain terms used in the following description are provided to help understanding of the present specification, and the use of these specific terms may be changed to other forms without departing from the technical spirit of the present specification. For example, in the case of signals, data, samples, pictures, frames, blocks, etc., each coding process may be appropriately replaced and interpreted.

Hereinafter, the term 'processing unit' in the present specification means a unit in which encoding / decoding processing processes such as prediction, transformation, and / or quantization are performed. Also, the processing unit may be interpreted to include a unit for a luminance component and a unit for a chroma component. For example, the processing unit may correspond to a block, a coding unit (CU), a prediction unit (PU), or a transform unit (TU).

Further, the processing unit may be interpreted as a unit for a luminance component or a unit for a color difference component. For example, the processing unit may correspond to a coding tree block (CTB), a coding block (CB), a PU or a transform block (TB) for the luminance component. Alternatively, the processing unit may correspond to CTB, CB, PU or TB for the color difference component. Further, the present invention is not limited thereto, and the processing unit may be interpreted to include a unit for a luminance component and a unit for a color difference component.

In addition, the processing unit is not necessarily limited to square blocks, and may be configured in a polygonal shape having three or more vertices.

In addition, hereinafter, a pixel, a pixel, or a coefficient (transformation coefficient or transformation coefficient that has undergone first-order transformation) is referred to as a sample in the present specification. And, using a sample may mean that a pixel value, a pixel value, or a coefficient (a transform coefficient or a transform coefficient that has undergone first-order transformation) is used.

1 shows an example of a video coding system according to an embodiment of the present specification.

The video coding system can include a source device 10 and a receiving device 20. The source device 10 may transmit the encoded video / video information or data to the receiving device 20 through a digital storage medium or a network in a file or streaming form.

The source device 10 may include a video source 11, an encoding device 12, and a transmitter 13. The receiving device 20 may include a receiver 21, a decoding device 22 and a renderer 23. The encoding device 10 may be referred to as a video / video encoding device, and the decoding device 20 may be referred to as a video / video decoding device. The transmitter 13 may be included in the encoding device 12. The receiver 21 may be included in the decoding device 22. The renderer 23 may include a display unit, and the display unit may be configured as a separate device or an external component.

The video source may acquire a video / image through a capture, synthesis, or generation process of the video / image. The video source may include a video / image capture device and / or a video / image generation device. The video / image capture device may include, for example, one or more cameras, a video / image archive including previously captured video / images, and the like. The video / image generating device may include, for example, a computer, a tablet and a smart phone, and the like (electronically) to generate the video / image. For example, a virtual video / image may be generated through a computer or the like, and in this case, the video / image capture process may be replaced by a process in which related data is generated.

The encoding device 12 may encode an input video / image. The encoding apparatus 12 may perform a series of procedures such as prediction, transformation, and quantization for compression and coding efficiency. The encoded data (encoded video / video information) may be output in the form of a bitstream.

The transmitting unit 13 may transmit the encoded video / video information or data output in the form of a bitstream to a receiving unit of a receiving device through a digital storage medium or a network in a file or streaming format. Digital storage media include universal serial bus (USB), secure digital (SD), compact disk (CD), digital video disk (DVD), bluray, hard disk drive (HDD), and solid state drive (SSD). It may include various storage media. The transmission unit 13 may include an element for generating a media file through a predetermined file format, and may include an element for transmission through a broadcast / communication network. The receiver 21 may extract the bitstream and transmit it to the decoding device 22.

The decoding apparatus 22 may decode a video / image by performing a series of procedures such as inverse quantization, inverse transformation, and prediction corresponding to the operation of the encoding apparatus 12.

The renderer 23 may render the decoded video / image. The rendered video / image may be displayed through the display unit.

2 is an embodiment of the present specification, and shows a schematic block diagram of an encoding device in which encoding of a video / image signal is performed. The encoding device 100 of FIG. 2 may correspond to the encoding device 12 of FIG. 1.

Referring to FIG. 2, the encoding apparatus 100 includes an image segmentation unit 110, a subtraction unit 115, a conversion unit 120, a quantization unit 130, an inverse quantization unit 140, and an inverse conversion unit 150, Includes an adder 155, a filtering unit 160, a decoded picture buffer (DPB) 170, an inter prediction unit 180, an intra prediction unit 185, and an entropy encoding unit 190 Can be. The inter prediction unit 180 and the intra prediction unit 185 may be collectively called a prediction unit. That is, the prediction unit may include an inter prediction unit 180 and an intra prediction unit 185. The transform unit 120, the quantization unit 130, the inverse quantization unit 140, and the inverse transform unit 150 may be included in a residual processing unit. The residual processing unit may further include a subtraction unit 115. The above-described image segmentation unit 110, subtraction unit 115, conversion unit 120, quantization unit 130, inverse quantization unit 140, inverse conversion unit 150, addition unit 155, filtering unit 160 ), The inter prediction unit 180, the intra prediction unit 185 and the entropy encoding unit 190 may be configured by one hardware component (for example, an encoder or processor) according to an embodiment. Also, the decoded picture buffer 170 may be configured by one hardware component (eg, a memory or digital storage medium) according to an embodiment.

The image splitter 110 may divide the input image (or picture, frame) input to the encoding apparatus 100 into one or more processing units. As an example, the processing unit may be referred to as a coding unit (CU). In this case, the coding unit may be recursively divided according to a quad-tree binary-tree (QTBT) structure from a coding tree unit (CTU) or a largest coding unit (LCU). For example, one coding unit may be divided into a plurality of coding units of a deeper depth based on a quad tree structure and / or a binary tree structure. In this case, for example, a quad tree structure may be applied first, and a binary tree structure may be applied later. Alternatively, a binary tree structure may be applied first. The coding procedure according to the present specification may be performed based on the final coding unit that is no longer split. In this case, the maximum coding unit may be directly used as a final coding unit based on coding efficiency according to image characteristics, or the coding unit may be recursively divided into coding units having a lower depth than optimal if necessary. The coding unit of the size of can be used as the final coding unit. Here, the coding procedure may include procedures such as prediction, transformation, and reconstruction, which will be described later. As another example, the processing unit may further include a prediction unit (PU) or a transformation unit (TU). In this case, the prediction unit and transform unit may be partitioned or partitioned from the above-described final coding unit, respectively. The prediction unit may be a unit of sample prediction, and the transformation unit may be a unit for deriving a transform coefficient and / or a unit for deriving a residual signal from the transform coefficient.

The unit may be used interchangeably with terms such as a block or area depending on the case. In a general case, the MxN block may represent samples of M columns and N rows or a set of transform coefficients. The sample may generally represent a pixel or a pixel value, and may indicate only a pixel / pixel value of a luma component or only a pixel / pixel value of a saturation component. The sample may be used as a term for one picture (or image) corresponding to a pixel or pel.

The encoding apparatus 100 subtracts a prediction signal (a predicted block, a prediction sample array) output from the inter prediction unit 180 or the intra prediction unit 185 from the input image signal (original block, original sample array) A signal (residual block, residual sample array) may be generated, and the generated residual signal is transmitted to the converter 120. In this case, as illustrated, a unit that subtracts a prediction signal (a prediction block, a prediction sample array) from an input image signal (original block, original sample array) in the encoding apparatus 100 may be referred to as a subtraction unit 115. The prediction unit may perform prediction on a block to be processed (hereinafter referred to as a current block), and generate a predicted block including prediction samples for the current block. The prediction unit may determine whether intra prediction or inter prediction is applied in units of a current block or CU. As described later in the description of each prediction mode, the prediction unit may generate various information regarding prediction, such as prediction mode information, and transmit it to the entropy encoding unit 190. The prediction information may be encoded by the entropy encoding unit 190 and output in the form of a bitstream.

The intra prediction unit 185 may predict the current block by referring to samples in the current picture. The referenced samples may be located in the neighborhood of the current block or may be located apart depending on a prediction mode. In intra prediction, prediction modes may include a plurality of non-directional modes and a plurality of directional modes. The non-directional mode may include, for example, a DC mode and a planar mode (planar mode). The directional mode may include, for example, 33 directional prediction modes or 65 directional prediction modes depending on the degree of detail of the prediction direction. However, this is an example, and more or less directional prediction modes may be used depending on the setting. The intra prediction unit 185 may determine a prediction mode applied to the current block using a prediction mode applied to neighboring blocks.

The inter prediction unit 180 may derive a predicted block for the current block based on a reference block (reference sample array) specified by a motion vector on the reference picture. At this time, to reduce the amount of motion information transmitted in the inter prediction mode, motion information may be predicted in units of blocks, subblocks, or samples based on the correlation of motion information between a neighboring block and a current block. The motion information may include a motion vector and a reference picture index. The motion information may further include inter prediction direction (L0 prediction, L1 prediction, Bi prediction, etc.) information. In the case of inter prediction, the neighboring block may include a spatial neighboring block existing in the current picture and a temporal neighboring block present in the reference picture. The reference picture including the reference block and the reference picture including the temporal neighboring block may be the same or different. The temporal neighboring block may be referred to by a name such as a collocated reference block or a colCU, and a reference picture including a temporal neighboring block may also be called a collocated picture (colPic). have. For example, the inter prediction unit 180 constructs a motion information candidate list based on neighboring blocks, and generates information indicating which candidates are used to derive a motion vector and / or reference picture index of the current block. can do. Inter prediction may be performed based on various prediction modes. For example, in the case of the skip mode and the merge mode, the inter prediction unit 180 may use motion information of neighboring blocks as motion information of the current block. In the skip mode, unlike the merge mode, the residual signal may not be transmitted. In the case of a motion vector prediction (MVP) mode, a motion vector of a current block is obtained by using a motion vector of a neighboring block as a motion vector predictor and signaling a motion vector difference. I can order.

The prediction signal generated by the inter prediction unit 180 or the intra prediction unit 185 may be used to generate a reconstructed signal or may be used to generate a residual signal.

The transform unit 120 may generate transform coefficients by applying a transform technique to the residual signal. For example, the transformation technique may include at least one of a discrete cosine transform (DCT), a discrete sine transform (DST), a Karhunen-Loeve transform (KLT), a graph-based transform (GBT), or a conditionally non-linear transform (CNT). It can contain. Here, GBT refers to a transformation obtained from this graph when it is said that the relationship information between pixels is graphically represented. CNT means a transform obtained by generating a predictive signal using all previously reconstructed pixels and based on it. Further, the transform process may be applied to pixel blocks having the same size of a square, or may be applied to blocks of variable sizes other than squares.

The quantization unit 130 quantizes the transform coefficients and transmits them to the entropy encoding unit 190, and the entropy encoding unit 190 encodes a quantized signal (information about quantized transform coefficients) and outputs it as a bitstream. have. Information about the quantized transform coefficients may be referred to as residual information. The quantization unit 130 may rearrange block-type quantized transform coefficients into a one-dimensional vector form based on a coefficient scan order, and the quantized transform based on the one-dimensional vector form quantized transform coefficients Information about coefficients may be generated. The entropy encoding unit 190 may perform various encoding operations such as exponential Golomb (CAVLC), context-adaptive variable length coding (CAVLC), and context-adaptive binary arithmetic coding (CABAC). The entropy encoding unit 190 may encode information necessary for video / image reconstruction (eg, a value of syntax elements) together with the quantized transform coefficients together or separately. The encoded information (eg, video / video information) may be transmitted or stored in the unit of a network abstraction layer (NAL) unit in the form of a bitstream. The bitstream can be transmitted over a network or stored on a digital storage medium. Here, the network may include a broadcasting network and / or a communication network, and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD. The signal output from the entropy encoding unit 190 may be configured as an internal / external element of the encoding apparatus 100 by a transmitting unit (not shown) and / or a storing unit (not shown) for storing, or the transmitting unit It may be a component of the entropy encoding unit 190.

The quantized transform coefficients output from the quantization unit 130 may be used to generate a prediction signal. For example, the residual signal may be reconstructed by applying inverse quantization and inverse transform to the quantized transform coefficients through the inverse quantization unit 140 and the inverse transform unit 150 in the loop. The adder 155 adds the reconstructed residual signal to the predicted signal output from the inter predictor 180 or the intra predictor 185, so that the reconstructed signal (restored picture, reconstructed block, reconstructed sample array) Can be created. If there is no residual for the block to be processed, such as when the skip mode is applied, the predicted block may be used as a reconstructed block. The adding unit 155 may be called a restoration unit or a restoration block generation unit. The generated reconstructed signal may be used for intra prediction of the next processing target block in the current picture, or may be used for inter prediction of the next picture through filtering as described below.

The filtering unit 160 may apply subjective filtering to the reconstructed signal to improve subjective / objective image quality. For example, the filtering unit 160 may generate a modified reconstructed picture by applying various filtering methods to the reconstructed picture, and may transmit the modified reconstructed picture to the decoded picture buffer 170. Various filtering methods may include, for example, deblocking filtering, sample adaptive offset, adaptive loop filter, bilateral filter, and the like. The filtering unit 160 may generate various information regarding filtering as described later in the description of each filtering method and transmit it to the entropy encoding unit 190. The filtering information may be encoded by the entropy encoding unit 190 and output in the form of a bitstream.

The modified reconstructed picture transmitted to the decoded picture buffer 170 may be used as a reference picture in the inter prediction unit 180. When the inter prediction is applied through the encoding apparatus 100, prediction mismatches in the encoding apparatus 100 and the decoding apparatus 200 may be avoided, and encoding efficiency may be improved.

The decoded picture buffer 170 may store the corrected reconstructed picture for use as a reference picture in the inter prediction unit 180.

3 is an embodiment of the present specification, and shows a schematic block diagram of a decoding apparatus in which decoding of a video signal is performed. The decoding device 200 of FIG. 3 may correspond to the decoding device 22 of FIG. 1.

Referring to FIG. 3, the decoding apparatus 200 includes an entropy decoding unit 210, an inverse quantization unit 220, an inverse transform unit 230, an adding unit 235, a filtering unit 240, and a decoded picture buffer (DPB). 250, an inter prediction unit 260, and an intra prediction unit 265. The inter prediction unit 260 and the intra prediction unit 265 may be collectively called a prediction unit. That is, the prediction unit may include an inter prediction unit 180 and an intra prediction unit 185. The inverse quantization unit 220 and the inverse conversion unit 230 may be collectively referred to as a residual processing unit. That is, the residual processing unit may include an inverse quantization unit 220 and an inverse conversion unit 230. The entropy decoding unit 210, the inverse quantization unit 220, the inverse transform unit 230, the addition unit 235, the filtering unit 240, the inter prediction unit 260, and the intra prediction unit 265 described above are embodiments. It may be configured by one hardware component (for example, a decoder or processor). Also, the decoded picture buffer 250 may be implemented by one hardware component (eg, a memory or digital storage medium) according to an embodiment.

When a bitstream including video / image information is input, the decoding apparatus 200 may restore an image in response to a process in which the video / image information is processed by the encoding apparatus 100 of FIG. 2. For example, the decoding apparatus 200 may perform decoding using a processing unit applied by the encoding apparatus 100. Thus, in decoding, the processing unit may be, for example, a coding unit, and the coding unit may be divided according to a quad tree structure and / or a binary tree structure from a coding tree unit or a largest coding unit. Then, the decoded video signal decoded and output through the decoding apparatus 200 may be reproduced through the reproduction apparatus.

The decoding apparatus 200 may receive the signal output from the encoding apparatus 100 of FIG. 2 in the form of a bitstream, and the received signal may be decoded through the entropy decoding unit 210. For example, the entropy decoding unit 210 may parse the bitstream to derive information (eg, video / image information) necessary for image reconstruction (or picture reconstruction). For example, the entropy decoding unit 210 decodes information in a bitstream based on a coding method such as exponential Golomb coding, CAVLC, or CABAC, and quantizes a value of a syntax element required for image reconstruction and a transform coefficient for residual. Can output In more detail, the CABAC entropy decoding method receives bins corresponding to each syntax element in a bitstream, and decodes syntax information of the decoding target syntax elements and surrounding and decoding target blocks, or symbols / bins decoded in the previous step. The context model is determined using the information of, and the probability of occurrence of the bin is predicted according to the determined context model to perform arithmetic decoding of the bin to generate a symbol corresponding to the value of each syntax element. have. At this time, the CABAC entropy decoding method may update the context model using the decoded symbol / bin information for the next symbol / bin context model after determining the context model. Among the information decoded by the entropy decoding unit 210, information regarding prediction is provided to a prediction unit (inter prediction unit 260 and intra prediction unit 265), and the entropy decoding unit 210 performs entropy decoding. The dual value, that is, quantized transform coefficients and related parameter information may be input to the inverse quantization unit 220. Also, information related to filtering among information decoded by the entropy decoding unit 210 may be provided to the filtering unit 240. Meanwhile, a receiving unit (not shown) receiving a signal output from the encoding apparatus 100 may be further configured as an internal / external element of the decoding apparatus 200, or the receiving unit may be a component of the entropy decoding unit 210. It might be.

The inverse quantization unit 220 may inverse quantize the quantized transform coefficients to output transform coefficients. The inverse quantization unit 220 may rearrange the quantized transform coefficients in a two-dimensional block form. In this case, reordering may be performed based on the coefficient scan order performed by the encoding apparatus 100. The inverse quantization unit 220 may perform inverse quantization on the quantized transform coefficients by using a quantization parameter (for example, quantization step size information), and obtain transform coefficients.

The inverse transform unit 230 inversely transforms the transform coefficients to obtain a residual signal (residual block, residual sample array).

The prediction unit may perform prediction on the current block and generate a predicted block including prediction samples for the current block. The prediction unit may determine whether intra prediction or inter prediction is applied to the current block based on information about prediction output from the entropy decoding unit 210, and may determine a specific intra / inter prediction mode.

The intra prediction unit 265 may predict the current block by referring to samples in the current picture. The referenced samples may be located in the neighborhood of the current block or spaced apart according to the prediction mode. In intra prediction, prediction modes may include a plurality of non-directional modes and a plurality of directional modes. The intra prediction unit 265 may determine a prediction mode applied to the current block using a prediction mode applied to neighboring blocks.

The inter prediction unit 260 may derive the predicted block for the current block based on a reference block (reference sample array) specified by a motion vector on the reference picture. At this time, to reduce the amount of motion information transmitted in the inter prediction mode, motion information may be predicted in units of blocks, subblocks, or samples based on the correlation of motion information between a neighboring block and a current block. The motion information may include a motion vector and a reference picture index. The motion information may further include inter prediction direction (L0 prediction, L1 prediction, Bi prediction, etc.) information. In the case of inter prediction, the neighboring block may include a spatial neighboring block existing in the current picture and a temporal neighboring block present in the reference picture. For example, the inter prediction unit 260 may construct a motion information candidate list based on neighboring blocks, and derive a motion vector and / or reference picture index of the current block based on the received candidate selection information. Inter prediction may be performed based on various prediction modes, and information about prediction may include information indicating a mode of inter prediction for a current block.

The adding unit 235 adds the obtained residual signal to the prediction signal (predicted block, prediction sample array) output from the inter prediction unit 260 or the intra prediction unit 265, thereby restoring signals (restored pictures, reconstructed blocks). , A reconstructed sample array). If there is no residual for the block to be processed, such as when the skip mode is applied, the predicted block may be used as a reconstructed block.

The adding unit 235 may be called a restoration unit or a restoration block generation unit. The generated reconstructed signal may be used for intra prediction of the next processing target block in the current picture, or may be used for inter prediction of the next picture through filtering as described below.

The filtering unit 240 may improve subjective / objective image quality by applying filtering to the reconstructed signal. For example, the filtering unit 240 may generate a modified reconstructed picture by applying various filtering methods to the reconstructed picture, and may transmit the modified reconstructed picture to the decoded picture buffer 250. Various filtering methods may include, for example, deblocking filtering, sample adaptive offset (SAO), adaptive loop filter (ALF), bilateral filter, and the like.

The corrected reconstructed picture transmitted to the decoded picture buffer 250 may be used as a reference picture by the inter prediction unit 260.

In the present specification, the embodiments described in the filtering unit 160, the inter prediction unit 180, and the intra prediction unit 185 of the encoding device 100 are respectively the filtering unit 240 and the inter prediction unit 260 of the decoding device. ) And the intra prediction unit 265 may be applied to the same or corresponding.

4 is an embodiment of the present specification, and is a structural diagram of a content streaming system.

The content streaming system to which the present specification is applied may largely include an encoding server 410, a streaming server 420, a web server 430, a media storage 440, a user device 450, and a multimedia input device 460. have.

The encoding server 410 serves to compress a content input from multimedia input devices such as a smartphone, camera, camcorder, etc. into digital data to generate a bitstream and transmit it to the streaming server 420. As another example, when the multimedia input devices 460 such as a smartphone, camera, and camcorder directly generate a bitstream, the encoding server 410 may be omitted.

The bitstream may be generated by an encoding method or a bitstream generation method to which the present specification is applied, and the streaming server 420 may temporarily store the bitstream in the process of transmitting or receiving the bitstream.

The streaming server 420 transmits multimedia data to the user device 450 based on a user request through the web server 430, and the web server 430 serves as an intermediary to inform the user of the service. When a user requests a desired service from the web server 430, the web server 430 delivers it to the streaming server 420, and the streaming server 420 transmits multimedia data to the user. At this time, the content streaming system may include a separate control server, in which case the control server serves to control commands / responses between devices in the content streaming system.

Streaming server 420 may receive content from media storage 440 and / or encoding server 410. For example, when content is received from the encoding server 410, the content may be received in real time. In this case, in order to provide a smooth streaming service, the streaming server 420 may store the bitstream for a predetermined time.

Examples of the user device 450 include mobile phones, smart phones, laptop computers, digital broadcasting terminals, personal digital assistants (PDAs), portable multimedia players (PMPs), navigation, and slate PCs. ), Tablet PC, ultrabook, wearable device (e.g., smartwatch, smart glass, head mounted display (HMD), digital TV) , Desktop computers, digital signage, and the like.

Each server in the content streaming system can be operated as a distributed server, and in this case, data received from each server can be distributed.

Block partitioning

The video / image coding method according to this document may be performed based on various detailed technologies, and the detailed description of each detailed technology is as follows. The techniques described below may be related to related procedures such as prediction, residual processing (transformation, quantization, etc.), syntax element coding, filtering, partitioning / segmentation, etc. in the video / image encoding / decoding procedure described above and / or described below. It is apparent to those skilled in the art.

Partitioning structure

Partitioning of picture into CTUs

Pictures may be divided into a sequence of coding tree units (CTUs). The CTU may correspond to a coding tree block (CTB). Alternatively, the CTU may include a coding tree block of luma samples and two coding tree blocks of corresponding chroma samples. In other words, for a picture containing three sample arrays, the CTU may include two corresponding blocks of chroma samples and an NxN block of luma samples.

5 shows an example of a picture divided into CTUs.

The maximum allowable size of the CTU for coding and prediction may be different from the maximum allowable size of the CTU for transformation. For example, the maximum allowable size of a luma block in a CTU may be 128x128 (even if the maximum size of luma CTUs is 64x64).

Partitioning of the CTUs using a tree structure

6 shows an example of multi-type tree splitting modes according to an embodiment of the present specification.

The CTU may be divided into CUs based on a quad-tree (QT) structure. The quadtree structure may be referred to as a quaternary tree structure. This is to reflect various local characteristics. Meanwhile, in this document, the CTU can be divided based on the division of a multi-type tree structure including a binary tree (BT) and a ternary tree (TT) as well as a quad tree. Hereinafter, the QTBT structure may include a quadtree and binary tree based partitioning structure, and the QTBTTT may include a quadtree, binary tree and ternary tree based partitioning structure. In addition, the QTBT structure may include a quadtree, binary tree, and ternary tree based splitting structure. In the coding tree structure, the CU can have a square or rectangular shape. The CTU can be first divided into a quadtree structure. Thereafter, leaf nodes having a quadtree structure may be further divided by a multitype tree structure. For example, as shown in FIG. 6, the multi-type tree structure may schematically include four division types.

The four split types shown in FIG. 6 are vertical binary splitting (SPLIT_BT_VER), horizontal binary splitting (SPLIT_BT_HOR), vertical ternary splitting (SPLIT_TT_VER), horizontal ternary splitting (horizontal ternary) splitting, SPLIT_TT_HOR). Leaf nodes of a multitype tree structure may be referred to as CUs. These CUs can be used as a unit for prediction and transformation procedures. In the exemplary embodiment of the present specification, CU, PU, and TU may have the same block size. However, when the maximum supported transform length is smaller than the width or height of the color component of the CU, the CU and the TU may have different block sizes.

7 shows an example of a signaling mechanism of partition partition information of a quadtree with nested multi-type tree structure with a multitype tree.

Here, the CTU is treated as a root of a quadtree, and is first divided into a quadtree structure. Each quadtree leaf node may then be further divided into a multitype tree structure. In the multi-type tree structure, a first flag (eg, mtt_split_cu_flag) is signaled to indicate whether the corresponding node is additionally partitioned. If the corresponding node is additionally divided, a second flag (eg, mtt_split_cu_vertical_flag) may be signaled to indicate a splitting direction. Thereafter, a third flag (eg, mtt_split_cu_binary_flag) may be signaled to indicate whether the partition type is binary partition or ternary partition. For example, based on mtt_split_cu_vertical_flag and mtt_split_cu_binary_flag, a multi-type tree splitting mode (MttSplitMode) of a CU may be derived as shown in Table 1 below.

8 exemplarily shows that a CTU is divided into multiple CUs based on a quadtree and nested multi-type tree structure.

Here, bold block edges represent quadtree partitioning, and the remaining edges represent multitype tree partitioning. A quadtree partition with a multitype tree can provide a content-adaptive coding tree structure. The CU may correspond to a coding block (CB). Alternatively, the CU may include a coding block of luma samples and two coding blocks of corresponding chroma samples. The size of the CU may be as large as the CTU, or may be configured in 4x4 units in luma sample units. For example, in the case of a 4: 2: 0 color format (or chroma format), the maximum chroma CB size may be 64x64 and the minimum chroma CB size may be 2x2.

In this document, for example, the maximum allowed luma TB size may be 64x64 and the maximum allowed chroma TB size may be 32x32. If the width or height of the CB divided according to the tree structure is greater than the maximum conversion width or height, the CB may be automatically (or implicitly) partitioned until the horizontal and vertical TB size limitations are satisfied.

Meanwhile, for a quad tree coding tree scheme involving a multitype tree, the following parameters may be defined and identified as a sequence parameter set (SPS) syntax element.

-CTU size: the root node size of a quaternary tree

-MinQTSize: the minimum allowed quaternary tree leaf node size

-MaxBtSize: the maximum allowed binary tree root node size

-MaxTtSize: the maximum allowed ternary tree root node size

-MaxMttDepth: the maximum allowed hierarchy depth of multi-type tree splitting from a quadtree leaf

-MinBtSize: the minimum allowed binary tree leaf node size

-MinTtSize: the minimum allowed ternary tree leaf node size

As an example of a quadtree coding tree structure with a multitype tree, the CTU size may be set to 64x64 blocks of 128x128 luma samples and two corresponding chroma samples (in 4: 2: 0 chroma format). In this case, MinOTSize can be set to 16x16, MaxBtSize is set to 128x128, MaxTtSzie can be set to 64x64, MinBtSize and MinTtSize (for width and height) can be set to 4x4, and MaxMttDepth can be set to 4. Quadtree splitting can be applied to CTU to generate quadtree leaf nodes. The quadtree leaf node may be referred to as a leaf QT node. Quadtree leaf nodes may have a size of 16x16 (ie MinOTSize) to a size of 128x128 (ie CTU size). If the leaf QT node is 128x128, it may not be additionally divided into a binary tree / ternary tree. This is because, even in this case, it exceeds MaxBtsize and MaxTtszie (ie, 64x64). In other cases, the leaf QT node may be further divided into a multi-type tree. Therefore, the leaf QT node is a root node for the multitype tree, and the leaf QT node may have a multitype tree depth (mttDepth) 0 value. If the multi-type tree depth reaches MaxMttdepth (eg 4), further partitioning may not be considered. If the width of the multitype tree node is equal to MinBtSize and less than or equal to 2xMinTtSize, additional horizontal splitting may no longer be considered. If the height of the multitype tree node is equal to MinBtSize and less than or equal to 2xMinTtSize, additional vertical splitting may not be considered any more.

To allow for the 64x64 luma block and 32x32 chroma pipeline design in the hardware decoder, TT segmentation can be forbidden in certain cases. For example, if the width or height of the luma coding block is greater than 64, as shown in FIG. 9, TT splitting may be prohibited. Also, for example, if the width or height of the chroma coding block is greater than 32, TT splitting may be prohibited.

9 shows an example in the case where TT splitting is limited for a 128x128 coding block.

In this document, the coding tree scheme may support luma and chroma blocks having a separate block tree structure. For P and B slices, luma and chroma CTBs in one CTU can be restricted to have the same coding tree structure. However, for I slices, luma and chroma blocks may have a separate block tree structure from each other. If the individual block tree mode is applied, the luma CTB may be divided into CUs based on a specific coding tree structure, and the chroma CTB may be divided into chroma CUs based on another coding tree structure. This may mean that a CU in an I slice is composed of a coding block of luma components or coding blocks of two chroma components, and a CU of a P or B slice can be composed of blocks of three color components.

In the above-described "Partitioning of the CTUs using a tree structure", the quadtree coding tree structure with a multi-type tree was described, but the structure in which the CU is divided is not limited to this. For example, the BT structure and the TT structure may be interpreted as a concept included in a multiple partitioning tree (MPT) structure, and a CU may be divided through a QT structure and an MPT structure. In an example in which a CU is split through a QT structure and an MPT structure, a syntax element (eg, MPT_split_type) including information about how many blocks a leaf node of the QT structure is divided into, and a leaf node of the QT structure are vertical and horizontal. A splitting structure may be determined by signaling a syntax element (eg, MPT_split_mode) including information on which direction to split.

In another example, the CU may be divided in a different way from the QT structure, BT structure or TT structure. That is, according to the QT structure, the CU of the lower depth is divided into 1/4 the size of the CU of the upper depth, or the CU of the lower depth is divided into 1/2 the size of the CU of the upper depth according to the BT structure, or according to the TT structure Unlike the CU of the lower depth, which is divided into 1/4 or 1/2 the size of the CU of the upper depth, the CU of the lower depth may be 1/5, 1/3, 3/8, 3 of the CU of the upper depth depending on the case. It may be divided into / 5, 2/3, or 5/8 size, and the method in which the CU is divided is not limited thereto.

Partitioning of the CTUs using tree structure

If the portion of the tree node block exceeds the bottom or right picture boundary, the tree node block is such that all samples of all coded CUs are within the picture boundaries. It can be restricted to be located. In this case, for example, the partitioning rule shown in Table 2 below may be applied.

Restrictions on redundant CU splits

10 exemplarily shows redundant partition patterns that may occur in binary tree partitions and ternary tree partitions.

The quadtree coding block structure with a multi-type tree can provide a very flexible block partitioning structure. Due to the division types supported in the multitype tree, different division patterns can potentially result in the same coding block structure in some cases. By limiting the occurrence of such redundant partition patterns, the data amount of partitioning information can be reduced.

As shown in FIG. 10, two levels of consecutive binary splits in one direction have the same coding block structure as binary partitions for the center partition after ternary splitting. . In this case, binary tree partitioning to the center partition of the ternary tree partitioning is prohibited. This prohibition can be applied to CUs of all pictures. When this specific division is prohibited, signaling of the corresponding syntax elements can be modified to reflect this prohibited case, thereby reducing the number of bits signaled for partitioning. For example, as in the example shown in FIG. 10, when binary tree partitioning for the CU's center partition is prohibited, the mtt_split_cu_binary_flag syntax element indicating whether the partition is a binary partition or a ternary partition is not signaled, and its value is It can be inferred by the decoder to zero.

Inter Prediction

Hereinafter, an inter prediction technique according to an embodiment of the present specification will be described. The inter prediction described below may be performed by the inter prediction unit 180 of the encoding apparatus 100 of FIG. 2 or the inter prediction unit 260 of the decoding apparatus 200 of FIG. 3.

The prediction unit of the encoding apparatus 100 / decoding apparatus 200 may perform inter prediction in block units to derive prediction samples. Inter prediction can represent a prediction derived in a manner dependent on data elements (eg, sample values, or motion information) of a picture (s) other than the current picture (Inter prediction can be a prediction derived in a manner that is dependent on data elements (eg, sample values or motion information) of picture (s) other than the current picture). When inter prediction is applied to a current block, based on a reference block (reference sample array) specified by a motion vector on a reference picture indicated by the reference picture index, a predicted block (predicted sample array) for the current block is derived. You can. At this time, in order to reduce the amount of motion information transmitted in the inter prediction mode, motion information of the current block may be predicted in units of blocks, subblocks, or samples based on the correlation of motion information between a neighboring block and a current block. The motion information may include a motion vector and a reference picture index. The motion information may further include inter prediction type (L0 prediction, L1 prediction, Bi prediction, etc.) information. When inter prediction is applied, the neighboring block may include a spatial neighboring block existing in the current picture and a temporal neighboring block present in the reference picture. The reference picture including the reference block and the reference picture including the temporal neighboring block may be the same or different. The temporal neighboring block may be referred to as a name such as a collocated reference block or a colCU, and a reference picture including a temporal neighboring block may also be referred to as a collocated picture (colPic). . For example, a motion information candidate list may be constructed based on neighboring blocks of the current block, and a flag indicating which candidate is selected (used) to derive a motion vector and / or reference picture index of the current block, or Index information may be signaled. Inter prediction may be performed based on various prediction modes. For example, in the case of the skip mode and the merge mode, motion information of a current block may be the same as motion information of a selected neighboring block. In the skip mode, unlike the merge mode, the residual signal may not be transmitted. In a motion vector prediction (MVP) mode, a motion vector of a selected neighboring block is used as a motion vector predictor, and a motion vector difference can be signaled. In this case, a motion vector of the current block may be derived using a sum of motion vector predictors and motion vector differences.

The video / video encoding procedure based on inter prediction and the inter prediction unit 180 in the encoding apparatus 100 may include, for example, the following.

11 and 12 illustrate a video / video encoding procedure based on inter prediction and an inter prediction unit 180 in the encoding apparatus 100.

The encoding apparatus 100 performs inter prediction on the current block (S1110). The encoding apparatus 100 may derive the inter prediction mode and motion information of the current block, and generate prediction samples of the current block. Here, the procedure for determining the inter prediction mode, deriving motion information, and generating prediction samples may be performed at the same time, or one procedure may be performed before the other procedure. For example, the inter prediction unit 180 of the encoding apparatus 100 may include a prediction mode determination unit 181, a motion information derivation unit 182, and a prediction sample derivation unit 183, and the prediction mode determination unit The prediction mode for the current block may be determined at 181, the motion information of the current block may be derived from the motion information derivation unit 182, and the prediction samples of the current block may be derived from the prediction sample derivation unit 183. For example, the inter prediction unit 180 of the encoding apparatus 100 searches for a block similar to the current block within a certain area (search area) of reference pictures through motion estimation, and It is possible to derive a reference block in which the difference is minimum or below a certain criterion. Based on this, a reference picture index indicating a reference picture in which the reference block is located may be derived, and a motion vector may be derived based on a position difference between the reference block and the current block. The encoding apparatus 100 may determine a mode applied to a current block among various prediction modes. The encoding apparatus 100 may compare the RD cost for various prediction modes and determine the optimal prediction mode for the current block.

For example, when the skip mode or the merge mode is applied to the current block, the encoding apparatus 100 configures a merge candidate list, which will be described later, and the current block among the reference blocks indicated by the merge candidates included in the merge candidate list. A reference block in which the difference from the current block is less than or equal to a certain criterion can be derived. In this case, a merge candidate associated with the derived reference block is selected, and merge index information indicating the selected merge candidate may be generated and signaled to the decoding apparatus 200. Motion information of a current block may be derived using motion information of a selected merge candidate.

As another example, when the (A) MVP mode is applied to the current block, the encoding apparatus 100 configures the (A) MVP candidate list, which will be described later, and (A) the motion vector predictor (MVP) candidates included in the MVP candidate list. The motion vector of the selected MVP candidate can be used as the MVP of the current block. In this case, for example, a motion vector indicating a reference block derived by the above-described motion estimation may be used as a motion vector of the current block, and among the MVP candidates, a motion vector having the smallest difference from the motion vector of the current block may be used. The MVP candidate to have may be the selected MVP candidate. A motion vector difference (MVD), which is a difference obtained by subtracting MVP from the motion vector of the current block, may be derived. In this case, information about the MVD may be signaled to the decoding device 200. In addition, when the (A) MVP mode is applied, the value of the reference picture index may be configured and reference signal index information may be separately signaled to the decoding apparatus 200.

The encoding apparatus 100 may derive residual samples based on the predicted samples (S1120). The encoding apparatus 100 may derive residual samples through comparison of original samples and prediction samples of the current block.

The encoding apparatus 100 encodes video information including prediction information and residual information (S1130). The encoding apparatus 100 may output encoded image information in the form of a bitstream. Prediction information is information related to a prediction procedure, and may include prediction mode information (eg, skip flag, merge flag, or mode index) and motion information. The motion information may include candidate selection information (eg, merge index, mvp flag, or mvp index) that is information for deriving a motion vector. Also, the motion information may include information on the MVD and / or reference picture index information. Also, the motion information may include information indicating whether L0 prediction, L1 prediction, or bi prediction is applied. The residual information is information about residual samples. The residual information may include information about quantized transform coefficients for residual samples.

The output bitstream may be stored in a (digital) storage medium and transmitted to a decoding device, or may be delivered to a decoding device through a network.

Meanwhile, as described above, the encoding apparatus may generate a reconstructed picture (including reconstructed samples and reconstructed blocks) based on the reference samples and the residual samples. This is because the encoding apparatus 100 derives the same prediction result as that performed by the decoding apparatus 200, and thus, it is possible to increase coding efficiency. Accordingly, the encoding apparatus 100 may store the reconstructed picture (or reconstructed samples, reconstructed block) in a memory and use it as a reference picture for inter prediction. As described above, an in-loop filtering procedure or the like may be further applied to the reconstructed picture.

13 and 14 illustrate a video / video decoding procedure based on inter prediction and an inter prediction unit in a decoding apparatus.

The decoding apparatus 200 may perform an operation corresponding to an operation performed in the encoding apparatus 100. The decoding apparatus 200 may perform prediction on the current block based on the received prediction information and derive prediction samples.

Specifically, the decoding apparatus 200 may determine a prediction mode for the current block based on the received prediction information (S1310). The decoding apparatus 200 may determine which inter prediction mode is applied to the current block based on the prediction mode information in the prediction information.

For example, the decoding apparatus 200 may determine whether a merge mode is applied to the current block or (A) MVP mode is determined based on a merge flag. Alternatively, the decoding apparatus 200 may select one of various inter prediction mode candidates based on the mode index. The inter prediction mode candidates may include skip mode, merge mode, and / or (A) MVP mode, or various inter prediction modes described below.

The decoding apparatus 200 derives motion information of the current block based on the determined inter prediction mode (S1320). For example, when the skip mode or the merge mode is applied to the current block, the decoding apparatus 200 may configure a merge candidate list, which will be described later, and select one of the merge candidates included in the merge candidate list. . The selection of the merge candidate may be performed based on a merge index. Motion information of a current block may be derived from motion information of a selected merge candidate. Motion information of the selected merge candidate may be used as motion information of the current block.

As another example, when the (A) MVP mode is applied to the current block, the decoding apparatus 200 configures the (A) MVP candidate list, which will be described later, and (A) the MVP candidate selected from among the MVP candidates included in the MVP candidate list The motion vector of can be used as the MVP of the current block. The selection of the MVP may be performed based on the selection information (MVP flag or MVP index) described above. In this case, the decoding apparatus 200 may derive the MVD of the current block based on information about the MVD, and may derive a motion vector of the current block based on the MVP and MVD of the current block. Also, the decoding apparatus 200 may derive a reference picture index of the current block based on the reference picture index information. The picture indicated by the reference picture index in the reference picture list for the current block may be derived as a reference picture referenced for inter prediction of the current block.

Meanwhile, as described later, motion information of the current block may be derived without configuring a candidate list, and in this case, motion information of the current block may be derived according to a procedure disclosed in the prediction mode described below. In this case, the candidate list configuration as described above may be omitted.

The decoding apparatus 200 may generate prediction samples for the current block based on the motion information of the current block (S1330). In this case, the decoding apparatus 200 may derive a reference picture based on the reference picture index of the current block, and derive predictive samples of the current block using samples of the reference block indicated by the motion vector of the current block on the reference picture. . In this case, as described later, a prediction sample filtering procedure for all or a part of the prediction samples of the current block may be further performed depending on the case.

For example, the inter prediction unit 260 of the decoding apparatus 200 may include a prediction mode determination unit 261, a motion information derivation unit 262, and a prediction sample derivation unit 263, and the prediction mode determination unit The prediction mode for the current block is determined based on the prediction mode information received at (181), and the motion information (motion vector) of the current block is based on the motion information received from the motion information deriving unit 182. And / or a reference picture index, etc.), and predicted samples of the current block may be derived from the predicted sample deriving unit 183.

The decoding apparatus 200 generates residual samples for the current block based on the received residual information (S1340). The decoding apparatus 200 may generate reconstructed samples for the current block based on the predicted samples and residual samples, and generate a reconstructed picture based on the reconstructed samples. (S1350). As described above, an in-loop filtering procedure may be further applied to the reconstructed picture.

As described above, the inter prediction procedure may include a step of determining an inter prediction mode, a step of deriving motion information according to the determined prediction mode, and performing a prediction (generating a predictive sample) based on the derived motion information.

Determination of inter prediction mode

Various inter prediction modes may be used for prediction of a current block in a picture. For example, various modes such as merge mode, skip mode, MVP mode, and affine mode may be used. Decoder side motion vector refinement (DMVR) mode, adaptive motion vector resolution (AMVR) mode, and the like may be further used as ancillary modes. The affine mode may also be called aaffine motion prediction mode. The MVP mode may also be called AMVP (advanced motion vector prediction) mode.

Prediction mode information indicating the inter prediction mode of the current block may be signaled from the encoding device to the decoding device 200. The prediction mode information may be included in the bitstream and received by the decoding apparatus 200. The prediction mode information may include index information indicating one of a plurality of candidate modes. Alternatively, the inter prediction mode may be indicated through hierarchical signaling of flag information. In this case, the prediction mode information may include one or more flags. For example, the encoding apparatus 100 signals whether a skip mode is applied by signaling a skip flag, and when a skip mode is not applied, indicates whether to apply a merge mode, and when a merge mode is not applied. It may be indicated that the MVP mode is applied or may further signal a flag for additional classification. The affine mode may be signaled in an independent mode, or may be signaled in a mode dependent on a merge mode or an MVP mode. For example, the affine mode may be configured as one candidate of the merge candidate list or the MVP candidate list, as described later.

Derivation of motion information according to inter prediction mode

The encoding apparatus 100 or the decoding apparatus 200 may perform inter prediction using motion information of a current block. The encoding apparatus 100 may derive optimal motion information for the current block through a motion estimation procedure. For example, the encoding apparatus 100 may search for a similar reference block having high correlation within a predetermined search range in a reference picture by using the original block in the original picture for the current block, in the unit of fractional pixels, through which motion information Can be derived. The similarity of the block can be derived based on the difference of phase-based sample values. For example, the similarity of a block may be calculated based on a sum of absolute difference (SAD) between a current block (or a template of the current block) and a reference block (or a template of a reference block). In this case, motion information may be derived based on a reference block having the smallest SAD in the search area. The derived motion information may be signaled to the decoding apparatus according to various methods based on the inter prediction mode.

Merge mode and skip mode

When the merge mode is applied, motion information of a current prediction block is not directly transmitted, and motion information of a current prediction block is derived using motion information of a neighboring prediction block. Accordingly, the encoding apparatus 100 may indicate motion information of the current prediction block by transmitting flag information indicating that the merge mode is used and a merge index indicating which prediction block is used.

The encoding apparatus 100 must search a merge candidate block used to derive motion information of a current prediction block in order to perform a merge mode. For example, up to five merge candidate blocks may be used, but the present specification is not limited thereto. In addition, the maximum number of merge candidate blocks may be transmitted in the slice header, and the present specification is not limited thereto. After finding the merge candidate blocks, the encoding apparatus 100 may generate a merge candidate list, and may select a merge candidate block having the smallest cost as a final merge candidate block.

This specification provides various embodiments of a merge candidate block constituting a merge candidate list.

The merge candidate list may use 5 merge candidate blocks, for example. For example, four spatial merge candidates and one temporal merge candidate can be used.

15 shows an example of a spatial merge candidate configuration for the current block.

The merge candidate list for the current block may be configured based on the procedure as shown in FIG. 16.

16 shows an example of a flowchart for configuring a merge candidate list according to an embodiment of the present specification.

The coding device (encoder / decoder) searches for spatial neighboring blocks of the current block and inserts the derived spatial merge candidates into the merge candidate list (S1610). For example, the spatial peripheral blocks may include blocks around the lower left corner of the current block, blocks around the left corner, blocks around the upper right corner, blocks around the upper corner, and blocks around the upper left corner. However, as an example, in addition to the spatial peripheral blocks described above, additional peripheral blocks such as a right peripheral block, a lower peripheral block, and a lower right peripheral block may be further used as the spatial peripheral blocks. The coding apparatus may detect available blocks by searching for spatial neighboring blocks based on priority, and derive motion information of the detected blocks as spatial merge candidates. For example, the encoding apparatus 100 or the decoding apparatus 200 searches the five blocks shown in FIG. 15 in the order of A1, B1, B0, A0, and B2, and sequentially indexes available candidates to merge candidates. It can be configured as a list.

The coding apparatus searches for temporal neighboring blocks of the current block and inserts the derived temporal merge candidate into the merge candidate list (S1620). The temporal peripheral block may be located on a reference picture that is a different picture from the current picture in which the current block is located. The reference picture in which the temporal neighboring block is located may be referred to as a collocated picture or a coll picture. The temporal neighboring blocks may be searched in the order of the lower right corner peripheral block and the lower right center block of the co-located block for the current block on the call picture. Meanwhile, when motion data compression is applied, specific motion information may be stored as representative motion information for each predetermined storage unit in a call picture. In this case, there is no need to store motion information for all blocks in the predetermined storage unit, thereby obtaining a motion data compression effect. In this case, the predetermined storage unit may be predetermined in units of 16x16 sample units, 8x8 sample units, or the like, or size information for a predetermined storage unit may be signaled from the encoding device 100 to the decoding device 200. have. When motion data compression is applied, motion information of a temporal peripheral block may be replaced with representative motion information of a certain storage unit in which the temporal peripheral block is located. That is, in this case, from an implementation point of view, an arithmetic right shift by a certain value based on the coordinates of the temporal periphery block (the upper left sample position), not the prediction block located at the coordinates of the temporal periphery block, and then covers the position of the arithmetic left shift A temporal merge candidate may be derived based on motion information of a prediction block. For example, if the constant storage unit is 2nx2n sample units, and the coordinates of the temporal peripheral block are (xTnb, yTnb), the corrected position ((xTnb >> n) << n), (yTnb >> n) Motion information of a prediction block located at << n)) may be used for temporal merge candidates. Specifically, for example, if the constant storage unit is 16x16 sample units, and the coordinates of the temporal peripheral block are (xTnb, yTnb), the corrected position ((xTnb >> 4) << 4), (yTnb >> 4) Motion information of the prediction block located at << 4)) can be used for temporal merge candidates. Or, for example, if the constant storage unit is 8x8 sample units, and the coordinates of the temporal peripheral block are (xTnb, yTnb), the corrected position ((xTnb >> 3) << 3), (yTnb >> 3 ) << 3)) motion information of the prediction block located at may be used for temporal merge candidate.

The coding apparatus may check whether the number of current merge candidates is smaller than the maximum number of merge candidates (S1630). The maximum number of merge candidates may be predefined or signaled from the encoding device 100 to the decoding device 200. For example, the encoding apparatus 100 may generate information on the number of maximum merge candidates, encode, and transmit the encoded information to the decoding apparatus 200 in the form of a bitstream. When the maximum number of merge candidates is filled, the subsequent candidate addition process may not proceed.

If the number of merge candidates is smaller than the maximum number of merge candidates, the coding apparatus inserts an additional merge candidate into the merge candidate list (S1640). Additional merge candidates are, for example, adaptive temporal motion vector prediction (ATMVP), combined bi-predictive merge candidates (if the slice type of the current slice is of type B) and / or zero vector merge. Candidates may be included.

17 shows an example of a flowchart for constructing a prediction candidate list (MVP candidate list).

Referring to FIG. 17, the coding apparatus searches for a spatial candidate block for motion vector prediction and inserts it into the prediction candidate list (S1710). For example, the coding apparatus may search for neighboring blocks according to a predetermined search order, and add information of neighboring blocks satisfying the condition for the spatial candidate block to the prediction candidate list (MVP candidate list).

After constructing the spatial candidate block list, the coding apparatus compares the number of spatial candidate lists included in the prediction candidate list with a preset reference number (eg, 2) (S1720). If the number of spatial candidate lists included in the prediction candidate list is greater than or equal to the reference number (for example, 2), the coding apparatus may end the construction of the prediction candidate list.

However, if the number of spatial candidate lists included in the prediction candidate list is smaller than the reference number (eg, 2), the coding device searches for the temporal candidate block and inserts it into the prediction candidate list (S1730), and the temporal candidate block is used If it is not possible, the zero motion vector is added to the prediction candidate list (S1740).

Hereinafter, a video encoding and decoding apparatus and method using current picture referencing (CPR) according to an embodiment of the present disclosure will be described.

This embodiment relates to a still image or video encoding / decoding method, and more specifically, a method of constructing a merge candidate list and a motion vector prediction list for efficient transmission of a motion vector when using a current video reference, and resolution of a motion vector It relates to a method for adaptively adjusting, a method and apparatus using a history-based motion vector prediction (HMVP) technology.

An embodiment of the present specification includes a method of adaptively adjusting the resolution of a motion vector in a method of using a part of the current image as a reference image.

In addition, an embodiment of the present specification includes a method of constructing a merge candidate list and a motion vector prediction list in a merge mode and a motion prediction mode when motion compensation is used as a reference image for a portion of the decoded current image.

In addition, embodiments of the present specification include a method of using HMVP technology in a method of using a part of the current image as a reference image.

In the implementation of the present specification, encoding efficiency may be improved by using a portion of the current image decoded in the process of compressing a still image or a video as a reference image.

The embodiment described below is a technique for deriving motion information and performing motion compensation by referring to the current image in the process of deriving motion information, and more details are described together with an improved method for deriving motion information for each component below. do.

Example 1

Decoder

18 shows another example of a decoding apparatus according to an embodiment of the present specification.

One embodiment of the present specification relates to a method of using a part of a current picture decoded in a decoding process of the current coding unit as a reference picture. Referring to FIG. 18, the decoder may include an entropy decoding unit, an inverse quantization unit, an inverse transform unit, a filtering unit, a decoded picture buffer, an inter prediction unit, a current image prediction unit, and an intra prediction unit. Then, the reconstructed video signal output through the decoder may be reproduced through the playback device.

The decoder may receive a signal output from the encoder, and the received signal may be entropy decoded through an entropy decoding unit. The inverse quantization unit obtains a transform coefficient from the entropy decoded signal using quantization step size information. The inverse transform unit inversely transforms the transform coefficients to obtain a residual signal. The reconstructed signal is generated by adding the obtained residual signal to the prediction signal output from the inter prediction unit, the current image prediction unit, or the intra prediction unit. The current image prediction unit may be included in the inter prediction unit. The filtering unit applies filtering to the reconstructed signal, outputs the filtered reconstructed signal to a playback device, or transmits it to a decoded picture buffer. The filtered signal transmitted to the decoded picture buffer may be used as a reference picture in the inter prediction unit.

19 shows a usable area of motion compensation when using motion compensation in the current image.

In general, when designing a video decoder, it is possible to support wave-like parallelism. That is, parallelization may be performed in units of lines of the largest coding unit, and in order to maintain a reference of a neighboring block in consideration of compression performance, when motion compensation is used in the current image, the upper largest coding unit is based on the current largest coding unit. The maximum coding unit on the right side by +1 or +2 in units of the maximum coding unit line may be used as a motion compensation area.

In another embodiment, an area included in a CTU of a specific size (eg, 64x64) located on the left side of the current block may be used as an area referenced in the current picture.

In an embodiment of the present specification, encoding efficiency may be improved when an I-slice without a reference image or the most similar motion compensation block exists in the current image by using the current image as a reference image.

Encoder

In the embodiment of the present specification, a part of the current picture (current picture) that has been encoded so far in the encoding process of the current coding unit may be used as a reference picture. In the same manner as the decoder, the encoder adds the current image to the reference image list in the current image prediction unit (current picture prediction unit), and finds a block most similar to the current block in a predetermined region among the regions that have already been coded. The optimal motion vector uses motion information prediction as in the conventional inter mode, and the motion vector difference value is transmitted to the decoder. A method of performing prediction on a current block using a part of the current image may be referred to as an intra block copy (IBC).

Two methods may be used to indicate whether motion compensation is used in the current image. First, it may be determined as follows whether motion compensation is used in the current image based on the reference image index.

if (DiffPicOrderCnt (currPic, RefPicList0 [ref_idx_l0 [x0] [y0]])!! 0)

The DiffPicOrderCnt function represents a difference in picture order count (POC) between two incoming images. The input value currPic means the current image including the current decoding target block, and RefPicList0 [ref_idx_l0 [x0] [y0]] means the reference video of the current decoding target block. Here, x0 and y0 indicate the position of the current block, and ref_idx_l0 indicates the reference image index in the 0 direction of the reference image list. Therefore, the reference image of the current block can be calculated using the reference image index received from the reference image list. The output value 0 by the DiffPicOrderCnt function means that the current image and the reference image are the same image.

Second, whether to use motion compensation in a current image in a coding unit or block unit unit may be transmitted in the form of a flag. In one embodiment, a flag indicating whether motion compensation is used in the current image may be defined as intra_bc_flag, and when this flag is 1, the corresponding block uses motion compensation in the current image, and if the flag is 0, This block does not use motion compensation in the current image.

Bitstream

Based on the above-described embodiment of the present invention, the encoded information (eg, encoded video / image information) derived by the encoding device may be output in the form of a bitstream. The encoding information may include a flag indicating whether to use the current image as a reference image. The encoded information may be transmitted or stored in units of network abstraction layer (NAL) units in the form of a bitstream. The bitstream can be transmitted over a network, or it can be stored in a non-transitory digital storage medium. In addition, as described above, the bitstream is not directly transmitted from the encoding device 100 to the decoding device 200, and may be streamed / downloaded through an external server (for example, a content streaming server). Here, the network may include a broadcasting network and / or a communication network, and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD.

Example 2

Decoder

20 shows an example of a decoding flowchart according to the HMVP method.

Referring to FIG. 20, the decoding apparatus 200 loads a table composed of HMVP candidates and decodes a block using HMVP candidates. After decoding the block, the decoding apparatus 200 updates the HMVP candidate using the decoded motion information.

21 and 22 show an example of a process of updating a table in the HMVP method.

In one embodiment, the table size S is set to 16, which means that up to 16 HMVP candidates can be added to the table. If there are more than 16 HMVP candidates derived from previously coded blocks in the table, a FIFO (first-in-first-out) rule is applied so that the table always includes the 16 recently coded motion candidates. do. 19 shows an example of a case where the FIFO rule is applied to remove the HMVP candidate used in the proposed method and add a new HMVP candidate.

In order to further improve coding efficiency, a constrained FIFO rule is introduced in which a redundancy check is first applied to determine whether the same HMVP exists in a table when HMVP is inserted into the table. 20, if duplicate HMVPs (HMVPs 2) are found, the same HMVPs are removed from the table and all HMVP candidates are moved in the preceding order (ie, the index is decreased by 1).

HMVP candidates can be used in the merge candidate list construction process. All HMVP candidates from the last entry to the first entry in the table are inserted after the TMVP (temporal MVP) candidate. Pruning is applied for HMVP candidates. When the total number of available merge candidates reaches the signaled maximum allowed merge candidates, the merge candidate list construction process ends.

Similarly, HMVP candidates can also be used in the AMVP candidate list construction process. The motion vectors of the last k HMVP candidates in the table can be inserted after the TMVP candidate. HMVP candidates having the same reference picture as the AMVP target reference picture can be used to construct the AMVP candidate list. For example, K may be 4.

Also, if the total number of merge candidates is greater than or equal to 15, a binarization method that adds a fixed length (with 3 bits) to a truncated unary to code a merge index Can be applied. The total number of merge candidates is represented by N _mrg , and the binarization method is tabulated as shown in Table 3 below.

Encoder

In the embodiment of the present specification, the HMVP encoder constructs a candidate list in the same manner as the decoder, and may use prediction of merge or AMVP.

Bitstream

Based on the above-described embodiment of the present invention, the encoded information (eg, encoded video / image information) derived by the encoding device may be output in the form of a bitstream. The encoding information may include a flag indicating whether to use the current image as a reference image. The encoded information may be transmitted or stored in units of network abstraction layer (NAL) units in the form of a bitstream. The bitstream can be transmitted over a network, or it can be stored in a non-transitory digital storage medium. In addition, as described above, the bitstream is not directly transmitted from the encoding device 100 to the decoding device 200, and may be streamed / downloaded through an external server (for example, a content streaming server). Here, the network may include a broadcasting network and / or a communication network, and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD.

Example 3

Decoder

23 shows an example of a flowchart for adaptively adjusting the resolution of a motion vector.

The present embodiment aims to lower the expressive power of motion information and obtain a gain in terms of bit rate by reducing the amount of information used for the transmission of motion vector difference (MVD).

The decoder first parses the MVD (S2305), decodes the flag for the AMVR mode (AMVR flag) indicating whether or not the present adaptive motion vector resolution (AMVR) is applied (with or without the use of a predefined default resolution), and the AMVR flag It is determined whether to apply the AMVR through (S2310). When the AMVR flag is 0, the resolution of the MVD is 1/4 and the scale value is designated as 0 in the same manner as in the previous step (S2330). When the AMVR flag is 1, additionally, a syntax element (AMVR accuracy information or AMVR accuracy index) indicating the unit of the AMVR is decoded, and the unit of the AMVR is checked through the AMVR accuracy information (S2315). For example, the AMVR accuracy information may indicate whether the unit of the AMVR is an integer unit or a 4-fold integer unit. As a result of checking the AMVR accuracy information, when the resolution of the MVD is an integer unit, the scale value is determined to be 1 in order to up-scale the MV from 1/4 unit to an integer unit (S2325). This is equivalent to applying a shift operation (<< 2) on MVD values. As a result of checking the AMVR accuracy information, when the resolution of the MVD is an integer unit of 4 times, the scale value is determined as 2 in order to up-scale the MV from 1/4 unit to 4 times integer unit (S2320). This is equivalent to applying a shift operation (<< 4) to the MVD value. The decoder performs scaling on the MVD according to the resolution of the MVD determined through the AMVR flag and AMVR accuracy information (S2335).

In this embodiment, the resolution of the MVD according to the AMVR flag and AMVR accuracy information is shown in Table 4 below.

Encoder

The encoder selects a better case through rate-distortion optimization between a case in which the resolution of the MVD to which the present embodiment is applied is lowered and transmitted in the original resolution, and an AMVR flag indicating whether to adjust the resolution. Can transmit. The AMVR flag indicates whether the resolution of the MVD is 1/4. When the AMVR flag is 0, the resolution of the MVD is 1/4, and the encoder no longer transmits information about the resolution of the MVD. If the AMVR flag is 0, the resolution of the MVD may be an integer unit rather than a 1/4 unit, and additionally, AMVR accuracy information indicating whether the resolution of the MVD is an integer unit or a 4-fold integer unit may be transmitted to the decoder. have.

Bitstream

Based on the above-described embodiment of the present invention, the encoded information (eg, encoded video / image information) derived by the encoding device may be output in the form of a bitstream. The encoding information may include an AMVR flag indicating whether the AMVR is used and AMVR accuracy information indicating whether the resolution of the MVD is an integer unit or an integer unit of 4 times. The encoded information may be transmitted or stored in units of network abstraction layer (NAL) units in the form of a bitstream. The bitstream can be transmitted over a network, or it can be stored in a non-transitory digital storage medium. In addition, as described above, the bitstream is not directly transmitted from the encoding device 100 to the decoding device 200, and may be streamed / downloaded through an external server (for example, a content streaming server). Here, the network may include a broadcasting network and / or a communication network, and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD.

Example 4

Decoder

The present embodiment relates to a method of adaptively adjusting the resolution of a motion vector in a method of using a part of the current image as a reference image during the decoding process of the current coding unit. When a part of the decoded current image is used as a reference image, if the resolution of the motion vector is 1/4, there is a problem in that a decoder needs an interpolation filter for the image. Accordingly, when motion compensation is performed using the current image as a reference image, motion compensation is performed in an integer unit.

However, in the AMVR technology that adaptively adjusts the resolution of the motion vector, the AMVR flag indicating whether the resolution of the motion vector for the current coding unit, that is, the resolution of the MVD is 1/4 unit, is transmitted. Therefore, when motion compensation is performed using the current image as a reference image, the AMVR flag is always set to 1. Here, the AMVR flag value 1 indicates that the resolution of the MVD is an integer unit.

In order to solve the problem that occurs when the AMVR flag is always 1 in the CPR using the current image as a reference image, a condition for checking whether the CPR is as shown in FIG. 24 may be added. The determination of whether to apply CPR may be performed by comparing the POC of the reference image with the POC of the current image. In addition, whether a CPR is applied may be checked through a flag indicating whether CPR is applied in a coding unit unit. As shown in FIG. 24, when the CPR condition is 0, both the AMVR flag and the AMVR accuracy information can be transmitted in the same way as the existing AMVR. On the other hand, when the CPR condition is 1, the AMVR flag is not transmitted and only AMVR accuracy information can be transmitted.

24 illustrates an example of a process of scaling an MVD using an AMVR according to an embodiment of the present specification.

Referring to FIG. 24, the decoder parses the MVD (S2405) and determines whether to apply CPR (S2410). If CPR is not applied, the decoder determines whether AMVR is applied (S2415), and if AMVR is not applied, sets the scale value to 0 (S2435). When AMVR is applied, the decoder determines whether the resolution of the MVD is an integer unit or a 4-fold integer unit (S2420). If the resolution of the MVD is not an integer unit of 4 times (when it is an integer unit), the decoder sets the scale value to 1 (S2430). When the resolution of the MVD is 4 times an integer unit, the decoder sets the scale value to 2 (S2425). Thereafter, the decoder performs scaling on the MVD according to the scale value (S2440).

On the other hand, when MVD scaling is performed according to the flowchart shown in FIG. 24, the decoder must continuously check whether CPR is in the process of decoding the bitstream, which may result in a decrease in the parsing throughput of the decoder. Therefore, as shown in FIGS. 25 to 27, the CPR condition checking process is removed before parsing of the AMVR flag, and the CPR condition checking can be adaptively performed in the decoding process.

25 illustrates another example of a process of scaling an MVD using an AMVR according to an embodiment of the present specification.

Referring to FIG. 25, the decoder parses the MVD (S2505) and determines whether to apply the AMVR (S2510). When AMVR is not applied, the decoder sets the scale value to 0 (S2530). Thereafter, the decoder determines whether CPR is applied (S2535), and increases the scale value by 1 when CPR is applied (S2540). When AMVR is applied, the decoder determines whether the resolution of the MVD is an integer unit or a 4-fold integer unit (S2515). When the resolution of the MVD is not an integer unit of 4 times (when it is an integer unit), the decoder sets the scale value to 1 (S2525). When the resolution of the MVD is 4 times an integer unit, the decoder sets the scale value to 2 (S2520). Thereafter, the decoder performs scaling on the MVD according to the scale value (S2545).

In FIG. 25, when CPR is applied, an integer pixel unit MVD may be used. When the AMVR flag is 0, the decoder additionally determines whether to apply CPR, and when the CPR is applied, the scale value can be increased by 1. That is, the resolution of the MVD according to the AMVR flag and the AMVR accuracy information (four-pel flag) may be configured as shown in Table 5 below.

26 illustrates another example of a process of scaling an MVD using an AMVR according to an embodiment of the present specification.

Referring to FIG. 26, the decoder parses the MVD (S2605) and determines whether to apply the AMVR (S2610). When AMVR is not applied, the decoder sets the scale value to 0 (S2630). Thereafter, the decoder determines whether CPR is applied (S2635), and increases the scale value by 1 when CPR is applied (S2640). When AMVR is applied, the decoder determines whether the resolution of the MVD is an integer unit or a 4-fold integer unit (S2615). If the resolution of the MVD is not an integer unit of 4 times (when it is an integer unit), the decoder sets the scale value to 1 (S2625). After the scale value is set to 1, the decoder determines whether CPR is applied (S2635) and increases the scale value by 1 when CPR is applied (S2640). When the resolution of the MVD is 4 times an integer unit, the decoder sets the scale value to 2 (S2620). Thereafter, the decoder performs scaling on the MVD according to the scale value (S2645).

In FIG. 26, when CPR is applied, an integer pixel unit and an MVD of a 4-fold integer pixel unit may be used. When the AMVR flag is 0 or 1 and the AMVR accuracy information (four-pel flag) is 0, the decoder may additionally determine whether to apply CPR and increase the scale value by 1 when CPR is applied. That is, the resolution of the MVD according to the AMVR flag and the AMVR accuracy information (four-pel flag) may be as shown in Table 6 below.

27 shows another example of a process of scaling an MVD using an AMVR according to an embodiment of the present specification.

Referring to FIG. 27, the decoder parses the MVD (S2705) and determines whether to apply the AMVR (S2710). When AMVR is not applied, the decoder sets the scale value to 0 (S2730). When AMVR is applied, the decoder determines whether the resolution of the MVD is an integer unit or a quadruple integer unit (S2715). When the resolution of the MVD is not an integer unit of 4 times (when it is an integer unit), the decoder sets the scale value to 1 (S2725). When the resolution of the MVD is 4 times an integer unit, the decoder sets the scale value to 2 (S2720). Thereafter, the decoder determines whether CPR is applied (S2735), and increases the scale value by 1 when CPR is applied (S2740). The decoder performs scaling on the MVD according to the scale value (S2745).

In FIG. 27, when CPR is applied, an integer pixel unit, a quadruple pixel unit, and a 16x integer pixel unit MVD may be used. According to the method of FIG. 27, it is always determined whether CPR is applied, and when CPR is applied, the scale value increases by one. That is, the resolution of the MVD according to the AMVR flag and the AMVR accuracy information (four-pel flag) may be as shown in Table 7.

The resolution of the integer pixel for CPR may be in the form of {1, 2}, {1, 2, 4}, {1, 4, 8} in addition to the above-described embodiment.

Encoder

The encoder selects a better case through rate-distortion optimization between a case in which the resolution of the MVD to which the present embodiment is applied is lowered and transmitted in the original resolution, and an AMVR flag indicating whether to adjust the resolution. Can transmit. The AMVR flag indicates whether the resolution of the MVD is 1/4. When the AMVR flag is 0, the resolution of the MVD is 1/4, and the encoder no longer transmits information about the resolution of the MVD. If the AMVR flag is 0, the resolution of the MVD may be an integer unit rather than a 1/4 unit, and additionally, AMVR accuracy information indicating whether the resolution of the MVD is an integer unit or a 4-fold integer unit may be transmitted to the decoder. have.

Bitstream

Based on the above-described embodiment of the present invention, the encoded information (eg, encoded video / image information) derived by the encoding device may be output in the form of a bitstream. The encoding information may include an AMVR flag indicating whether the AMVR is used and AMVR accuracy information indicating whether the resolution of the MVD is an integer unit or an integer unit of 4 times. The encoded information may be transmitted or stored in units of network abstraction layer (NAL) units in the form of a bitstream. The bitstream can be transmitted over a network, or it can be stored in a non-transitory digital storage medium. In addition, as described above, the bitstream is not directly transmitted from the encoding device 100 to the decoding device 200, and may be streamed / downloaded through an external server (for example, a content streaming server). Here, the network may include a broadcasting network and / or a communication network, and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD.

Example 5

Decoder

This embodiment provides a method of constructing a merge candidate list in a merge mode when motion compensation is performed on a part of a decoded current image as a reference image during a decoding process of the current coding unit.

28 illustrates an example of a method of constructing a merge candidate list according to an embodiment of the present specification.

First, the decoder searches for a spatial candidate block for constructing a merge candidate list, and inserts motion vector information of an available spatial candidate block into the merge candidate list (S2810). The spatial candidate block may include a left block adjacent to the current block, a left-lower block, an upper block, an upper-right block, and an upper-left block. Thereafter, the decoder searches for a temporal candidate block and inserts information on a motion vector of the available temporal candidate block into the merge candidate list (S2820). After constructing the merge candidate list from the spatial and temporal neighbor candidate blocks, if the size of the merge list does not reach the maximum merge size, the decoder adds the combined merge candidate to the merge candidate list (S2830).

If the configured merge candidate list is less than the maximum merge candidate list and CPR is applied to the current block, the decoder may add the following candidates to the merge candidate list according to the size of the current block (S2840). Here, W represents the width of the current block, and H represents the height of the current block. If only one merge candidate is added, the merge candidates that can be added are as follows.

(1) (-W, 0)

(2) (0, -H)

(3) (-2W, 0)

(4) (0, -2H)

(5) (-W, -H)

(6) (-2W, -2H)

When two merge candidates are added, the merge candidates that can be added are as follows.

(1) (-W, 0), (0, -H)

(2) (0, -H), (-W, 0)

(3) (-2W, 0), (0, -2H)

(4) (0, -2H), (-2W, 0)

(5) (-W, -H), (-2W, -2H)

(6) (-2W, -2H), (-W, -H)

As described above, the block size-based motion vector candidate according to the CPR condition is added only when a motion compensation candidate referring to the current image (that is, CPR is applied) in a previous merge candidate list is included below a certain threshold value. Can be. Here, the threshold value may correspond to a value from 0 to the maximum number of merge candidate list numbers. Also, the decoder may insert the zero vector candidate into the merge candidate list (S2850).

In addition, an embodiment of the present disclosure provides a method of constructing a motion vector prediction (MVP) list in a motion vector prediction (MVP) mode when motion compensation is used in a current image.

29 illustrates an example of a method of constructing a motion vector prediction list according to an embodiment of the present specification.

First, the decoder searches for a spatial candidate block for constructing a motion vector prediction list, and inserts information of a motion vector of an available spatial candidate block into a motion vector prediction list (S2910). The spatial candidate block may include a left block adjacent to the current block, a left-lower block, an upper block, an upper-right block, and an upper-left block. The decoder compares the number of spatial candidate blocks with the maximum motion prediction size (eg 2), and when the configured motion vector prediction candidate is less than the maximum motion prediction size (eg 2), searches for a temporal candidate block and uses available temporal candidates Information about the motion vector of the block may be inserted into the motion vector prediction list (S2930). After constructing the motion vector prediction list from the spatial and temporal neighboring candidate blocks, the decoder compares the number of candidate blocks with the maximum motion prediction size (eg, 2) (S2940), and the size of the motion vector prediction list does not reach the maximum merge size. Otherwise, the decoder inserts the block size-based motion vector candidate of the present embodiment into the motion vector prediction list (S2950).

When the configured motion vector prediction list is less than the maximum motion vector prediction list, when the reference image of the current block is the current image (ie, when CPR is applied), the following candidates are added to the motion vector prediction list according to the size of the current block. Can be added. Here, W represents the width of the current block, and H represents the height of the current block. If only one merge candidate is added, the merge candidates that can be added are as follows.

(1) (-W, 0)

(2) (0, -H)

(3) (-2W, 0)

(4) (0, -2H)

(5) (-W, -H)

(6) (-2W, -2H)

When two merge candidates are added, the merge candidates that can be added are as follows.

(1) (-W, 0), (0, -H)

(2) (0, -H), (-W, 0)

(3) (-2W, 0), (0, -2H)

(4) (0, -2H), (-2W, 0)

(5) (-W, -H), (-2W, -2H)

(6) (-2W, -2H), (-W, -H)

Encoder

The encoder uses the same method as the merge candidate list construction method by the decoder and the motion vector prediction list construction method.

Bitstream

Based on the above-described embodiment of the present invention, the encoded information (eg, encoded video / image information) derived by the encoding device may be output in the form of a bitstream. The encoded information may be transmitted or stored in units of network abstraction layer (NAL) units in the form of a bitstream. The bitstream can be transmitted over a network, or it can be stored in a non-transitory digital storage medium. In addition, as described above, the bitstream is not directly transmitted from the encoding device 100 to the decoding device 200, and may be streamed / downloaded through an external server (for example, a content streaming server). Here, the network may include a broadcasting network and / or a communication network, and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD.

Example 6

Decoder

This embodiment provides a method of applying HMVP while using a part of the decoded current image as a reference image during the decoding process of the current coding unit.

30 shows an example of a flowchart for performing HMVP according to an embodiment of the present specification.

The decoder initializes the HMVP buffer in slice units, and determines whether it is the first CTU in a CTU-row for each CTU. In the flowchart of FIG. 30, the decoder determines a CTU with (ctu_idx% Num) of 0 as the first CTU in a specific CTU-row. At this time, Num means the number of CTUs in each CTU-row. For the first CTU, the decoder initializes the buffer, otherwise it keeps the buffer. Thereafter, each CU undergoes an AMVP or merge process, and a candidate stored in the HMVP buffer may be included as an MVP candidate. The MVP determined in the AMVP or merge process is stored in the HMVP buffer and the processes shown in FIG. 30 are repeated.

More specifically, referring to FIG. 30, the decoder sets the HMVP buffer initialization (S3010) and the CTU index (ctu_idx) to 0, and through the loop process (S3020) for repetitive operation, the current CTU is the first CTU in the CTU-row. Whether or not it is determined (S3030). If the current CTU is the first CTU in the CTU-row (ctu_idx% Num == 0), the coding apparatus performs HMVP buffer initialization (S3040). Thereafter, the coding apparatus sets the coding unit index (cu_idx) to 0, and then performs an HMVP buffer update (S3060) through a loop process (S3050). After the HMVP update, the coding apparatus increases the CU index for coding for the next CU (cu_idx ++) and increases the CTU index for coding of the next CTU when coding for the CTU is completed (ctu_idx ++).

If the current image is included among the reference images of the current image (ie, CPR is applied), the following MV may be used to initialize the HMVP buffer. Here, W represents the width of the current block, and H represents the height of the current block.

(1) (-W, 0), (0, -H)

(2) (0, -H), (-W, 0)

(3) (-2W, 0), (0, -2H)

(4) (0, -2H), (-2W, 0)

(5) (-W, -H), (-2W, -2H)

(6) (-2W, -2H), (-W, -H)

31 shows another example of a flowchart for performing HMVP according to an embodiment of the present specification.

In addition, an embodiment of the present specification proposes another method of initializing the history management buffer when applying HMVP. In this embodiment, the dependency of the CTU unit can be eliminated by initializing the HMVP buffer every CTU. In the method proposed in this embodiment, since the HMVP buffer is initialized in CTU units, MVP of blocks existing in the CTU can be stored, and various MVPs in the same CTU can be used as candidates, and conditions for initialization can be reduced.

More specifically, referring to FIG. 31, the decoder sets the CTU index to 0 (ctu_idx = 0) and performs HMVP buffer initialization through a loop process 3110 (S3120). Thereafter, the decoder sets the CU index to 0 (cu_idx = 0) and performs HMVP buffer update through a loop process (S3130) (S3140). Then, the decoder updates the CU index for coding the next CU in the CTU (cu_idx ++), and updates the CTU index for coding for the next CTU when coding of all CUs in the CTU is completed (ctu_idx ++).

If the current image is included among the reference images of the current image (ie, CPR is applied), the following MV may be used to initialize the HMVP buffer. Here, W represents the width of the current block, and H represents the height of the current block.

(1) (-W, 0), (0, -H)

(2) (0, -H), (-W, 0)

(3) (-2W, 0), (0, -2H)

(4) (0, -2H), (-2W, 0)

(5) (-W, -H), (-2W, -2H)

(6) (-2W, -2H), (-W, -H)

Encoder

The encoder may perform the same operation as the HMVP configuration method performed by the decoder, and if initialization of the HMVP buffer is required, the encoder performs initialization in the same way as the decoder.

Bitstream

Based on the above-described embodiment of the present invention, the encoded information (eg, encoded video / image information) derived by the encoding device may be output in the form of a bitstream. The encoded information may be transmitted or stored in units of network abstraction layer (NAL) units in the form of a bitstream. The bitstream can be transmitted over a network, or it can be stored in a non-transitory digital storage medium. In addition, as described above, the bitstream is not directly transmitted from the encoding device 100 to the decoding device 200, and may be streamed / downloaded through an external server (for example, a content streaming server). Here, the network may include a broadcasting network and / or a communication network, and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD.

32 shows an example of a flowchart for performing prediction according to an embodiment of the present specification. Each process of FIG. 32 may be performed by the inter prediction unit 180 of the encoding device 100 or the inter prediction unit 260 of the decoding device 200.

In step S3210, the decoder (or encoder) acquires a motion vector difference value (MVD) for the current block.

In step S3220, the decoder determines the resolution of the MVD based on the information on the adaptive motion vector resolution (AMVR) and the reference picture.

For example, the information on the AMVR includes an AVMR flag indicating whether to use a predefined native resolution (eg, 1/4) and AMVR accuracy information indicating a unit of AVMR, and the information on the reference picture is , POC of a reference picture.

In addition, the decoder may compare the POC of the current block with the POC of the current block after parsing the AMVR flag or AMVR accuracy information, as shown in FIGS. 25 to 27. For example, as shown in FIG. 25, the decoder determines whether the AMVR is applied using the AMVR flag, and when the AMVR is not applied, the resolution of the MVD based on whether the POC of the current block and the POC of the reference picture are the same. Can decide.

In addition, as shown in Table 5, when the AMVR flag is 1 and the AMVR accuracy information indicates a 4-pel unit (when the four-pel flag is 1), the resolution of the MVD is determined to be 4 , When the AVMR flag is 1 and the AMVR accuracy information indicates an integer unit (when the four-pel flag is 0), the resolution of the MVD is determined to be 1, the AVMR flag is 0, the POC of the current block and the reference picture If the POC of (if CPR is not applied), the resolution of the MVD is determined to be 1/4, the AVMR flag is 0, and the POC of the current block and the POC of the reference picture are the same (if CPR is applied), the MVD The resolution of can be determined as 1.

Also, as shown in Table 6, when the AMVR flag is 1 and the AMVR accuracy information indicates a 4-pel unit, the resolution of the MVD is determined to be 4, the AVMR flag is 1, and the AMVR accuracy information is an integer. When the POC of the current block and the POC of the reference picture are different while indicating the unit, the resolution of the MVD is determined to be 1, the AVMR flag is 1, and the AMVR accuracy information indicates the integer unit, while the POC of the current block and the POC of the reference picture are In the same case, if the resolution of the MVD is 4, the AVMR flag is 0 and the POC of the current block and the POC of the reference picture are the same, the resolution of the MVD is determined as 1, the AVMR flag is 0, and the POC of the current block When the POCs of the reference pictures are different, the resolution of the MVD may be determined as 1/4.

Also, as shown in Table 7, when the AMVR flag is 1 and the AMVR accuracy information indicates a 4-pel unit while the POC of the current block and the POC of the reference picture are the same, the resolution of the MVD is determined to be 16, When the AMVR flag is 1 and the AMVR accuracy information indicates a 4-pel unit, and the POC of the current block and the POC of the reference picture are different, the resolution of the MVD is determined as 4, the AMVR flag is 1, and the AMVR accuracy If the POC of the current block and the POC of the reference picture are the same while the information indicates an integer unit, the resolution of the MVD is determined to be 4, the AMVR flag is 1, and the AMVR accuracy information indicates the integer unit, while referring to the POC of the current block. If the POCs of the pictures are different, the resolution of the MVD is determined to be 1, the AMVR flag is 0 and the POC of the current block and the POC of the reference picture are the same, the resolution of the MVD is determined to 1, the AMVR flag is 0 and the current block. POC and Reference If the POC of the destination are different, the resolution of the MVD may be determined as 1/4.

In step S3220, the decoder scales the MVD according to the determined resolution, and in steps S3240 and S3250, the decoder determines a motion vector for the current block based on the scaled MVD and a motion vector predictor, and A prediction sample for the current block is generated based on the vector.

33 shows an example of a block diagram of an apparatus for processing an image signal according to an embodiment of the present specification. The video signal processing device of FIG. 33 may correspond to the encoding device 100 of FIG. 2 or the decoding device 200 of FIG. 3.

The image processing apparatus 3300 for processing the image signal includes a memory 3320 for storing the image signal, and a processor 3310 for processing the image signal while being combined with the memory.

The processor 3610 according to an embodiment of the present disclosure may be configured with at least one processing circuit for processing an image signal, and may process an image signal by executing instructions for encoding or decoding the image signal. That is, the processor 3610 may encode the original image data or decode the encoded image signal by executing the above-described encoding or decoding methods.

The embodiments described herein may be implemented and implemented on a processor, microprocessor, controller, or chip. For example, the functional units illustrated in each drawing may be implemented and implemented on a computer, processor, microprocessor, controller, or chip.

The processing method to which the present specification is applied may be produced in the form of a program executed by a computer, and may be stored in a computer readable recording medium. Multimedia data having a data structure according to the present specification may also be stored in a computer-readable recording medium. The computer-readable recording medium includes all kinds of storage devices and distributed storage devices in which computer-readable data is stored. The computer-readable recording medium includes, for example, Blu-ray Disc (BD), Universal Serial Bus (USB), ROM, PROM, EPROM, EEPROM, RAM, CD-ROM, magnetic tape, floppy disk and optical. It may include a data storage device. In addition, the computer-readable recording medium includes media implemented in the form of a carrier wave (for example, transmission via the Internet). Also, the bitstream generated by the encoding method may be stored in a computer-readable recording medium or transmitted through a wired or wireless communication network.

Further, the embodiments of the present specification may be implemented as a computer program product using program codes, and the program codes may be executed on a computer by the embodiments of the present specification. The program code can be stored on a computer readable carrier.

The decoding device and the encoding device to which the present specification is applied may be included in a digital device. The term "digital device" includes, for example, all digital devices capable of performing at least one of transmission, reception, processing, and output of data, content, and services. Here, the processing of the data, content, service, etc. by the digital device includes an operation of encoding and / or decoding data, content, service, and the like. These digital devices are paired or connected (hereinafter referred to as 'pairing') with other digital devices, external servers, etc. through a wire / wireless network to transmit and receive data. Convert accordingly.

Digital devices include, for example, standing devices such as network TV, HBBTV (Hybrid Broadcast Broadband TV), smart TV (Smart TV), IPTV (internet protocol television), PC (Personal Computer), and the like. , PDA (Personal Digital Assistant), smart phones (Smart Phone), tablet PCs (Tablet PC), notebooks, mobile devices (mobile devices or handheld devices). For convenience, a digital TV is illustrated in Figure 2.3-4, which will be described later, and a mobile device is illustrated and described as an example of a digital device in Figure 2.3-3.

Meanwhile, the term "wired / wireless network" described herein refers to a communication network that supports various communication standards or protocols for interconnection and / or data transmission and reception between digital devices or digital devices and external servers. . Such a wired / wireless network may include both a communication network to be supported in the current or future by a standard and a communication protocol therefor, for example, Universal Serial Bus (USB), Composite Video Banking Sync (CVBS), component, S-Video (Analog), communication standards or protocols for wired connections such as DVI (Digital Visual Interface), HDMI (High Definition Multimedia Interface), RGB, D-SUB, Bluetooth, Radio Frequency Identification (RFID), and infrared communication (Infrared Data Association (IrDA), Ultra Wideband (UWB), ZigBee, Digital Living Network Alliance (DLNA), Wireless LAN (WLAN) (Wi-Fi), Wireless broadband (Wibro), World Interoperability for Microwave (Wimax) Access), HSDPA (High Speed Downlink Packet Access), LTE (Long Term Evolution), Wi-Fi Direct (Direct) can be formed by a communication standard for wireless connection.

Hereinafter, in the case of merely referring to a digital device in the present specification, it may mean a fixed device or a mobile device or include both depending on context.

Meanwhile, the digital device is an intelligent device that supports, for example, a broadcast reception function, a computer function or support, and at least one external input, e-mail, web browsing through a wired / wireless network described above ( It can support web browsing, banking, games, and applications. In addition, the digital device may include an interface for supporting at least one input or control means (hereinafter referred to as an input means) such as a handwritten input device, a touch screen, and a space remote control. The digital device can use a standardized general-purpose operating system (OS). For example, a digital device can add, delete, modify, and update various applications on a general-purpose OS kernel. You can configure and provide a user-friendly environment.

On the other hand, the external input described in this specification includes external input devices, that is, all input means or digital devices connected to the above-mentioned digital devices by wire / wireless and capable of transmitting / receiving related data through them. Here, the external input is, for example, a high-definition multimedia interface (HDMI), a game device such as a play station or an X-box, a smart phone, a tablet PC, a printer, a digital digital device such as a smart TV. Includes all devices.

In addition, the term "server (server)" described in this specification means a client (client), that is, includes all digital devices or systems that supply data to the above-described digital device, referred to as a processor (processor) Also. Examples of such a server include a web page or a portal server that provides web content, an advertising server that provides advertising data, a content server that provides content, and a social media server (SNS). Social Network Service) may include an SNS server providing a service, a service server provided by a manufacturer, or a manufacturing server.

In addition, "channel (channel)" described herein refers to a path (path), means (means), etc. for transmitting and receiving data, for example, a broadcasting channel (broadcasting channel). Here, the broadcast channel is expressed in terms of a physical channel, a virtual channel, and a logical channel according to the activation of digital broadcasting. A broadcast channel can be called a broadcast network. As described above, a broadcast channel refers to a channel for providing or accessing broadcast content provided by a broadcasting station, and the broadcast content is mainly based on real-time broadcasting and is also called a live channel. . However, recently, the medium for broadcasting has become more diversified, and non-real time broadcasting is also activated in addition to real-time broadcasting. As a result, live channels include not only real-time broadcasting, but also non-real-time broadcasting in some cases. It may be understood as a term meaning the entire channel.

In this specification, "arbitrary channel" is further defined in relation to a channel in addition to the above-described broadcast channel. The arbitrary channel may be provided together with a service guide such as an electronic program guide (EPG) along with a broadcast channel, and a service guide, a GUI (Graphic User Interface) or an OSD screen (On-Screen Display) with only a random channel screen).

On the other hand, unlike a broadcast channel having a predetermined channel number between transceivers, a random channel is a channel randomly allocated by a receiver, and a channel number that is not basically overlapped with a channel number for expressing the broadcast channel is allocated. For example, when a specific broadcast channel is tuned, the receiver receives a broadcast signal that transmits broadcast content and signaling information therefor through the tuned channel. Here, the receiver parses channel information from the signaling information, and configures a channel browser, an EPG, and the like based on the parsed channel information and provides it to the user. When the user makes a channel change request through the input means, the receiver responds accordingly.

As described above, since the broadcast channel is a content previously promised between the transmitting and receiving terminals, when a random channel is repeatedly allocated with the broadcast channel, it may cause confusion or confusion of the user, so it is preferable not to repeatedly allocate as described above. . On the other hand, even if the random channel number is not duplicated with the broadcast channel number as described above, there is still a possibility of confusion in the channel surfing process of the user, and it is required to allocate the random channel number in consideration of this. This is because any channel according to the present specification can also be implemented to be accessed as a broadcast channel in the same manner in response to a user's request to switch a channel through an input means in the same way as a conventional broadcast channel. Accordingly, the random channel number is a type in which characters are written in parallel, such as random channel-1, random channel-2, and the like, for the convenience of discrimination or identification of the user's random channel access and broadcast channel number. It can be defined and marked as. On the other hand, in this case, although the display of an arbitrary channel number may be implemented by recognizing in a numeric form, such as the number of the broadcasting channel, in a form in which characters are written in parallel as in random channel-1. In addition, an arbitrary channel number may be provided in a numeric form, such as a broadcast channel, and a channel number may be defined and displayed in various ways distinguishable from a broadcast channel, such as video channel-1, title-1, and video-1. have.

The digital device executes a web browser for a web service, and provides various types of web pages to the user. Here, the web page also includes a web page including a video (video content), in this specification, the video is processed separately or independently from the web page. In addition, the separated video can be implemented to allocate an arbitrary channel number as described above, provide it through a service guide, or the like, and be output according to a channel switching request by a user in a service guide or broadcast channel viewing process. In addition, for services such as broadcast content, games, and applications, in addition to web services, predetermined content, images, audio, items, etc. are separately processed from the broadcast content, games, and the application itself, and for playback, processing, etc. Any channel number can be assigned and implemented as described above.

34 is a diagram schematically showing an example of a service system including a digital device.

Service systems including digital devices include a content provider (CP) 3410, a service provider (SP) 3420, a network provider (NP) 3430, and a home network end user (HNED). ) (Customer) (3440). Here, the HNED 3440 is, for example, a client 3400, that is, a digital device. The content provider 3410 produces and provides various content. As shown in FIG. 34 as the content provider 3410, terrestrial broadcaster, cable SO (System Operator) or MSO (Multiple SO), satellite broadcaster (satellite broadcaster) , Various Internet broadcasters, and private content providers (Private CPs). Meanwhile, the content provider 3410 provides various applications in addition to broadcast content.

The service provider 3420 provides content provided by the content provider 3410 as a service package to the HNED 3440. For example, the service provider 3420 of FIG. 34 packages the first terrestrial broadcast, the second terrestrial broadcast, the cable MSO, the satellite broadcast, various Internet broadcasts, applications, etc., and provides them to the HNED 3440.

The service provider 3420 provides services to the client 300 in a uni-cast or multi-cast manner. Meanwhile, the service provider 3420 may transmit data to a plurality of pre-registered clients 3400 at a time. For this, an Internet Group Management Protocol (IGMP) protocol may be used.

The above-described content provider 3410 and service provider 3420 may be identical or single entities. For example, the content provided by the content provider 3410 may be service packaged and provided to the HNED 3440 to perform the function of the service provider 3420 or vice versa.

The network provider 3430 provides a network for data exchange between the content provider 3410 or / and the service provider 3420 and the client 3400.

The client 3400 can establish a home network to transmit and receive data.

Meanwhile, the content provider 3410 or / and the service provider 3420 in the service system may use conditional access or content protection means to protect transmitted content. In this case, the client 300 may use a processing means such as a cable card (POD: Point of Deployment), a DCAS (Downloadable CAS), etc. in response to the limited reception or content protection.

In addition, the client 3400 may also use a bidirectional service through a network network (or communication network). In this case, rather, the client 3400 may perform the function of the content provider, and the existing service provider 3420 may receive it and transmit it back to another client.

35 is a block diagram illustrating a digital device according to an embodiment. Here, FIG. 35 may correspond to, for example, the client 3400 of FIG. 34, and refers to the digital device described above.

The digital device 3500 includes a network interface 3501, a TCP / IP manager 3502, a service delivery manager 3503, an SI decoder 3504, Demultiplexer (demux) 3505, audio decoder 3506, video decoder 3507, display A / V and OSD module 3508, service control manager control manager (3509), service discovery manager (service discovery manager) (3510), SI & metadata database (SI & Metadata DB) (3511), metadata manager (metadata manager) (3512), service manager (3513), UI And a manager 3514 and the like.

The network interface unit 3501 receives or transmits Internet protocol (IP) packets through a network. That is, the network interface unit 3501 receives services, content, and the like from the service provider 3420 through the network.

The TCP / IP manager 3502 is configured to transmit packets between the source and destination of IP packets received by the digital device 3500 and IP packets transmitted by the digital device 3500. Get involved. Then, the TCP / IP manager 3502 classifies the received packet (s) to correspond to an appropriate protocol, and the service delivery manager 3505, the service discovery manager 3510, the service control manager 3509, and the metadata manager 3512 ), And the like. The service delivery manager 3503 is responsible for controlling received service data. For example, the service delivery manager 3503 may use RTP / RTCP when controlling real-time streaming data. When the real-time streaming data is transmitted using RTP, the service delivery manager 3503 parses the received data packet according to the RTP and transmits it to the demultiplexer 3505 or control of the service manager 3513 According to the SI & metadata database (3511). Then, the service delivery manager 3503 uses the RTCP to feed back the network reception information to the server providing the service. The demultiplexer 3505 demultiplexes the received packets into audio, video, and system information (SI) data, and transmits them to the audio / video decoder 3506/3507 and the SI decoder 3504, respectively.

The SI decoder 3504 decodes service information such as program specific information (PSI), program and system information protocol (PSIP), and digital video broadcasting-service information (DVB-SI).

In addition, the SI decoder 3504 stores the decoded service information in the SI & metadata database 3511, for example. The service information stored in this way may be read and used by the corresponding configuration, for example, at the request of a user.

The audio / video decoder 3506/3507 decodes each audio data and video data demultiplexed by the demultiplexing unit 405. The audio data and video data thus decoded are provided to the user through the display unit 3508.

The application manager may include, for example, a UI manager 3514 and a service manager 3513. The application manager may manage the overall state of the digital device 3500, provide a user interface, and manage other managers.

The UI manager 3514 provides a graphical user interface (GUI) for a user using an on-screen display (OSD) or the like, and receives a key input from a user to perform device operation according to the input. For example, when the UI manager 3514 receives a key input for channel selection from a user, the UI manager 3510 transmits the key input signal to the service manager 3513.

The service manager 3513 controls a manager associated with a service, such as a service delivery manager 3503, a service discovery manager 3510, a service control manager 3509, and a metadata manager 3512.

In addition, the service manager 3513 creates a channel map and selects a channel using the channel map according to a key input received from the user interface manager 3514. Then, the service manager 3513 receives the service information of the channel from the SI decoder 3504 and sets the audio / video packet identifier (PID) of the selected channel to the demultiplexer 3505. The PID set in this way is used in the demultiplexing process described above. Accordingly, the demultiplexer 3505 filters the audio data, video data, and SI data using the PID.

The service discovery manager 3510 provides information necessary to select a service provider that provides a service. When a signal regarding channel selection is received from the service manager 3513, the service discovery manager 3510 finds a service using the information.

The service control manager 3509 is responsible for selecting and controlling services. For example, the service control manager 3509 uses IGMP or RTSP when a user selects a live broadcasting service such as a conventional broadcasting method, and selects a service such as video on demand (VOD). The RTSP is used to select and control services. The RTSP protocol may provide a trick mode for real-time streaming. In addition, the service control manager 3509 may initialize and manage a session through the IMS gateway 3550 using an IP multimedia subsystem (IMS) and a session initiation protocol (SIP). The protocols are one embodiment, and other protocols may be used according to implementation examples.

The metadata manager 3512 manages metadata associated with the service and stores the metadata in the SI & metadata database 3511.

The SI & metadata database 3511 includes service information decoded by the SI decoder 3504, metadata managed by the metadata manager 3512, and information necessary to select a service provider provided by the service discovery manager 3510. To save. In addition, the SI & metadata database 3511 can store set-up data and the like for the system.

The SI & metadata database 3511 may be implemented using non-volatile RAM (NVRAM), flash memory, or the like.

Meanwhile, the IMS gateway 3550 is a gateway that collects functions necessary for accessing the IMS-based IPTV service.

36 is a configuration block diagram illustrating another embodiment of a digital device. In particular, FIG. 36 illustrates a block diagram of a mobile device as another embodiment of a digital device.

Referring to FIG. 36, the mobile device 3600 includes a wireless communication unit 3610, an audio / video (A / V) input unit 3620, a user input unit 3630, a sensing unit 3640, an output unit 3650, It may include a memory 3660, an interface unit 3670, a control unit 3680, and a power supply unit 3690. The components shown in FIG. 36 are not essential, so a mobile device with more or fewer components may be implemented.

The wireless communication unit 3610 may include one or more modules that enable wireless communication between the mobile device 3600 and the wireless communication system or between a mobile device and a network in which the mobile device is located. For example, the wireless communication unit 3610 may include a broadcast reception module 3611, a mobile communication module 3612, a wireless Internet module 3613, a short-range communication module 3614, a location information module 3615, and the like. .

The broadcast receiving module 3611 receives broadcast signals and / or broadcast related information from an external broadcast management server through a broadcast channel. Here, the broadcast channel may include a satellite channel and a terrestrial channel. The broadcast management server may mean a server that generates and transmits broadcast signals and / or broadcast-related information or a server that receives previously generated broadcast signals and / or broadcast-related information and transmits them to a terminal. The broadcast signal may include a TV broadcast signal, a radio broadcast signal, and a data broadcast signal, and may also include a TV broadcast signal or a radio broadcast signal combined with a data broadcast signal.

The broadcast related information may mean information related to a broadcast channel, broadcast program, or broadcast service provider. Broadcast-related information may also be provided through a mobile communication network. In this case, it may be received by the mobile communication module 3612.

Broadcast-related information may exist in various forms, for example, an electronic program guide (EPG) or an electronic service guide (ESG).

The broadcast receiving module 3611 includes, for example, ATSC, digital video broadcasting-terrestrial (DVB-T), satellite (DVB-S), media forward link only (MediaFLO), handheld (DVB-H), ISDB-T ( Digital broadcasting signals can be received using digital broadcasting systems such as integrated services digital broadcast-terrestrial. Of course, the broadcast receiving module 511 may be configured to be suitable for other broadcasting systems as well as the digital broadcasting system described above.

The broadcast signal and / or broadcast-related information received through the broadcast receiving module 3611 may be stored in the memory 3660.

The mobile communication module 3612 transmits and receives wireless signals to and from at least one of a base station, an external terminal, and a server on a mobile communication network. The wireless signal may include various types of data according to transmission and reception of a voice signal, a video call signal, or a text / multimedia message.

The wireless Internet module 3613 may include a module for wireless Internet access, or may be built in or external to the mobile device 3600. Wireless Internet technology may include wireless LAN (WLAN) (Wi-Fi), wireless broadband (Wibro), world interoperability for microwave access (Wimax), and high speed downlink packet access (HSDPA).

The short-range communication module 3614 refers to a module for short-range communication. Bluetooth, radio frequency identification (RFID), infrared data association (IrDA), ultra wideband (UWB), ZigBee, RS-232, RS-485, etc. may be used as short range communication technology. You can.

The location information module 3615 is a module for obtaining location information of the mobile device 3600, and may use a global position system (GPS) module as an example.

The A / V input unit 3620 is for audio or / and video signal input, and may include a camera 361 and a microphone 3622. The camera 361 processes image frames such as still images or moving images obtained by an image sensor in a video call mode or a shooting mode. The processed image frame may be displayed on the display portion 3651.

The image frames processed by the camera 361 may be stored in the memory 3660 or transmitted to the outside through the wireless communication unit 3610. Two or more cameras 361 may be provided depending on the use environment.

The microphone 3362 receives an external sound signal by a microphone in a call mode or a recording mode, a voice recognition mode, etc., and processes it as electrical voice data. The processed voice data may be converted into a form transmittable to a mobile communication base station through the mobile communication module 3612 and output. A variety of noise reduction algorithms for removing noise generated in the process of receiving an external sound signal may be implemented in the microphone 3362.

The user input unit 3630 generates input data for the user to control the operation of the terminal. The user input unit 3630 may be configured as a key pad, a dome switch, a touch pad (static pressure / power outage), a jog wheel, a jog switch, or the like.

The sensing unit 3640 displays the current state of the mobile device 3600, such as the open / closed state of the mobile device 3600, the location of the mobile device 3600, presence or absence of user contact, orientation of the mobile device, acceleration / deceleration of the mobile device, and the like. It senses and generates a sensing signal for controlling the operation of the mobile device 3600. For example, when the mobile device 3600 is moved or tilted, the position or tilt of the mobile device may be sensed. In addition, it is also possible to sense whether the power supply unit 3690 is supplied with power, whether the interface unit 3670 is coupled with external devices, or the like. Meanwhile, the sensing unit 3640 may include a proximity sensor 3641 including near field communication (NFC).

The output unit 3650 is for generating output related to vision, hearing, or tactile sense, and may include a display unit 3651, an audio output module 3652, an alarm unit 3603, and a haptic module 3654. have.

The display unit 3651 displays (outputs) information processed by the mobile device 3600. For example, when the mobile device is in a call mode, a user interface (UI) or a graphic user interface (GUI) related to the call is displayed. When the mobile device 3600 is in a video call mode or a photographing mode, the photographed or / and received video or UI, GUI is displayed.

The display portion 3651 includes a liquid crystal display (LCD), a thin film transistor-liquid crystal display (TFT LCD), an organic light-emitting diode (OLED), and a flexible display ( flexible display) and a 3D display.

Some of these displays may be of a transparent type or a light transmissive type so that the outside can be seen through them. This may be referred to as a transparent display, and a typical example of the transparent display is a transparent OLED (TOLED). The rear structure of the display portion 3651 may also be configured as a light transmissive structure. With this structure, the user can see an object located behind the terminal body through an area occupied by the display unit 3651 of the terminal body.

Two or more display units 3651 may exist depending on the implementation form of the mobile device 3600. For example, in the mobile device 3600, a plurality of display units may be spaced apart from one surface or integrally disposed, or may be respectively disposed on different surfaces.

When the display unit 3651 and a sensor detecting a touch operation (hereinafter referred to as a “touch sensor”) form a mutual layer structure (hereinafter referred to as a “touch screen”), the display unit 3651 inputs other than an output device It can also be used as a device. The touch sensor may have, for example, a form of a touch film, a touch sheet, a touch pad, and the like.

The touch sensor may be configured to convert a change in pressure applied to a specific portion of the display portion 3651 or capacitance generated in a specific portion of the display portion 3651 into an electrical input signal. The touch sensor may be configured to detect not only the touched position and area, but also pressure at the time of touch.

If there is a touch input to the touch sensor, the corresponding signal (s) is sent to the touch controller. The touch controller processes the signal (s) and then transmits corresponding data to the control unit 3680. As a result, the control unit 3680 can know which area of the display unit 3651 has been touched.

A proximity sensor 3641 may be disposed in an inner area of the mobile device surrounded by the touch screen or near the touch screen. The proximity sensor refers to a sensor that detects the presence or absence of an object approaching a predetermined detection surface or an object present in the vicinity without mechanical contact using electromagnetic force or infrared rays. Proximity sensors have a longer lifespan and higher utilization than contact sensors.

Examples of the proximity sensor include a transmission type photoelectric sensor, a direct reflection type photoelectric sensor, a mirror reflection type photoelectric sensor, a high frequency oscillation type proximity sensor, a capacitive type proximity sensor, a magnetic type proximity sensor, and an infrared proximity sensor. When the touch screen is capacitive, it is configured to detect the proximity of the pointer due to a change in electric field according to the proximity of the pointer. In this case, the touch screen (touch sensor) may be classified as a proximity sensor.

Hereinafter, for convenience of description, the act of causing the pointer to be recognized as being located on the touch screen without being touched by the pointer on the touch screen is referred to as a “proximity touch”, and on the touch screen The act of actually touching the pointer is referred to as "contact touch." The location on the touch screen that is a proximity touch with the pointer means a location where the pointer is perpendicular to the touch screen when the pointer is touched close.

The proximity sensor detects a proximity touch and a proximity touch pattern (eg, proximity touch distance, proximity touch direction, proximity touch speed, proximity touch time, proximity touch position, proximity touch movement state, etc.). Information corresponding to the sensed proximity touch operation and the proximity touch pattern may be output on the touch screen.

The audio output module 3652 may output audio data received from the wireless communication unit 3610 or stored in the memory 3660 in a call signal reception, call mode or recording mode, voice recognition mode, broadcast reception mode, or the like. The sound output module 3652 may also output sound signals related to functions (for example, call signal reception sound, message reception sound, etc.) performed by the mobile device 3600. The sound output module 3652 may include a receiver, a speaker, and a buzzer.

The alarm unit 3603 outputs a signal for notifying the occurrence of the event of the mobile device 3600. Examples of events generated in the mobile device include call signal reception, message reception, key signal input, and touch input. The alarm unit 3603 may also output a signal for notifying the occurrence of an event in a form other than a video signal or an audio signal, for example, vibration.

The video signal or the audio signal may also be output through the display unit 3651 or the audio output module 3652, so that the display unit and the audio output modules 3651 and 3652 may be classified as a part of the alarm unit 3653.

The haptic module 3654 generates various tactile effects that the user can feel. A typical example of the tactile effect generated by the haptic module 3654 is vibration. The intensity and pattern of vibration generated by the haptic module 3654 can be controlled. For example, different vibrations may be synthesized and output or sequentially output.

The haptic module 3654, in addition to vibration, is a pin array that vertically moves with respect to the contact surface of the skin, blast or suction force of the air through the nozzle or intake, grazing on the skin surface, contact with the electrode, and stimulation such as electrostatic force. Various tactile effects can be generated, such as an effect caused by an effect and an effect of reproducing a feeling of cold and warm using an element capable of absorbing heat or generating heat.

The haptic module 3654 can not only deliver the tactile effect through direct contact, but can also be implemented so that the user can feel the tactile effect through muscle sensations such as fingers or arms. Two or more haptic modules 3654 may be provided according to a configuration aspect of the mobile device 3600.

The memory 3660 may store a program for the operation of the control unit 3680, and may temporarily store input / output data (eg, a phone book, a message, a still image, a video, etc.). The memory 3660 may store data related to various patterns of vibration and sound output when a touch is input on the touch screen.

The memory 3660 includes a flash memory type, a hard disk type, a multimedia card micro type, and a card type memory (for example, SD or XD memory, etc.), Random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, It may include a storage medium of at least one of a magnetic disk and an optical disk. The mobile device 3600 may operate in connection with a web storage that performs a storage function of the memory 3660 on the Internet.

The interface unit 3670 serves as a passage with all external devices connected to the mobile device 3600. The interface unit 3670 receives data from an external device, receives power, and transmits data to each component inside the mobile device 3600, or allows data inside the mobile device 3600 to be transmitted to the external device. For example, wired / wireless headset port, external charger port, wired / wireless data port, memory card port, port for connecting devices equipped with an identification module, audio input / output (I / O) port, A video I / O port, an earphone port, and the like may be included in the interface unit 3670.

The identification module is a chip that stores various information for authenticating the usage rights of the mobile device 3600, a user identification module (UIM), a subscriber identification module (SIM), and a universal user authentication module ( universal subscriber identity module, USIM). The device provided with the identification module (hereinafter referred to as an 'identification device') may be manufactured in a smart card format. Therefore, the identification device may be connected to the terminal 3600 through a port.

When the mobile terminal 3600 is connected to an external cradle, the interface unit 3670 becomes a passage through which power from the cradle is supplied to the mobile terminal 3600, or various command signals input from the cradle by the user. It can be a passage to the mobile terminal. Various command signals or power input from the cradle may be operated as signals for recognizing that the mobile terminal is correctly mounted on the cradle.

The control unit 3680 typically controls the overall operation of the mobile device. For example, it performs related control and processing for voice calls, data communication, video calls, and the like. The control unit 3680 may include a multimedia module 3801 for playing multimedia. The multimedia module 3801 may be implemented in the control unit 3680 or may be implemented separately from the control unit 3680. The control unit 3680, particularly the multimedia module 3801, may include the encoding device 100 and / or the decoding device 200 described above.

The control unit 3680 may perform pattern recognition processing capable of recognizing handwriting input or picture drawing input performed on a touch screen as characters and images, respectively.

The power supply 3690 receives external power and internal power under the control of the control unit 3680 and supplies power required for the operation of each component.

The various embodiments described herein can be implemented in a computer- or similar device-readable recording medium using, for example, software, hardware, or a combination thereof.

According to a hardware implementation, the embodiments described herein include application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), It may be implemented using at least one of a processor, a controller, micro-controllers, microprocessors, and electrical units for performing other functions.In some cases, the embodiments described herein may include a control unit ( 3680) can be implemented by itself.

According to the software implementation, embodiments such as procedures and functions described herein may be implemented as separate software modules. Each of the software modules can perform one or more functions and operations described herein. Software code can be implemented in a software application written in an appropriate programming language. Here, the software code is stored in the memory 3660 and can be executed by the control unit 3680.

37 is a configuration block diagram illustrating another embodiment of a digital device.

Other examples of the digital device 3700 include a broadcast receiving unit 3705, an external device interface unit 3735, a storage unit 3740, a user input interface unit 3750, a control unit 3770, a display unit 3780, audio It may include an output unit 385, a power supply unit 3790 and a photographing unit (not shown). Here, the broadcast reception unit 3705 may include at least one tuner 3710, a demodulation unit 3720, and a network interface unit 3730. However, in some cases, the broadcast receiving unit 3705 may include a tuner 3710 and a demodulator 3720, but may not include the network interface unit 3730, and vice versa. Also, although the broadcast receiving unit 3705 is not illustrated, a multiplexer is provided to multiplex the signal demodulated by the demodulator 3720 through the tuner 3710 and the signal received through the network interface unit 3730. It might be. In addition, although the broadcast receiving unit 4025 is not shown, a demultiplexer may be provided to demultiplex the multiplexed signal or demultiplex the demodulated signal or the signal that has passed through the network interface unit 3730. have.

The tuner 3710 receives an RF broadcast signal by tuning a channel selected by a user or all pre-stored channels among radio frequency (RF) broadcast signals received through an antenna. In addition, the tuner 3710 converts the received RF broadcast signal into an intermediate frequency (IF) signal or a baseband signal.

For example, if the received RF broadcast signal is a digital broadcast signal, it is converted into a digital IF signal (DIF), and if it is an analog broadcast signal, it is converted into an analog baseband video or audio signal (CVBS / SIF). That is, the tuner 3710 can process both digital broadcast signals or analog broadcast signals. The analog baseband video or audio signal (CVBS / SIF) output from the tuner 3710 may be directly input to the controller 3770.

In addition, the tuner 3710 may receive a single carrier RF broadcast signal according to an advanced television system committee (ATSC) method or a multiple carrier RF broadcast signal according to a digital video broadcasting (DVB) method.

Meanwhile, the tuner 3710 may sequentially tune and receive RF broadcast signals of all broadcast channels stored through a channel storage function among RF broadcast signals received through an antenna, and convert them into an intermediate frequency signal or a baseband signal. .

The demodulator 3720 receives and demodulates the digital IF signal DIF converted by the tuner 3710. For example, when the digital IF signal output from the tuner 3710 is an ATSC method, the demodulator 3720 performs 8-vestigal side band (8-VSB) demodulation, for example. Also, the demodulator 3720 may perform channel decoding. To this end, the demodulator 3720 is equipped with a trellis decoder, a de-interleaver, a Reed-Solomon decoder, and the like, trellis decoding, deinterleaving, and Reed Soloman decoding can be performed.

For example, when the digital IF signal output from the tuner 3710 is a DVB method, the demodulator 3720 performs coded orthogonal frequency division modulation (COFDMA) demodulation, for example. Also, the demodulator 3720 may perform channel decoding. To this end, the demodulator 3720 may include a convolution decoder, a deinterleaver, and a Reed-Soloman decoder, and perform convolutional decoding, deinterleaving, and Reed Soloman decoding.

The demodulator 3720 may output a stream signal TS after demodulation and channel decoding. In this case, the stream signal may be a signal in which a video signal, an audio signal or a data signal is multiplexed. For example, the stream signal may be an MPEG-2 transport stream (TS) in which an MPEG-2 standard video signal and a Dolby AC-3 standard audio signal are multiplexed. Specifically, the MPEG-2 TS may include a header of 4 bytes and a payload of 184 bytes.

On the other hand, the above-described demodulator 3720 may be provided separately according to the ATSC method and the DVB method. That is, the digital device may separately include an ATSC demodulator and a DVB demodulator.

The stream signal output from the demodulator 3720 may be input to the controller 3770. The control unit 3770 may control demultiplexing, image / audio signal processing, and the like, and control an image output through the display unit 3780 and an audio output unit through the audio output unit 3785.

The external device interface unit 3735 provides an environment such that various external devices are interfaced to the digital device 3700. To this end, the external device interface unit 3735 may include an A / V input / output unit (not shown) or a wireless communication unit (not shown).

The external device interface unit 3735 includes digital versatile disk (DVD), blu-ray, game devices, cameras, camcorders, computers (laptops, tablets), smartphones, Bluetooth devices, and cloud It can be connected to external devices such as (cloud) by wire / wireless. The external device interface unit 3735 transmits a video, audio, or data (including image) signal input from the outside through the connected external device to the controller 3770 of the digital device. The controller 3770 may control the processed video, audio, or data signal to be output to a connected external device. To this end, the external device interface unit 3735 may further include an A / V input / output unit (not shown) or a wireless communication unit (not shown).

A / V input / output unit, USB terminal, CVBS (composite video banking sync) terminal, component terminal, S-video terminal (analog), DVI (DVI) so that video and audio signals from external devices can be input to the digital device 3700. digital visual interface (HDMI) terminal, a high definition multimedia interface (HDMI) terminal, an RGB terminal, a D-SUB terminal, and the like.

The wireless communication unit may perform short-range wireless communication with other electronic devices. The digital device 3700 includes, for example, Bluetooth, radio frequency identification (RFID), infrared data association (IrDA), ultra wideband (UWB), ZigBee, digital living network alliance (DLNA). Networks may be connected to other electronic devices according to a communication protocol.

Also, the external device interface unit 3735 may be connected to at least one of various set-top boxes and various terminals described above, and perform input / output operations with the set-top box.

Meanwhile, the external device interface unit 3735 may receive an application or a list of applications in an adjacent external device, and transmit it to the control unit 3770 or the storage unit 3740.

The network interface unit 3730 provides an interface for connecting the digital device 3700 with a wired / wireless network including an Internet network. The network interface unit 3730 may include, for example, an Ethernet terminal or the like for connection with a wired network, and for example, a wireless LAN (WLAN) (Wi-) for connection with a wireless network. Fi), wireless broadband (Wibro), world interoperability for microwave access (Wimax), and high speed downlink packet access (HSDPA) communication standards.

The network interface unit 3730 may transmit or receive data with other users or other digital devices through a connected network or another network linked to the connected network. In particular, some content data stored in the digital device 3700 may be transmitted to another user registered in advance in the digital device 3700 or a selected user among other digital devices or a selected digital device.

On the other hand, the network interface unit 3730 may access a predetermined web page through a connected network or another network linked to the connected network. That is, it is possible to connect to a predetermined web page through a network, and transmit or receive data with the corresponding server. In addition, content or data provided by a content provider or a network operator may be received. That is, it is possible to receive content such as a movie, advertisement, game, VOD, broadcast signal, and related information provided by a content provider or a network provider through a network. Also, it is possible to receive update information and update files of firmware provided by a network operator. It can also send data to the Internet or content providers or network operators.

Also, the network interface unit 3730 may select and receive a desired application from among applications that are open to the public through a network.

The storage unit 3740 may store programs for processing and controlling each signal in the control unit 3770, and may store signal-processed video, audio, or data signals.

Also, the storage unit 3740 may perform a function for temporarily storing video, audio, or data signals input from the external device interface unit 3735 or the network interface unit 3730. The storage unit 3740 may store information regarding a predetermined broadcast channel through a channel memory function.

The storage unit 3740 may store an application or a list of applications input from the external device interface unit 3735 or the network interface unit 3730.

Also, the storage unit 3740 may store various platforms described later.

The storage unit 3740 includes, for example, a flash memory type, a hard disk type, a multimedia card micro type, and a card type memory (for example, SD or XD) Memory, etc.), RAM (RAM), and ROM (EEPROM, etc.). The digital device 3700 may play and provide content files (video files, still image files, music files, document files, application files, etc.) stored in the storage unit 3740 to the user.

37 illustrates an embodiment in which the storage unit 3740 is provided separately from the control unit 3770, the scope of the present specification is not limited thereto. That is, the storage unit 3740 may be included in the control unit 3770.

The user input interface unit 3750 transmits a signal input by the user to the control unit 3770 or a signal from the control unit 3770 to the user.

For example, the user input interface unit 3750 controls power on / off, channel selection, and screen setting from the remote control device 3800 according to various communication methods such as RF communication method and infrared (IR) communication method. The signal may be received and processed or the control signal of the control unit 3770 may be transmitted to the remote control device 3800.

In addition, the user input interface unit 3750 may transmit a control signal input from a local key (not shown) such as a power key, a channel key, a volume key, and a set value to the controller 3770.

The user input interface unit 3750 transmits a control signal input from a sensing unit (not shown) that senses a user's gesture to the control unit 3770 or senses a signal from the control unit 3770 (Not shown). Here, the sensing unit (not shown) may include a touch sensor, a voice sensor, a position sensor, and a motion sensor.

The controller 3770 de-multiplexes the stream input through the tuner 3710, the demodulator 3720, or the external device interface unit 3735 or processes the demultiplexed signals to generate a signal for video or audio output. And output. The control unit 3770 may include the aforementioned encoding device and / or decoding device.

The image signal processed by the control unit 3770 may be input to the display unit 3780 and displayed as an image corresponding to the corresponding image signal. Also, the image signal processed by the control unit 3770 may be input to an external output device through the external device interface unit 3735.

The audio signal processed by the control unit 3770 may be audio output to the audio output unit 3785. In addition, the audio signal processed by the control unit 3770 may be input to an external output device through the external device interface unit 3735.

Although not illustrated in FIG. 37, the control unit 3770 may include a demultiplexing unit, an image processing unit, and the like.

The control unit 3770 may control the overall operation of the digital device 3700. For example, the control unit 3770 may control the tuner 3710 to tune the RF broadcast corresponding to a channel selected by a user or a pre-stored channel.

The control unit 3770 may control the digital device 3700 by a user command input through the user input interface unit 3750 or an internal program. In particular, it is possible to access a network and download a desired application or application list into the digital device 3700.

For example, the control unit 3770 controls the tuner 3710 to input a signal of a selected channel according to a predetermined channel selection command received through the user input interface unit 3750. And it processes the video, audio or data signal of the selected channel. The control unit 3770 allows the channel information or the like selected by the user to be output through the display unit 3780 or the audio output unit 385 along with the processed image or audio signal.

As another example, the control unit 3770 may be input from an external device input through the external device interface unit 3735, for example, a camera or camcorder, according to an external device image playback command received through the user input interface unit 3750. The video signal or the audio signal can be output through the display unit 3780 or the audio output unit 3785.

Meanwhile, the control unit 3770 may control the display unit 3780 to display an image. For example, a broadcast image input through the tuner 3710, an external input image input through the external device interface unit 3735, an image input through the network interface unit, or an image stored in the storage unit 3740. , It can be controlled to be displayed on the display unit 3780. At this time, the image displayed on the display unit 3780 may be a still image or a video, and may be a 2D image or a 3D image.

Also, the control unit 3770 may control to play the content. The content at this time may be content stored in the digital device 3700, broadcast content received, or external input content input from the outside. The content may be at least one of a broadcast image, an external input image, an audio file, a still image, a connected web screen, and a document file.

On the other hand, when entering the application view item, the controller 3770 may control to display a list of applications or applications that can be downloaded from the digital device 3700 or from an external network.

The control unit 3770 may control to install and operate an application downloaded from an external network along with various user interfaces. Also, an image related to an application to be executed can be controlled to be displayed on the display unit 3780 by the user's selection.

On the other hand, although not shown in the drawing, it is also possible that a channel browsing processing unit for generating a thumbnail image corresponding to a channel signal or an external input signal is further provided.

The channel browsing processing unit receives a stream signal (TS) output from the demodulator 3720 or a stream signal output from the external device interface unit 3735, extracts an image from the input stream signal, and generates a thumbnail image. You can.

The generated thumbnail image can be input to the control unit 3770 as it is or encoded. Also, the generated thumbnail image may be encoded in a stream form and input to the control unit 3770. The controller 3770 may display a list of thumbnails having a plurality of thumbnail images on the display unit 3780 using the input thumbnail images. Meanwhile, the thumbnail images in the thumbnail list can be updated sequentially or simultaneously. Accordingly, the user can easily grasp the contents of a plurality of broadcast channels.

The display unit 3780 converts the image signal, data signal, OSD signal, or image signal received from the external device interface unit 3735 processed by the control unit 3770 into R, G, and B signals, respectively. Generate a drive signal.

The display unit 3780 may be a PDP, LCD, OLED, flexible display, 3D display, or the like.

Meanwhile, the display unit 3780 may be configured as a touch screen and used as an input device in addition to an output device.

The audio output unit 3785 receives a signal processed by the control unit 3770, for example, a stereo signal, a 3.1 channel signal, or a 5.1 channel signal, and outputs the audio. The audio output unit 3785 may be implemented as various types of speakers.

Meanwhile, in order to sense a user's gesture, as described above, a sensing unit (not shown) having at least one of a touch sensor, a voice sensor, a position sensor, and a motion sensor may be further provided in the digital device 3700. . The signal detected by the sensing unit (not shown) may be transmitted to the control unit 3770 through the user input interface unit 3750.

Meanwhile, a photographing unit (not shown) for photographing a user may be further provided. Image information photographed by the photographing unit (not shown) may be input to the control unit 3770.

The control unit 3770 may detect a user's gesture by individually or in combination with an image captured by a photographing unit (not shown) or a detected signal from a sensing unit (not shown).

The power supply unit 3790 supplies the corresponding power to the entire digital device 3700.

Particularly, power is supplied to a control unit 3770 that can be implemented in the form of a system on chip (SOC), a display unit 3780 for displaying an image, and an audio output unit 3785 for audio output. You can.

To this end, the power supply unit 3790 may include a converter (not shown) that converts AC power into DC power. On the other hand, for example, when the display unit 3780 is implemented as a liquid crystal panel having a plurality of backlight lamps, a PWM-operable inverter (not shown) may be further provided for driving luminance or dimming. have.

The remote control device 3800 transmits a user input to the user input interface unit 3750. To this end, the remote control device 3800, Bluetooth (bluetooth), RF (radio frequency) communication, infrared (IR) communication, UWB (Ultra Wideband), ZigBee (ZigBee) method can be used.

In addition, the remote control device 3800 may receive an image, audio or data signal output from the user input interface unit 3750, display it on the remote control device 3800, or output voice or vibration.

The digital device 3700 described above may be a digital broadcast receiver capable of processing a fixed or mobile ATSC type or DVB type digital broadcast signal.

In addition, the digital device according to the present specification may omit some components or further include components not illustrated, as required. On the other hand, as described above, the digital device does not have a tuner and a demodulator, and may receive and play content through a network interface unit or an external device interface unit.

38 is a block diagram illustrating a detailed configuration of the control unit of FIGS. 35 to 37 to illustrate one embodiment.

Examples of the control unit, demultiplexing unit 3810, image processing unit 3820, on-screen display (OSD) generating unit 3840, mixer (mixer) 3850, frame rate converter (FRC) ) 3855, and a formatter 3860. In addition, although not illustrated, the control unit may further include a voice processing unit and a data processing unit.

The demultiplexing unit 3810 demultiplexes the input stream. For example, the demultiplexer 3810 may demultiplex the input MPEG-2 TS video, audio, and data signals. Here, the stream signal input to the demultiplexer 3810 may be a stream signal output from a tuner or demodulator or an external device interface.

The image processing unit 3820 performs image processing of the demultiplexed image signal. To this end, the image processing unit 3820 may include an image decoder 3825 and a scaler 3835.

The video decoder 3825 decodes the demultiplexed video signal, and the scaler 3835 scales the resolution of the decoded video signal to be output from the display unit.

The video decoder 3825 may support various standards. For example, the video decoder 3825 performs the function of the MPEG-2 decoder when the video signal is encoded in the MPEG-2 standard, and the video signal is encoded in the digital multimedia broadcasting (DMB) method or the H.264 standard. In this case, the function of the H.264 decoder can be performed.

Meanwhile, the video signal decoded by the video processing unit 3820 is input to the mixer 3850.

The OSD generation unit 3840 generates OSD data according to a user input or by itself. For example, the OSD generating unit 3840 generates data for displaying various data on a screen of a display unit in a graphic or text form based on a control signal of the user input interface unit. The generated OSD data includes various data such as a user interface screen of a digital device, various menu screens, widgets, icons, and viewing rate information.

The OSD generator 3840 may generate data for displaying subtitles of broadcast images or broadcast information based on EPG.

The mixer 3850 mixes the OSD data generated by the OSD generating unit 3840 and the image signal processed by the image processing unit and provides it to the formatter 3860. Because the decoded video signal and OSD data are mixed, the OSD is displayed overlaid on the broadcast video or the external input video.

A frame rate converter (FRC) 3855 converts a frame rate of an input video. For example, the frame rate converter 3855 may convert the input 60Hz image frame rate to have a frame rate of, for example, 120Hz or 240Hz, depending on the output frequency of the display unit. As described above, various methods may exist in the method for converting the frame rate. For example, when the frame rate converter 3855 converts the frame rate from 60 Hz to 120 Hz, the same first frame is inserted between the first frame and the second frame, or the first frame and the second frame are predicted from the first frame. It can be converted by inserting 3 frames. As another example, when the frame rate converter 3855 converts the frame rate from 60 Hz to 240 Hz, three or more identical frames or predicted frames may be inserted and converted between existing frames. Meanwhile, if a separate frame conversion is not performed, the frame rate conversion unit 3855 may be bypassed.

The formatter 3860 changes the output of the input frame rate conversion unit 3855 to match the output format of the display unit. For example, the formatter 3860 may output R, G, and B data signals, and the R, G, and B data signals may be output as low voltage differential signaling (LVDS) or mini-LVDS Can be. In addition, when the output of the input frame rate conversion unit 3855 is a 3D video signal, the formatter 3860 may support 3D service through the display unit by configuring and outputting a 3D format according to the output format of the display unit.

Meanwhile, a voice processing unit (not shown) in the control unit may perform voice processing of a demultiplexed voice signal. The voice processing unit (not shown) may support various audio formats. For example, even when an audio signal is encoded in formats such as MPEG-2, MPEG-4, AAC, HE-AAC, AC-3, BSAC, a decoder corresponding thereto may be provided and processed.

In addition, the voice processing unit (not shown) in the control unit may process a base, treble, volume control, and the like.

The data processing unit (not shown) in the control unit may perform data processing of the demultiplexed data signal. For example, the data processing unit can decode the demultiplexed data signal even when it is encoded. Here, the encoded data signal may be EPG information including broadcast information such as a start time and an end time of a broadcast program broadcast on each channel.

Meanwhile, the above-described digital device is an example according to the present specification, and each component may be integrated, added, or omitted depending on the specification of the actual digital device. That is, if necessary, two or more components may be combined into one component, or one component may be subdivided into two or more components. In addition, the function performed in each block is for describing an embodiment of the present specification, and the specific operation or device does not limit the scope of the present specification.

Meanwhile, the digital device may be an image signal processing device that performs signal processing of an image stored in the device or an input image. As another example of the video signal processing apparatus, the display unit 3780 and the audio output unit 3785 shown in FIG. 37 are excluded from the set-top box (STB), the above-described DVD player, Blu-ray player, game device, computer And the like can be further exemplified.

39 is a diagram illustrating an example in which a screen of a digital device displays a main image and a sub image simultaneously, according to an embodiment.

The digital device according to an embodiment may simultaneously display the main image 3910 and the auxiliary image 3920 on the screen 3900. The main image 3910 may be called a first image, and the auxiliary image 3920 may be called a second image. The main image 3910 and the auxiliary image 3920 may include, but are not limited to, moving images, still images, electronic program guide (EPG), graphical user interface (GUI), on-screen display (OSD), and the like. The main image 3910 may be an image that is relatively smaller in size than the screen 3900 of the electronic device while being simultaneously displayed with the auxiliary image 3920 on the screen 3900 of the electronic device, and may be referred to as a picture in picture (PIP). Also referred to as. In FIG. 39, the main image 3910 is illustrated as being displayed on the upper left of the screen 3900 of the digital device, but the location where the main image 3910 is displayed is not limited thereto, and the main image 3910 is a digital device. It can be displayed at any location within the screen 3900 of.

The main image 3910 and the auxiliary image 3920 may be directly or indirectly related to each other. As an example, the main image 3910 is a streaming video, and the auxiliary image 3920 may be a GUI that sequentially displays thumbnails of videos including information similar to the streaming video. As another example, the main image 3910 may be a broadcasted image, and the auxiliary image 3920 may be an EPG. As another example, the main image 3910 may be a broadcast image, and the auxiliary image 3920 may be a GUI. Examples of the main image 3910 and the auxiliary image 3920 are not limited thereto.

In one embodiment, the main image 3910 is a broadcast image received through a broadcasting channel, and the auxiliary image 3920 may be information related to a broadcast image received through a broadcast channel. Information related to a broadcast video received through a broadcast channel may include, for example, EPG information including a comprehensive channel schedule, broadcast program detailed information, and broadcast program review information, but is not limited thereto.

In another embodiment, the main image 3910 is a broadcast image received through a broadcast channel, and the auxiliary image 3920 may be an image generated based on information pre-stored in a digital device. The image generated based on the information pre-stored in the digital device may include, for example, a basic user interface (UI) of the EPG, basic channel information, an image resolution manipulation UI, and a bedtime reservation UI. Does not work.

In another embodiment, the main image 3910 is a broadcast image received through a broadcast channel, and the auxiliary image 3920 may be information related to a broadcast image received through a network. The information related to the broadcast image received through the network may be, for example, information obtained through a network-based search engine. More specifically, for example, information related to a character currently being displayed on the main image 3910 may be obtained through a network-based search engine.

However, the example is not limited to this, and information related to a broadcast image received through a network may be obtained by using, for example, an artificial intelligence (AI) system. More specifically, for example, an estimated-location in map of a place currently being displayed on the main image 3910 can be obtained by using network-based deep-learning, and digital The device may receive information about the estimated location on the map of the place currently being displayed on the main image 3910 through the network.

The digital device according to an embodiment may receive at least one of image information of the main image 3910 and image information of the auxiliary image 3920 from the outside. The video information of the main video 3910 includes, for example, a broadcast signal received through a broadcasting channel, a source code information of the main video 3910, and a main received through a network network. IP packet (internet protocol packet) information of the image 3910 may be included, but is not limited thereto. Similarly, the video information of the secondary video 3920 includes, for example, a broadcast signal received through a broadcast channel, source code information of the secondary video 3920, IP packet information of the secondary video 3920 received through a network, etc. It may include, but is not limited to. The digital device may decode and use video information of the main video 3910 or video information of the secondary video 3920 received from the outside. However, in some cases, the digital device may store image information of the main image 3910 or image information of the auxiliary image 3920 internally.

The digital device may display the main image 3910 and the auxiliary image 3920 on the screen 3900 of the digital device based on the image information of the main image 3910 and information related to the auxiliary image 3920.

In one example, the decoding apparatus 200 of the digital device includes a main image decoding apparatus and an auxiliary image decoding apparatus, and the main image decoding apparatus and the auxiliary image decoding apparatus are image information and auxiliary image 3920 of the main image 3910, respectively. ) Can decode video information. The renderer includes a main image renderer (first renderer) and an auxiliary image renderer (second renderer), and the main image renderer displays the main image 3910 on the screen of the digital device (3900) based on information decoded by the main image decoding device. ), And the auxiliary image renderer may display the auxiliary image 3920 in the second area of the screen 3900 of the digital device based on the information decoded by the auxiliary image decoding device. .

In another example, the decoding apparatus 200 of the digital device may decode image information of the main image 3910 and image information of the auxiliary image 3920. Based on the information decoded by the decoding apparatus 200, the renderer may process the main image 3910 and the auxiliary image 3920 together to be simultaneously displayed on the screen 3900 of the digital device.

That is, according to this document, it is possible to provide a method for processing an image service in a digital device. According to the image service processing method, receiving image information, decoding a (main) image based on the image information, rendering or displaying the decoded image in a first area on the display, and second in the display And rendering or displaying an auxiliary image in the area. In this case, the step of decoding the first image may follow the decoding procedure in the decoding apparatus 200 according to FIG. 3 described above. For example, as described above, decoding the first image may include deriving prediction samples for the current block based on inter or intra prediction, and residual samples for the current block based on the received residual information. And generating reconstruction samples based on predictive samples and / or residual samples. Additionally, decoding the first image may include performing an in-loop filtering procedure on the reconstructed picture including the reconstructed samples.

For example, the auxiliary image may be an electronic program guide (EPG), an on-screen display (OSD), or a graphical user interface (GUI). For example, the video information may be received through a broadcast network, and information on the auxiliary video may be received through the broadcast network. For example, the image information may be received through a communication network, and information regarding the auxiliary image may be received through the communication network. For example, the video information may be received through a broadcast network, and information regarding the auxiliary video may be received through a communication network. For example, the image information may be received through a broadcasting network or a communication network, and information regarding the auxiliary image may be stored in a storage medium in the digital device.

The embodiments described above are those in which the components and features of the present specification are combined in a predetermined form. Each component or feature should be considered optional unless stated otherwise. Each component or feature may be implemented in a form that is not combined with other components or features. In addition, it is also possible to constitute an embodiment of the present specification by combining some components and / or features. The order of the operations described in the embodiments herein can be changed. Some configurations or features of one embodiment may be included in other embodiments, or may be replaced with corresponding configurations or features of other embodiments. It is apparent that claims that do not have an explicit citation relationship in the claims may be combined to form an embodiment or be included as a new claim by amendment after filing.

In the case of implementation by firmware or software, an embodiment of the present specification may be implemented in the form of a module, procedure, function, etc. that performs the functions or operations described above. The software code can be stored in memory and driven by a processor. The memory is located inside or outside the processor, and can exchange data with the processor by various means already known.

It will be apparent to those skilled in the art that the present specification may be embodied in other specific forms without departing from essential features of the present specification. Therefore, the above detailed description should not be construed as limiting in all respects and should be considered illustrative. The scope of the present specification should be determined by rational interpretation of the appended claims, and all changes within the equivalent scope of the present specification are included in the scope of the present specification.

The preferred embodiments of the present specification described above have been disclosed for purposes of illustration, and those skilled in the art can improve and change various other embodiments within the technical spirit and technical scope of the present specification disclosed in the appended claims. , Replacement or addition will be possible.

Claims (14)

1. A method for decoding an image signal using inter prediction, comprising:

   Obtaining a motion vector difference (MVD) for the current block;

   Determining a resolution of the MVD based on information on adaptive motion vector resolution (AMVR) and information on a reference picture;

   Scaling the MVD according to the resolution;

   Determining a motion vector for the current block based on the scaled MVD and a motion vector predictor; And

   And generating a prediction sample for the current block based on the motion vector.
2. According to claim 1,

   Information about the AMVR,

   AVMR flag indicating whether to use a predefined native resolution and AMVR accuracy information indicating a unit of the AVMR,

   Information about the reference picture,

   And a picture order count (POC) of the reference picture.
3. According to claim 2,

   Determining the resolution of the MVD,

   And comparing the POC of the current block with the POC of the reference picture after parsing the AMVR flag or AMVR accuracy information.
4. According to claim 2,

   Determining the resolution of the MVD,

   Determining whether the AMVR is applied using the AMVR flag; And

   And if the AMVR is not applied, determining the resolution of the MVD based on whether the POC of the current block and the POC of the reference picture are the same.
5. According to claim 2,

   When the AMVR flag is 1 and the AMVR accuracy information indicates a 4-pel unit, the resolution of the MVD is determined to be 4,

   When the AVMR flag is 1 and the AMVR accuracy information indicates an integer unit, the resolution of the MVD is determined to be 1,

   When the AVMR flag is 0 and the POC of the current block is different from the POC of the reference picture, the resolution of the MVD is determined to be 1/4,

   If the AVMR flag is 0, and the POC of the current block is the same as the POC of the reference picture, the resolution of the MVD is determined to be 1.
6. According to claim 2,

   When the AMVR flag is 1 and the AMVR accuracy information indicates a 4-pel unit, the resolution of the MVD is determined to be 4,

   When the AVMR flag is 1 and the AMVR accuracy information indicates an integer unit and the POC of the current block is different from the POC of the reference picture, the resolution of the MVD is determined to be 1,

   When the AVMR flag is 1 and the AMVR accuracy information indicates an integer unit, while the POC of the current block and the POC of the reference picture are the same, the resolution of the MVD is determined to be 4,

   When the AVMR flag is 0 and the POC of the current block and the POC of the reference picture are the same, the resolution of the MVD is determined to be 1,

   When the AVMR flag is 0 and the POC of the current block is different from the POC of the reference picture, the resolution of the MVD is determined to be 1/4.
7. According to claim 2,

   When the AMVR flag is 1 and the AMVR accuracy information indicates a 4-pel unit, and the POC of the current block and the POC of the reference picture are the same, the resolution of the MVD is determined to be 16,

   When the AMVR flag is 1 and the AMVR accuracy information indicates a 4-pel unit and the POC of the current block is different from the POC of the reference picture, the resolution of the MVD is determined to be 4,

   When the AMVR flag is 1 and the AMVR accuracy information indicates an integer unit, while the POC of the current block and the POC of the reference picture are the same, the resolution of the MVD is determined to be 4,

   When the AMVR flag is 1 and the AMVR accuracy information indicates an integer unit and the POC of the current block is different from the POC of the reference picture, the resolution of the MVD is determined to be 1,

   When the AMVR flag is 0 and the POC of the current block and the POC of the reference picture are the same, the resolution of the MVD is determined to be 1,

   When the AMVR flag is 0 and the POC of the current block is different from the POC of the reference picture, the resolution of the MVD is determined to be 1/4.
8. An apparatus for decoding an image signal using inter prediction, comprising:

   A memory for storing the video signal; And

   And a processor coupled with the memory,

   The processor,

   Obtain a motion vector difference (MVD) for the current block,

   The resolution of the MVD is determined based on information on an adaptive motion vector resolution (AMVR) and information on a reference picture,

   Scale the MVD according to the resolution,

   Determine a motion vector for the current block based on the scaled MVD and a motion vector predictor,

   And generating a prediction sample for the current block based on the motion vector.
9. The method of claim 8,

   Information about the AMVR,

   AVMR flag indicating whether to use a predefined native resolution and AMVR accuracy information indicating a unit of the AVMR,

   Information about the reference picture,

   And a picture order count (POC) of the reference picture.
10. The method of claim 9,

    The processor,

    And parsing the POC of the current block and the POC of the reference picture after parsing the AMVR flag or AMVR accuracy information.
11. The method of claim 9,

    The processor,

    Determine whether the AMVR is applied using the AMVR flag,

    When the AMVR is not applied, the device is configured to determine the resolution of the MVD based on whether the POC of the current block and the POC of the reference picture are the same.
12. The method of claim 9,

    When the AMVR flag is 1 and the AMVR accuracy information indicates a 4-pel unit, the resolution of the MVD is determined to be 4,

    When the AVMR flag is 1 and the AMVR accuracy information indicates an integer unit, the resolution of the MVD is determined to be 1,

    When the AVMR flag is 0 and the POC of the current block is different from the POC of the reference picture, the resolution of the MVD is determined to be 1/4,

    When the AVMR flag is 0 and the POC of the current block and the POC of the reference picture are the same, the resolution of the MVD is determined to be 1.
13. The method of claim 9,

    When the AMVR flag is 1 and the AMVR accuracy information indicates a 4-pel unit, the resolution of the MVD is determined to be 4,

    When the AVMR flag is 1 and the AMVR accuracy information indicates an integer unit and the POC of the current block is different from the POC of the reference picture, the resolution of the MVD is determined to be 1,

    When the AVMR flag is 1 and the AMVR accuracy information indicates an integer unit, while the POC of the current block and the POC of the reference picture are the same, the resolution of the MVD is determined to be 4,

    When the AVMR flag is 0 and the POC of the current block and the POC of the reference picture are the same, the resolution of the MVD is determined to be 1,

    When the AVMR flag is 0 and the POC of the current block is different from the POC of the reference picture, the resolution of the MVD is determined to be 1/4.
14. The method of claim 9,

    When the AMVR flag is 1 and the AMVR accuracy information indicates a 4-pel unit, and the POC of the current block and the POC of the reference picture are the same, the resolution of the MVD is determined to be 16,

    When the AMVR flag is 1 and the AMVR accuracy information indicates a 4-pel unit and the POC of the current block is different from the POC of the reference picture, the resolution of the MVD is determined to be 4,

    When the AMVR flag is 1 and the AMVR accuracy information indicates an integer unit, while the POC of the current block and the POC of the reference picture are the same, the resolution of the MVD is determined to be 4,

    When the AMVR flag is 1 and the AMVR accuracy information indicates an integer unit and the POC of the current block is different from the POC of the reference picture, the resolution of the MVD is determined to be 1,

    When the AMVR flag is 0 and the POC of the current block and the POC of the reference picture are the same, the resolution of the MVD is determined to be 1,

    When the AMVR flag is 0 and the POC of the current block is different from the POC of the reference picture, the resolution of the MVD is determined to be 1/4.

Images