KR20200000543A - Method and apparatus for image enhancement using supervised learning - Google Patents

Abstract

Disclosed are a method for improving an image using supervised learning and a device thereof. According to one aspect of an embodiment of the present invention, a method for decrypting an image by using a CNN-based filter comprises a step of inputting at least one of a quantization parameter map and a block division map and a first picture to the CNN-based filter and a step of outputting a second picture. The quantization parameter map represents the information regarding a decrypting unit constituting the first picture and the block division map represents the information regarding a divided region constituting the first picture. Therefore, the present invention is capable of solving a quantization error and a blocking degradation.

Description

Method and apparatus for image enhancement using supervised learning {Method and apparatus for image enhancement using supervised learning}

The present invention relates to an image improvement method and apparatus using supervised learning.

The contents described in this section merely provide background information on the present invention and do not constitute a prior art.

Image coding technology using deep learning technology is attracting attention by many experts. In-loop filters, bilateral filter, deblocking filter, and sample adaptive offset (SAO), are replaced with one convolutional neural network filter learned with deep learning technology. BDBR (Bjontegaard-delta bit rate) gain of% was achieved under all intra testing conditions (common testing conditions). It is expected that the deep learning technology will be greatly utilized as the experiment proves that the deep learning technology can be effectively combined with the existing image encoding / decoding technology.

Deep learning is a field of machine learning that refers to a set of methods for learning the core structure between large amounts of data through a combination of nonlinear transformations connected in multiple layers. Machine learning can be broadly classified into supervised learning, unsupervised learning, and reinforcement learning. Among these, supervised learning refers to a method of performing learning in a state in which a label that is an explicit answer to the data to be learned is given.

A representative technique of supervised learning is an artificial neural network (hereinafter referred to as NN) which is an engineering model of the motion structure of the human brain. The structure in which NN is stacked in layers (layers) is called a deep neural network. In general, an NN has a structure in which all nodes of each layer are completely connected, and a structure connected to convolution kernels to be suitable for image processing is referred to as a convolutional neural network (hereinafter referred to as 'CNN').

1 is a diagram illustrating a CNN structure for improving an image.

Referring to FIG. 1, the CNN includes an input layer 110, an output layer 120, and a convolutional layer 130. The convolutional layer 130 may be composed of a plurality of layers 132, 134, 136, 138, and 140. Probability distribution models can be constructed based on all convolutional layers.

CNN is largely divided into learning process and inference process. In the learning process, an image to be improved in quality, that is, an image, is input to the input layer as data. The convolution kernel coefficients of each convolution layer are initialized before training, and the error backpropagation algorithm is trained to minimize the error between the image of the output layer and the label of the output layer, that is, the image of the original quality. do. The accuracy of the output layer may vary due to input / output layers, convolutional layer design, and / or error back propagation algorithms during the learning process.

Subsequently, in the deduction process, the convolution kernel coefficients calculated through a plurality of learning processes may be applied to deduce the image having the improved image quality from the image to be improved.

The present embodiment has a main object to provide a method and apparatus for improving an image and solving quantization error and blocking degradation in an image encoder and a decoder.

According to an aspect of the present embodiment, a method of decoding an image using a CNN-based filter, the method comprising: inputting at least one of a quantization parameter map and a block division map and a first picture to the CNN-based filter; Outputting two pictures, wherein the quantization parameter map indicates information about coding units constituting the first picture, and the block partition map indicates information about divided regions constituting the first picture It provides a method for decoding an image characterized by.

According to another aspect of the present embodiment, in the image decoding apparatus using a CNN-based filter, an input unit for receiving at least one of a quantization parameter map and a block division map and a first picture, the quantization parameter map input to the input unit and the A filter unit for applying coefficients of the CNN-based filter to at least one of the block division maps and the first picture, and at least one of the quantization parameter map and the block division map and the CNN-based filter to the first picture An output unit configured to output a second picture by applying coefficients, wherein the quantization parameter map indicates a coding unit constituting the first picture, and the block partition map indicates information about a divided region constituting the first picture It provides a video decoding apparatus characterized in that.

As described above, according to the present embodiment, an image improvement, a quantization error, and a blocking artifact may be solved using a filter learned through supervised learning.

1 is a diagram illustrating a CNN structure for improving an image;
2 is an exemplary block diagram of an image encoding apparatus capable of implementing techniques of this disclosure;
3 is an exemplary block diagram of an image decoding apparatus that may implement techniques of the present disclosure;
4 is a diagram illustrating a CNN-based filter according to an embodiment of the present invention;
5A to 5C are diagrams illustrating a structure of a CNN according to a position of a concatenated layer according to an embodiment of the present invention;
6A to 6C illustrate data to be input to an input layer according to an embodiment of the present invention;
7A and 7B illustrate an example of a block partitioning map according to an embodiment of the present invention;
8A to 8C are views illustrating another example of a block division map according to an embodiment of the present invention;
9A to 9C illustrate block division maps for adjusting the strength of deblocking according to an embodiment of the present invention;
10 is a flowchart of decoding an image using a CNN-based filter according to the present disclosure;
11 is a diagram schematically illustrating a configuration of an apparatus for decoding an image according to the present disclosure.

Hereinafter, some embodiments of the present invention will be described in detail through exemplary drawings. In adding reference numerals to the components of each drawing, it should be noted that the same reference numerals are assigned to the same components as much as possible even though they are shown in different drawings. In addition, in describing the present invention, when it is determined that the detailed description of the related well-known configuration or function may obscure the gist of the present invention, the detailed description thereof will be omitted.

In addition, in describing the component of this invention, terms, such as 1st, 2nd, A, B, (a), (b), can be used. These terms are only for distinguishing the components from other components, and the nature, order or order of the components are not limited by the terms. Throughout the specification, when a part is said to include, 'include' a certain component, which means that it may further include other components, except to exclude other components unless otherwise stated. . In addition, as described in the specification. The terms 'unit' and 'module' refer to a unit that processes at least one function or operation, which may be implemented by hardware or software or a combination of hardware and software.

2 is an exemplary block diagram of an image encoding apparatus that may implement techniques of this disclosure.

The image encoding apparatus includes a block divider 210, a predictor 220, a subtractor 230, a transformer 240, a quantizer 245, an encoder 250, an inverse quantizer 260, and an inverse transform unit ( 265, an adder 270, a filter unit 280, and a memory 290. The image encoding apparatus may be implemented as a hardware chip, or each component may be implemented in software and one or more microprocessors may be implemented to execute a function of software corresponding to each component.

One image (video) is composed of a plurality of pictures. Each picture is divided into a plurality of regions, and encoding is performed for each region. For example, one picture is divided into one or more slices and / or tiles, and each slice or tile is divided into one or more coding tree units (CTUs). Each CTU is divided into one or more coding units (CUs) by a tree structure. Information applied to each CU is encoded as the syntax of the CU, and information commonly applied to the CUs included in one CTU is encoded as the syntax of the CTU.

The block dividing unit 210 determines the size of a coding tree unit (CTU). Information on the size of the CTU (CTU size) is encoded as a syntax of a Sequence Parameter Set (SPS) or Picture Parameter Set (PPS) and transmitted to the image decoding apparatus. After dividing each picture constituting an image into a plurality of coding tree units (CTUs) having a determined size, the block dividing unit 210 recursively uses a tree structure to divide the CTUs. Divide. In the present disclosure, the block dividing unit 210 may further generate a block dividing map for indicating information on the divided region of each picture.

The prediction unit 220 generates a prediction block by predicting the current block. The predictor 220 includes an intra predictor 222 and an inter predictor 224.

In general, current blocks within a picture may each be predictively coded. Prediction of the current block is generally performed using an intra prediction technique (using data from a picture containing the current block) or an inter prediction technique (using data from a picture coded before a picture containing the current block). Can be performed. Inter prediction includes both unidirectional prediction and bidirectional prediction.

The intra predictor 222 predicts pixels in the current block by using pixels (reference pixels) positioned around the current block in the current picture including the current block. There are a plurality of intra prediction modes according to the prediction direction.

The intra predictor 222 may determine an intra prediction mode to use to encode the current block. In some examples, intra prediction unit 222 may encode the current block using several intra prediction modes and select an appropriate intra prediction mode to use from the tested modes. For example, intra predictor 222 calculates rate distortion values using rate-distortion analysis for several tested intra prediction modes, and has the best rate distortion characteristics among the tested modes. Intra prediction mode may be selected.

The intra predictor 222 selects one intra prediction mode from among the plurality of intra prediction modes, and predicts the current block by using a neighboring pixel (reference pixel) and an operation formula determined according to the selected intra prediction mode. Information on the selected intra prediction mode is encoded by the encoder 250 and transmitted to the image decoding apparatus.

The inter prediction unit 224 generates a prediction block for the current block through a motion compensation process. The block most similar to the current block is searched in the coded and decoded reference picture before the current picture, and a predicted block for the current block is generated using the searched block. A motion vector corresponding to a displacement between the current block in the current picture and the prediction block in the reference picture is generated. In general, motion estimation is performed on a luma component, and a motion vector calculated based on the luma component is used for both the luma component and the chroma component. The motion information including the information about the reference picture and the motion vector used to predict the current block is encoded by the encoder 250 and transmitted to the image decoding apparatus.

The subtractor 230 subtracts the prediction block generated by the intra predictor 222 or the inter predictor 124 from the current block to generate a residual block.

The converter 240 converts the residual signal in the residual block having pixel values of the spatial domain into a transform coefficient of the frequency domain. The transform unit 240 may convert the residual signals in the residual block using the size of the current block as a conversion unit, or divide the residual block into a plurality of smaller subblocks and convert the residual signals in a subblock-sized transform unit. You can also convert. There may be various ways of dividing the residual block into smaller subblocks. For example, it may be divided into sub-blocks of a predetermined same size, or a quadtree (QT) scheme may be used in which the residual block is a root node.

The quantization unit 245 quantizes the transform coefficients output from the transform unit 240, and outputs the quantized transform coefficients to the encoder 250.

The encoder 250 generates a bitstream by encoding the quantized transform coefficients by using an encoding method such as CABAC. In addition, the encoder 250 encodes information such as a CTU size, a QT split flag, a BT split flag, a split type, and the like related to block division, so that the image decoding apparatus may split the block in the same manner as the image encoding apparatus.

The encoder 250 encodes information on a prediction type indicating whether the current block is encoded by intra prediction or inter prediction, and encodes the intra prediction information according to the prediction type (ie, the intra prediction mode). Information) or inter prediction information (information about a reference picture and a motion vector) is encoded.

The inverse quantizer 260 inversely quantizes the quantized transform coefficients output from the quantizer 245 to generate transform coefficients. The inverse transformer 265 restores the residual block by converting the transform coefficients output from the inverse quantizer 260 from the frequency domain to the spatial domain.

The adder 270 reconstructs the current block by adding the reconstructed residual block and the predicted block generated by the predictor 220. The pixels in the reconstructed current block are used as reference pixels when intra prediction of the next order of blocks.

The filter unit 280 filters the reconstructed pixels to reduce blocking artifacts, ringing artifacts, blurring artifacts, and the like caused by block-based prediction and transformation / quantization. Do this. The filter unit 280 may include a deblocking filter 282 and a SAO filter 284.

The deblocking filter 280 filters the boundaries between the reconstructed blocks to remove blocking artifacts caused by block-by-block encoding / decoding, and the SAO filter 284 adds an additional block to the deblocking filtered image. Perform filtering. The SAO filter 284 is a filter used to compensate for the difference between the reconstructed pixel and the original pixel caused by lossy coding.

The reconstructed block filtered through the deblocking filter 282 and the SAO filter 284 is stored in the memory 290. When all the blocks in a picture are reconstructed, the reconstructed picture is used as a reference picture for inter prediction of a block in a picture to be encoded later.

3 is an exemplary block diagram of an image decoding apparatus that may implement techniques of the present disclosure.

The image decoding apparatus includes an image decompressor 300 including a decoder 310, an inverse quantizer 320, an inverse transformer 330, a predictor 340, an adder 350, and a filter 360. And a memory 370. Like the image encoding apparatus of FIG. 2, the image decoding apparatus may be implemented by each component as a hardware chip, or may be implemented by software and a microprocessor to execute a function of software corresponding to each component.

The decoder 310 decodes the bitstream received from the image encoding apparatus, extracts information related to block division, determines a current block to be decoded, and includes prediction information and residual signal information necessary for reconstructing the current block. Extract

The decoder 310 extracts information on the CTU size from a Sequence Parameter Set (SPS) or Picture Parameter Set (PPS) to determine the size of the CTU, and divides the picture into a CTU of the determined size. The CTU is determined as the highest layer of the tree structure, that is, the root node, and the CTU is partitioned using the tree structure by extracting partition information about the CTU.

In addition, when the decoder 310 determines the current block to be decoded by splitting the tree structure, the decoder 310 extracts information on a prediction type indicating whether the current block is intra predicted or inter predicted.

When the prediction type information indicates intra prediction, the decoder 310 extracts a syntax element for intra prediction information (intra prediction mode) of the current block.

When the prediction type information indicates inter prediction, the decoder 310 extracts a syntax element for the inter prediction information, that is, a motion vector and information indicating a reference picture to which the motion vector refers.

Meanwhile, the decoder 310 extracts information on the quantized transform coefficients of the current block as information on the residual signal.

The inverse quantization unit 320 inversely quantizes the quantized transform coefficients, and the inverse transform unit 330 inversely transforms the inverse quantized transform coefficients from the frequency domain to the spatial domain to generate a residual block for the current block.

The predictor 340 includes an intra predictor 342 and an inter predictor 344. The intra predictor 342 is activated when the intra prediction is the prediction type of the current block, and the inter predictor 344 is activated when the intra prediction is the prediction type of the current block.

The intra predictor 342 determines the intra prediction mode of the current block among the plurality of intra prediction modes from the syntax element for the intra prediction mode extracted from the decoder 310, and references pixels around the current block according to the intra prediction mode. Predict the current block using

The inter prediction unit 344 determines the motion vector of the current block and the reference picture to which the motion vector refers by using the syntax elements of the intra prediction mode extracted from the decoder 310, and determines the current motion vector and the reference picture. Predict the block.

The adder 350 reconstructs the current block by adding the residual block output from the inverse transformer and the prediction block output from the inter predictor or the intra predictor. The pixels in the reconstructed current block are utilized as reference pixels when intra prediction of a block to be decoded later.

By sequentially reconstructing the current blocks corresponding to the CUs by the image reconstructor 300, a CTU composed of CUs and a picture composed of CTUs are reconstructed.

The filter unit 360 includes a deblocking filter 362 and a SAO filter 364. The deblocking filter 362 deblocks and filters the boundary between the reconstructed blocks to remove blocking artifacts caused by block-by-block decoding. The SAO filter 364 performs additional filtering on the reconstructed block after the deblocking filtering to compensate for the difference between the reconstructed pixel and the original pixel caused by lossy coding. The reconstructed block filtered through the deblocking filter 362 and the SAO filter 364 is stored in the memory 370. When all the blocks in a picture are reconstructed, the reconstructed picture is used as a reference picture for inter prediction of a block in a picture to be encoded later.

In the present disclosure, a CNN-based filter having the functions of the deblocking filters 182 and 362 and the SAO filters 284 and 464 will be described in detail. The CNN-based filter according to the present disclosure may be used in both an image encoding apparatus and an image decoding apparatus.

In addition, although the present disclosure describes YUV as an example of information constituting a picture, it may be applied to RGB, YCbCr, and the like. That is, the YUV for improving image quality may be RGB for improving image quality or YCbCr for improving image quality.

4 illustrates a CNN-based filter according to an embodiment of the present invention.

At least one of a quantization parameter (QP) map 403 and a block partition map 405 and a YUV 401 for improving image quality are input to an input layer, and a YUV difference ( 421 is output. Here, the YUV 401 to improve the image quality may be the YUV 401 restored from the bit stream received from the encoder, and means a YUV in which the original YUV is artificially or unartificially damaged. In addition, a hint (not shown) may also be input to the input layer.

First, in the learning process, the coefficients of the CNN-based filter, that is, the coefficients of the convolution kernel, are trained such that the YUV difference 421 outputted to the output layer is the difference between the original YUV and the YUV that will improve the image quality. The convolution kernel is available in 2D and 3D forms. The CNN-based filter 411 is for improving the quality of the input YUV, and the final output of the CNN-based filter 411 is referred to as YUV, that is, the YUV 431 with improved image quality.

Here, the YUV to improve the image quality may be filtered for each channel or at once.

The size of the QP map may be set to the same resolution as the input YUV to be filtered, and the value of the QP map may be filled with QP values used in coding units in the YUV plane, for example, blocks or sub-blocks. At this time, if the YUV is filtered for each channel, one map may be configured with the QP value of the channel to be filtered. If the YUV is filtered at one time, the QP values of the three channels may consist of three separate maps or one map having an average QP value.

As a method for improving the accuracy of the CNN technique, block mode map information useful for the learning process may be added as hint information in addition to the QP map, the block division map, and the image to be improved in quality as an input layer. Here, the block mode map may be filled with a mode value used in a coding unit, for example, a block or a sub block. For example, the information may be information for distinguishing whether a block is encoded in an intra mode or an inter mode, and the information may be represented by a number. In this case, the convolution kernel coefficient that is a result of the learning process may be set to include not only the data of the input layer but also a hint. Basically, the input layer and output layer of the learning process and the inference process of the CNN technique should be configured identically.

Subsequently, in the inference process, the YUV with improved image quality is generated from the YUV, the quantization parameter map, and the block partitioning map to improve the image quality by applying the coefficients of the CNN-based filter obtained in the learning process.

5A to 5C are diagrams illustrating the structure of a CNN according to a position of a concatenated layer according to an embodiment of the present invention.

In detail, the YUV 501, the quantization parameter map 503, and the block partition map 505, which will improve the image quality input to the input layer 510, may be concatenated through the concatenate layer 520 during the CNN process. Can be. However, the position of the contiguous layer 520 may be changed.

5A illustrates that a concatenated layer 520 is positioned immediately after the input layer 510 so that a YUV 501, a quantization parameter map 503, and a block partition map 505 are input to the input layer 510 to improve image quality. It shows the structure of the CNN that is immediately connected.

FIG. 5B illustrates the structure of the CNN in which the concatenation layer 520 is located between the convolution layers 530. FIG. 5C illustrates the structure of the CNN in which the concatenation layer 520 is located directly in front of the output layer 540.

6A through 6C illustrate data to be input to an input layer according to an embodiment of the present invention.

In detail, FIG. 6A illustrates that a Y plane (for example, a Y coding tree block (CTB)) to improve image quality is omitted in a pixel value of luminance luma to improve image quality. QP map applied to the Y plane to improve the image quality, Figure 6c shows a block division map of the Y plane to improve the image quality.

Hereinafter, various structures of a block division map according to an embodiment of the present invention will be described. The block partitioning map indicates whether a block is divided or not, so that the processing of the partitioned boundary of the block and the inner region of the block may be differently performed during the CNN's learning process and inference process.

7A and 7B are diagrams illustrating an example of a block partitioning map according to an embodiment of the present invention.

The block division map may be set to the same resolution as the YUV plane to be filtered and may be configured to indicate whether to block the partition. For example, when the YUV plane is composed of a coding tree block including a plurality of coding blocks, that is, coding blocks (CBs), the block partitioning map may represent a division boundary of the coding blocks in the coding tree block. FIG. 7A illustrates a coding tree block divided by a quadtree plus binary tree (QTBT) scheme, and FIG. 7B illustrates a block partitioning map according to the coding tree block. Referring to FIG. 7B, the boundaries of the coding blocks are indicated by '1' and the inside of the coding blocks are indicated by '0' in the block division map.

8A to 8B are diagrams illustrating another example of a block partitioning map according to an embodiment of the present invention.

In FIGS. 8A to 8B, when the YUV plane consists of coding tree blocks, blocking deterioration processing on boundaries of the coding tree blocks may not be possible, and thus an extra area α may be added to the YUV plane. 8A and 8B show an example in which 2 pixels are set as the extra area α, but other values may be set. In addition, the boundary of the coding tree block may be indicated by the extra area, and whether or not the block is split may be displayed also in the area outside the coding tree block. If the filtering includes an extra area, an area overlapping with another adjacent coding tree block is generated after the filtering, and the overlapping area may be processed as an average value. Specifically, referring to FIG. 8C, when the coding tree blocks 801 and 803 including the extra areas are adjacent to each other, an overlapping area 805 is formed. The overlapping region 805 may be set to an average value of values of adjacent coding tree blocks 801 and 803.

9A to 9C illustrate block division maps for adjusting the strength of deblocking according to an embodiment of the present invention.

In the previous embodiment, the boundary of the coding block is distinguished by 1 pixel. When the value of the 1 pixel is 0, the inside of the coding block is represented, and the 1 is the boundary of the coding block.

In FIG. 9A, the boundaries of the coding blocks are represented by the number of pixels (or, pixel width, luma sample line, luma sample length, etc.) in order to control the strength of de-blocking. The number of pixels may be determined by at least one of a size of a coding block, a value of a quantization parameter, and an encoding mode. For example, as shown in FIG. 9A, when the coding block is large, the number of pixels may be set to two, and when the coding block is small, the number of pixels may be set to one. In addition, if the value of the quantization parameter is large, the number of pixels may be increased. If the value of the quantization parameter is small, the number of pixels may be set. As another example, when the encoding mode is intra, the number of the pixels may be increased, and when the encoding mode is inter, the number of the pixels may be set to be small. These can all be set in reverse.

The number of pixels may mean the number of pixels located at a block boundary to be updated by filtering. For example, when a 3 pixel value in one block located at the block boundary line is to be updated, the boundary of the block may be indicated by 3 pixels in the block division map. As another example, the number of pixels may mean the number of pixels located at a block boundary line to be referred to for filtering. For example, when filtering is performed by referring to 4 pixel values of a block located at a block boundary line, the block boundary may be indicated by 4 pixels in the block division map.

In FIG. 9B, the boundary values of the coding blocks are differently represented in order to control the strength of deblocking. The boundary value of the coding block may be determined by at least one of a size of a coding block, a value of a quantization parameter, an encoding mode, a number of pixels to be updated, and a number of pixels to be referred to for filtering. As shown in FIG. 7A, when the coding block is large, the value of the quantization parameter is large, or the coding mode is intra, the boundary value of the coding block can be set large, and conversely, the coding block is small, or the value of the quantization parameter is small. If the encoding mode is inter, the boundary value of the coding block may be set small. Both of these can also be set in reverse.

FIG. 9C shows the number of pixels at the coding block boundary and the boundary value of the coding block in order to adjust the strength of deblocking. Descriptions thereof will be omitted herein as described with reference to FIGS. 9A and 9B.

As described above, the configured block partitioning map is used in the learning process to help the CNN filter to operate as a strong deblocking filter.

10 is a flowchart of decoding an image using a CNN-based filter according to the present disclosure.

At least one of a quantization parameter map and a block partitioning map and a YUV for improving image quality are input to the CNN-based filter (1001). The quantization parameter map may be set to the same resolution as the YUV to improve the image quality. The block partitioning map may be displayed differently between the partitioned boundary of the block and the inner region of the block. The number and value of pixels representing the partitioned boundary of the block in the block division map are determined by at least one of the size of the coding block, the value of the quantization parameter, the encoding mode, the number of pixels to be updated, and the number of pixels to be referred to for filtering. Can be determined.

As an input of a YUV, a quantization parameter map, and a block partitioning map for improving image quality, a YUV having improved image quality is output using the coefficients of the CNN-based filter learned by using the original YUV as a final output (1003). When a hint such as a block mode map is additionally input to the CNN-based filter, the hint of the CNN-based filter is additionally input and learned.

The processes shown in FIG. 10 may be embodied as computer readable codes on a computer readable recording medium. The computer-readable recording medium includes all kinds of recording devices in which data that can be read by a computer system is stored. That is, the computer-readable recording medium may be a magnetic storage medium (for example, ROM, floppy disk, hard disk, etc.), an optical reading medium (for example, CD-ROM, DVD, etc.) and a carrier wave (for example, the Internet Storage medium). The computer readable recording medium can also be distributed over network coupled computer systems so that the computer readable code is stored and executed in a distributed fashion.

11 is a diagram schematically illustrating a configuration of an apparatus for decoding an image according to the present disclosure.

The apparatus for decoding the image may include an input unit 1101, a filter unit 1103, and an output unit 1105. Other configurations may be included, but a description of components not directly related to the present disclosure will be omitted.

The input unit 1101 receives at least one of a quantization parameter map and a block partitioning map and a YUV for improving image quality. The quantization parameter map may be set to the same resolution as the YUV to improve the image quality, and the block partitioning map may be displayed differently between the partitioned boundary of the block and the internal region of the block. The number and value of pixels representing the divided boundary of the block in the block division map may be determined by at least one of a size of a coding block, a value of a quantization parameter, and an encoding mode.

The filter unit 1103 applies coefficients of the learned CNN-based filter to at least one of the quantization parameter map and the block division map input to the input unit 1101 and YUV to improve the image quality.

 The output unit 1105 outputs at least one of the input quantization parameter map and the block division map and the YUV having the improved image quality generated by applying the coefficients of the CNN-based filter learned to the YUV to improve the image quality.

In the present disclosure, the input unit 1101, the filter unit 1103, and the output unit 1105 are described separately, but may be integrated into one configuration and implemented, or one configuration may be implemented by dividing into multiple configurations.

The above description is merely illustrative of the technical idea of the present embodiment, and those skilled in the art to which the present embodiment belongs may make various modifications and changes without departing from the essential characteristics of the present embodiment. Therefore, the present embodiments are not intended to limit the technical idea of the present embodiment but to describe the present invention, and the scope of the technical idea of the present embodiment is not limited by these embodiments. The scope of protection of the present embodiment should be interpreted by the following claims, and all technical ideas within the scope equivalent thereto should be construed as being included in the scope of the present embodiment.

Claims (16)

In the video decoding method using a CNN (Convolutional Neural Network) based filter,
Inputting at least one of a quantization parameter map and a block division map and a first picture to the CNN-based filter; And
Outputting a second picture,
The quantization parameter map indicates information about coding units constituting the first picture, and the block division map indicates information about divided regions constituting the first picture. The method of claim 1,
And a coefficient of the CNN-based filter is trained using at least one of a quantization parameter map and a block division map, a third picture, and an original picture. The method of claim 1,
The quantization parameter map is set to the same resolution as the first picture. The method of claim 1,
And inputting a block mode map indicating an encoding mode to the CNN-based filter. The method of claim 1,
And the block division map displays different values of the divided boundary of the block and the internal region of the block. The method of claim 1,
The number of pixels representing the divided boundary of the block in the block division map is determined by at least one of the size of the coding block, the value of the quantization parameter, the encoding mode, the number of pixels to be updated, and the number of pixels to be referred to for filtering. An image decoding method characterized by. The method of claim 1,
In the block division map, a value of a pixel representing a partitioned boundary of a block is determined by at least one of a size of a coding block, a value of a quantization parameter, an encoding mode, a number of pixels to be updated, and a number of pixels to be referred to for filtering. An image decoding method characterized by. The method of claim 1,
The coefficient of the CNN-based filter is an image decoding method, characterized in that received from the device for encoding the image. In the image decoding apparatus using a CNN (Convolutional Neural Network) based filter,
An input unit configured to receive at least one of a quantization parameter map and a block division map and a first picture;
A filter unit which applies coefficients of the CNN-based filter to at least one of the quantization parameter map and the block division map and the first picture input to the input unit; And
An output unit for outputting a second picture by applying at least one of the quantization parameter map and the block division map and coefficients of the CNN-based filter to the first picture,
And wherein the quantization parameter map indicates information about coding units constituting the first picture, and the block partition map indicates information about divided regions constituting the first picture. The method of claim 9,
The coefficient of the CNN-based filter is trained using at least one of a quantization parameter map, a block division map, a third picture, and an original picture. The method of claim 9,
And the quantization parameter map is set to the same resolution as the first picture. The method of claim 9,
The input unit further receives a block mode map indicating an encoding mode,
And the coefficient of the CNN-based filter is additionally learned by further inputting the block mode map. The method of claim 9,
And the block partitioning map differently displays the divided boundary of the block and the value of the inner region of the block. The method of claim 9,
The number of pixels representing the divided boundary of the block in the block division map is determined by at least one of the size of the coding block, the value of the quantization parameter, the encoding mode, the number of pixels to be updated, and the number of pixels to be referred to for filtering. An image decoding device. The method of claim 9,
In the block division map, a value of a pixel representing a partitioned boundary of a block is determined by at least one of a size of a coding block, a value of a quantization parameter, an encoding mode, a number of pixels to be updated, and a number of pixels to be referred to for filtering. An image decoding device. The method of claim 9,
And a coefficient of the CNN-based filter is received from an apparatus for encoding an image.

Images