KR102648464B1 - Method and apparatus for image enhancement using supervised learning - Google Patents

Abstract

Disclosed is an image improvement method and device using supervised learning.
According to one aspect of the present embodiment, in a method of decoding an image using a CNN-based filter, inputting at least one of a quantization parameter map and a block partition map and a first picture into the CNN-based filter, and A step of outputting two pictures, wherein the quantization parameter map represents information about coding units constituting the first picture, and the block partition map represents information about a divided region constituting the first picture. Provides a method of decoding an image characterized by .

Description

Image improvement method and apparatus using supervised learning {Method and apparatus for image enhancement using supervised learning}

The present invention relates to an image improvement method and device using supervised learning.

The content described in this section simply provides background information about the present invention and does not constitute prior art.

Video encoding technology using deep learning technology is receiving attention from many experts. By replacing the in-loop filters, bilateral filter, deblocking filter, and sample adaptive offset (SAO) with a single convolutional neural network filter learned using deep learning technology, 3.57 % BDBR (Bjontegaard-delta bit rate) gain was achieved under all intra-testing conditions (common testing conditions). As experiments have proven that deep learning technology can be effectively combined with existing video encoding and decoding technology, it is expected that deep learning technology will be greatly utilized.

Deep learning is a field of machine learning and refers to a set of methods that learn the core structure between large amounts of data through a combination of non-linear transformation techniques connected to multiple layers. Machine learning can be broadly classified into supervised learning, unsupervised learning, and reinforcement learning depending on the learning method. Among these, supervised learning refers to a method of performing learning in a state where a label, which is an explicit correct answer to the learning data, is given.

A representative technique among supervised learning is an artificial neural network (Neural Network, hereinafter referred to as 'NN'), which is an engineered model of the operating structure of the human brain. A structure in which NNs are stacked deeply into multiple layers is called a deep neural network. In general, NN has a structure in which all nodes in each layer are fully connected, and the structure connected with convolution kernels to be suitable for image processing is called a convolutional neural network (hereinafter referred to as 'CNN').

Figure 1 is a diagram showing a CNN structure for image improvement.

Referring to Figure 1, CNN includes an input layer 110, an output layer 120, and a convolution layer 130. The convolutional layer 130 may be composed of multiple layers 132, 134, 136, 138, and 140. A probability distribution model can be constructed based on all convolutional layers.

CNN is largely divided into a learning process and an inference process. During the learning process, the image to be improved is input to the input layer as data. The convolution kernel coefficients of each convolution layer are initialized before learning, and the error backpropagation algorithm is used to minimize the error between the image, which is the data of the output layer, and the label of the output layer, that is, the image of the original quality. do. The accuracy of the output layer may vary depending on the input/output layer, convolution layer design, and/or error backpropagation algorithm during the learning process.

In the subsequent inference process, the image with improved quality can be inferred from the image to be improved by applying the convolution kernel coefficient calculated through multiple learning processes.

The main purpose of this embodiment is to provide a method and device that can improve images and solve quantization errors and blocking degradation in image encoders and decoders.

According to one aspect of the present embodiment, in a method of decoding an image using a CNN-based filter, inputting at least one of a quantization parameter map and a block partition map and a first picture into the CNN-based filter, and A step of outputting two pictures, wherein the quantization parameter map represents information about coding units constituting the first picture, and the block partition map represents information about a divided region constituting the first picture. Provides a method for decoding an image characterized by .

According to another aspect of the present embodiment, in an image decoding device using a CNN-based filter, an input unit that receives at least one of a quantization parameter map and a block partition map and a first picture, the quantization parameter map input to the input unit, and the A filter unit for applying coefficients of the CNN-based filter to at least one of the block partition map and the first picture, and at least one of the quantization parameter map and the block partition map and the CNN-based filter to the first picture. An output unit that applies a coefficient to output a second picture, wherein the quantization parameter map represents a coding unit constituting the first picture, and the block partition map is information about a divided region constituting the first picture. Provides a video decoding device characterized in that it represents.

As described above, according to this embodiment, image improvement, quantization error, and blocking artifacts can be resolved using a filter learned through supervised learning.

Figure 1 is a diagram showing a CNN structure for image improvement,
2 is a diagram illustrating an example block diagram of a video encoding device capable of implementing the techniques of the present disclosure;
3 is a diagram illustrating an example block diagram of an image decoding device capable of implementing the techniques of the present disclosure;
Figure 4 is a diagram showing a CNN-based filter according to an embodiment of the present invention;
Figures 5a to 5c are diagrams showing the structure of a CNN according to the location of the concatenated layer according to an embodiment of the present invention;
6A to 6C are diagrams showing data to be input to an input layer according to an embodiment of the present invention;
7A and 7B are diagrams showing an example of a block division map according to an embodiment of the present invention;
8A to 8C are diagrams showing another example of a block division map according to an embodiment of the present invention;
9A to 9C are diagrams showing block division maps for adjusting the strength of deblocking according to an embodiment of the present invention;
FIG. 10 is a flow chart illustrating an image decoding using a CNN-based filter according to the present disclosure;
Figure 11 is a diagram schematically showing the configuration of an apparatus for decoding an image according to the present disclosure.

Hereinafter, some embodiments of the present invention will be described in detail through illustrative drawings. When adding reference numerals to components in each drawing, it should be noted that identical components are given the same reference numerals as much as possible even if they are shown in different drawings. Additionally, in describing the present invention, if it is determined that a detailed description of a related known configuration or function may obscure the gist of the present invention, the detailed description will be omitted.

Additionally, when describing the components of the present invention, terms such as first, second, A, B, (a), and (b) may be used. These terms are only used to distinguish the component from other components, and the nature, sequence, or order of the component is not limited by the term. Throughout the specification, when a part is said to 'include' or 'have' a certain component, this means that it does not exclude other components but may further include other components, unless specifically stated to the contrary. . In addition, ‘…’ stated in the specification. Terms such as 'unit' and 'module' refer to a unit that processes at least one function or operation, and may be implemented as hardware, software, or a combination of hardware and software.

FIG. 2 is a diagram illustrating an example block diagram of a video encoding device capable of implementing the techniques of the present disclosure.

The image encoding device includes a block division unit 210, a prediction unit 220, a subtractor 230, a transform unit 240, a quantization unit 245, an encoder 250, an inverse quantization unit 260, and an inverse transform unit ( 265), an adder 270, a filter unit 280, and a memory 290. Each component of an image encoding device may be implemented as a hardware chip, or may be implemented as software and one or more microprocessors may be implemented to execute software functions corresponding to each component.

One image (video) consists of multiple pictures. Each picture is divided into a plurality of regions and encoding is performed for each region. For example, one picture is divided into one or more slices and/or tiles, and each slice or tile is divided into one or more Coding Tree Units (CTUs). And each CTU is divided into one or more CUs (Coding Units) by a tree structure. Information applied to each CU is encoded as the syntax of the CU, and information commonly applied to CUs included in one CTU is encoded as the syntax of the CTU.

The block division unit 210 determines the size of the CTU (Coding Tree Unit). Information about the size of the CTU (CTU size) is encoded as a syntax of SPS (Sequence Parameter Set) or PPS (Picture Parameter Set) and transmitted to the video decoding device. The block division unit 210 divides each picture constituting the image into a plurality of CTUs (Coding Tree Units) of a determined size, and then recursively divides the CTUs using a tree structure. Divide. In the present disclosure, the block division unit 210 may additionally generate a block division map to indicate information about the divided regions of each picture.

The prediction unit 220 predicts the current block and generates a prediction block. The prediction unit 220 includes an intra prediction unit 222 and an inter prediction unit 224.

In general, each current block in a picture can be coded predictively. Prediction of the current block is typically performed using intra prediction techniques (using data from the picture containing the current block) or inter prediction techniques (using data from pictures coded before the picture containing the current block). It can be done. Inter prediction includes both one-way prediction and two-way prediction.

The intra prediction unit 222 predicts pixels within the current block using pixels (reference pixels) located around the current block within the current picture including the current block. There are multiple intra prediction modes depending on the prediction direction.

The intra prediction unit 222 may determine the intra prediction mode to be used to encode the current block. In some examples, intra prediction unit 222 may encode the current block using multiple intra prediction modes and select an appropriate intra prediction mode to use from the tested modes. For example, the intra prediction unit 222 calculates rate-distortion values using rate-distortion analysis for several tested intra-prediction modes, and selects the rate-distortion values with the best rate-distortion characteristics among the tested modes. You can also select intra prediction mode.

The intra prediction unit 222 selects one intra prediction mode from a plurality of intra prediction modes and predicts the current block using surrounding pixels (reference pixels) and an operation equation determined according to the selected intra prediction mode. Information about the selected intra prediction mode is encoded by the encoder 250 and transmitted to the video decoding device.

The inter prediction unit 224 generates a prediction block for the current block through a motion compensation process. The block most similar to the current block is searched within a reference picture that has been encoded and decoded before the current picture, and a prediction block for the current block is generated using the searched block. Then, a motion vector corresponding to the displacement between the current block in the current picture and the prediction block in the reference picture is generated. Typically, motion estimation is performed on the luma component, and motion vectors calculated based on the luma component are used for both the luma component and the chroma component. Motion information including information about reference pictures and motion vectors used to predict the current block is encoded by the encoder 250 and transmitted to the video decoding device.

The subtractor 230 generates a residual block by subtracting the prediction block generated by the intra prediction unit 222 or the inter prediction unit 124 from the current block.

The converter 240 converts the residual signal in the residual block with pixel values in the spatial domain into a transform coefficient in the frequency domain. The converter 240 can convert the residual signals in the residual block using the size of the current block as a conversion unit, or divide the residual block into a plurality of smaller subblocks and convert the residual signals into a conversion unit of the subblock size. You can also convert it. There may be various ways to divide the residual block into smaller subblocks. For example, it may be divided into predefined subblocks of the same size, or QT (quadtree) type division may be used with the residual block as the root node.

The quantization unit 245 quantizes the transform coefficients output from the transform unit 240 and outputs the quantized transform coefficients to the encoder 250.

The encoder 250 generates a bitstream by encoding the quantized transform coefficients using an encoding method such as CABAC. In addition, the encoder 250 encodes information such as CTU size, QT split flag, BT split flag, and split type related to block splitting, so that the video decoding device can split the block in the same way as the video coding device.

The encoder 250 encodes information about the prediction type indicating whether the current block is encoded by intra prediction or inter prediction, and generates intra prediction information (i.e., information about the intra prediction mode) according to the prediction type. information) or inter prediction information (information about reference pictures and motion vectors) is encoded.

The inverse quantization unit 260 inversely quantizes the quantized transform coefficients output from the quantization unit 245 to generate transform coefficients. The inverse transform unit 265 restores the residual block by converting the transform coefficients output from the inverse quantization unit 260 from the frequency domain to the spatial domain.

The addition unit 270 restores the current block by adding the restored residual block and the prediction block generated by the prediction unit 220. Pixels in the restored current block are used as reference pixels when intra-predicting the next block.

The filter unit 280 filters the restored pixels to reduce blocking artifacts, ringing artifacts, blurring artifacts, etc. that occur due to block-based prediction and transformation/quantization. Perform. The filter unit 280 may include a deblocking filter 282 and a SAO filter 284.

The deblocking filter 280 filters the boundaries between restored blocks to remove blocking artifacts caused by block-level encoding/decoding, and the SAO filter 284 adds additional information to the deblocking-filtered image. Perform filtering. The SAO filter 284 is a filter used to compensate for the difference between the restored pixel and the original pixel caused by lossy coding.

The restored block filtered through the deblocking filter 282 and the SAO filter 284 is stored in the memory 290. When all blocks in one picture are reconstructed, the reconstructed picture is later used as a reference picture for inter prediction of blocks in the picture to be encoded.

FIG. 3 is a diagram illustrating an example block diagram of an image decoding device capable of implementing the techniques of the present disclosure.

The image decoding device includes an image restorer 300 including a decoder 310, an inverse quantization unit 320, an inverse transform unit 330, a prediction unit 340, and an adder 350, and a filter unit 360. and memory 370. Like the video encoding device of FIG. 2, each component of the video decoding device may be implemented as a hardware chip, or may be implemented as software and a microprocessor may be implemented to execute the software function corresponding to each component.

The decoder 310 decodes the bitstream received from the video encoding device, extracts information related to block division, determines the current block to be decoded, and provides prediction information and residual signal information necessary to restore the current block. Extract .

The decoder 310 extracts information about the CTU size from the SPS (Sequence Parameter Set) or PPS (Picture Parameter Set), determines the size of the CTU, and divides the picture into CTUs of the determined size. Then, the CTU is determined as the highest layer of the tree structure, that is, the root node, and the CTU is divided using the tree structure by extracting the division information about the CTU.

Additionally, when the decoding unit 310 determines the current block to be decoded by dividing the tree structure, it extracts information about the prediction type indicating whether the current block is intra-predicted or inter-predicted.

When prediction type information indicates intra prediction, the decoder 310 extracts syntax elements for intra prediction information (intra prediction mode) of the current block.

When the prediction type information indicates inter prediction, the decoder 310 extracts syntax elements for the inter prediction information, that is, information indicating a motion vector and a reference picture to which the motion vector refers.

Meanwhile, the decoder 310 extracts information about the quantized transform coefficients of the current block as information about the residual signal.

The inverse quantization unit 320 inversely quantizes the quantized transform coefficients, and the inverse transformation unit 330 inversely transforms the inverse quantized transform coefficients from the frequency domain to the spatial domain to restore the residual signals, thereby generating a residual block for the current block.

The prediction unit 340 includes an intra prediction unit 342 and an inter prediction unit 344. The intra prediction unit 342 is activated when the prediction type of the current block is intra prediction, and the inter prediction unit 344 is activated when the prediction type of the current block is intra prediction.

The intra prediction unit 342 determines the intra prediction mode of the current block among a plurality of intra prediction modes from the syntax elements for the intra prediction mode extracted from the decoder 310, and determines the intra prediction mode of the current block according to the intra prediction mode. Use these to predict the current block.

The inter prediction unit 344 determines the motion vector of the current block and the reference picture referenced by the motion vector using the syntax elements for the intra prediction mode extracted from the decoder 310, and uses the motion vector and the reference picture to determine the current block. Predict blocks.

The adder 350 restores the current block by adding the residual block output from the inverse transform unit and the prediction block output from the inter prediction unit or intra prediction unit. Pixels in the restored current block are used as reference pixels when intra-predicting a block to be decoded later.

By sequentially restoring current blocks corresponding to CUs by the image restorer 300, a CTU made up of CUs and a picture made up of CTUs are restored.

The filter unit 360 includes a deblocking filter 362 and a SAO filter 364. The deblocking filter 362 deblocks and filters the boundaries between restored blocks to remove blocking artifacts caused by block-level decoding. The SAO filter 364 performs additional filtering on the reconstructed block after deblocking filtering to compensate for the difference between the reconstructed pixel and the original pixel caused by lossy coding. The restored block filtered through the deblocking filter 362 and the SAO filter 364 is stored in the memory 370. When all blocks in one picture are reconstructed, the reconstructed picture is later used as a reference picture for inter prediction of blocks in the picture to be encoded.

In this disclosure, a CNN-based filter with the functions of deblocking filters 182 and 362 and SAO filters 284 and 464 is described in detail. The CNN-based filter according to the present disclosure can be used in both an image encoding device and an image decoding device.

Additionally, in this disclosure, YUV is used as an example as information constituting a picture, but it can be applied to RGB, YCbCr, etc. That is, YUV, which will improve image quality, can be RGB, which will improve image quality, or YCbCr, which will improve image quality.

Figure 4 is a diagram showing a CNN-based filter according to an embodiment of the present invention.

When at least one of the quantization parameter (QP) map 403 and the block partition map 405 and YUV 401 to improve image quality are input to the input layer, the YUV difference (401) is input to the output layer. 421) is output. Here, the YUV 401 to improve picture quality may be a YUV 401 restored from a bit stream received from an encoder, and refers to a YUV in which the original YUV has been artificially or non-artificially damaged. Additionally, a hint (not shown) may also be input to the input layer.

First, in the learning process, the coefficients of the CNN-based filter, that is, the coefficients of the convolution kernel, are learned so that the YUV difference 421 output to the output layer is the difference between the original YUV and the YUV that will improve image quality. Here, the convolution kernel can be in both 2D (dimension) and 3D forms. The CNN-based filter 411 is intended to improve the image quality of the input YUV, and the final output of the CNN-based filter 411 is called YUV, that is, YUV 431 with improved image quality.

Here, YUV to improve picture quality can be filtered for each channel or all at once.

The size of the QP map is set to the same resolution as the input YUV to be filtered, and the value of the QP map can be filled with the QP value used in a coding unit within the YUV plane, for example, a block or subblock. At this time, if YUV is filtered for each channel, one map can be constructed with the QP value of the channel to be filtered. If YUV is filtered at once, the QP values of the three channels can be composed of three separate maps or one map with the average QP value.

As a way to increase the accuracy of the CNN technique, in addition to the QP map, block division map, and image to be improved quality as input layers, block mode map information useful in the learning process can be added as hint information. Here, the block mode map may be filled with mode values used in coding units, for example, blocks or subblocks. For example, it may be information that can distinguish whether a block is encoded in intra mode or inter mode, and the information may be expressed in numbers. At this time, the convolution kernel coefficient, which is the result of the learning process, can be set to include not only the data of the input layer but also a hint. Basically, the input layer and output layer of the CNN technique's learning process and inference process must be configured identically.

Afterwards, in the inference process, the coefficients of the CNN-based filter obtained in the learning process are applied to generate a YUV with improved image quality from the YUV, the quantization parameter map, and the block partition map.

Figures 5a to 5c are diagrams showing the structure of a CNN according to the location of the concatenated layer according to an embodiment of the present invention.

Specifically, YUV 501, quantization parameter map 503, and block division map 505, which will improve the image quality input to the input layer 510, will be concatenated through the concatenate layer 520 during the CNN process. You can. However, the position of the contiguous layer 520 may be changed.

Figure 5a shows that the concatenated layer 520 is located immediately behind the input layer 510, and YUV 501, quantization parameter map 503, and block division map 505, which will improve image quality, are input as the input layer 510. It shows the structure of a CNN that is immediately concatenated.

Figure 5b shows the structure of a CNN in which the concatenation layer 520 is located between the convolution layers 530, and Figure 5c shows the structure of a CNN in which the concatenation layer 520 is located right in front of the output layer 540.

Figures 6A to 6C are diagrams showing data to be input to the input layer according to an embodiment of the present invention.

Specifically, Figure 6a shows that the pixel value of luminance (luma) that will improve picture quality as a Y plane (e.g., Y coding tree block, CTB) that will improve picture quality is omitted, and Figure 6b shows QP map applied to the Y plane to improve image quality, Figure 6c shows a block division map of the Y plane to improve image quality.

Hereinafter, various structures of a block partition map according to an embodiment of the present invention will be described. The block division map indicates whether the block is divided, and helps to process the divided boundaries of the block and the internal area of the block differently during the CNN learning process and inference process.

7A and 7B are diagrams showing an example of a block division map according to an embodiment of the present invention.

The block division map is set to the same resolution as the YUV plane to be filtered, and can be composed of a value indicating whether the block is divided. For example, when the YUV plane is composed of a coding tree block including a plurality of coding blocks, that is, coding blocks (CB), the block partition map may indicate the division boundary line of the coding block within the coding tree block. FIG. 7A shows a coding tree block divided by a quadtree plus binary tree (QTBT) method, and FIG. 7B shows a block division map according to the coding tree block. Referring to FIG. 7B, in the block division map, the boundary of the coding block is indicated as '1' and the inside of the coding block is indicated as '0'.

Figures 8a and 8b are diagrams showing another example of a block division map according to an embodiment of the present invention.

In FIGS. 8A and 8B, when the YUV plane is composed of a coding tree block, blocking deterioration processing on the boundary of the coding tree block may not be possible, so an extra area (α) may be added to the YUV plane. Figures 8a and 8b show an example in which 2 pixels are set as the extra area (α), but other values may be set. Additionally, the boundary of the coding tree block can be indicated by the extra area, and whether or not the block is divided can be indicated for the area outside the coding tree block. If filtering includes an extra area, an area overlapping with another adjacent coding tree block is created after the filtering, and the overlapping area can be processed as an average value. Specifically, referring to FIG. 8C, when coding tree blocks 801 and 803 including redundant areas are adjacent, an overlapping area 805 is created. The overlapping area 805 may be set to the average value of the values of adjacent coding tree blocks 801 and 803.

FIGS. 9A to 9C are diagrams showing block division maps for adjusting the strength of deblocking according to an embodiment of the present invention.

In the previous embodiment, the boundary of the coding block was divided into 1 pixel. If the value of 1 pixel was 0, it indicated the inside of the coding block, and if the value of 1 pixel was 1, it indicated the border of the coding block.

In FIG. 9A, the boundary of the coding block is indicated by the number of pixels (or pixel width, luma sample line, luma sample length, etc.) to adjust the strength of de-blocking. The number of pixels may be determined by at least one of the size of the coding block, the value of the quantization parameter, and the encoding mode. For example, as shown in FIG. 9A, if the coding block is large, the number of pixels can be set to 2, and if the coding block is small, the number of pixels can be set to 1. Additionally, if the quantization parameter value is large, the number of pixels can be set to be large, and if the quantization parameter value is small, the number of pixels can be set to be small. As another example, if the encoding mode is intra, the number of pixels can be set to be large, and if the encoding mode is inter, the number of pixels can be set to be small. These can all be set in reverse.

The number of pixels may refer to the number of pixels located at the block border to be updated through filtering. For example, when you want to update the value of 3 pixels in a block located at the block boundary, the boundary of the block can be displayed with 3 pixels in the block division map. As another example, the number of pixels may refer to the number of pixels located at the block border to be referenced for filtering. For example, when filtering is performed by referring to the 4 pixel values within a block located at the block boundary, the boundary of the block can be displayed with 4 pixels in the block division map.

In Figure 9b, the boundary values of coding blocks are displayed differently to adjust the strength of deblocking. The boundary value of the coding block may be determined by at least one of the size of the coding block, the value of the quantization parameter, the coding mode, the number of pixels to be updated, and the number of pixels to be referenced for filtering. As in FIG. 7A, if the coding block is large, the value of the quantization parameter is large, or the encoding mode is intra, the boundary value of the coding block can be set large. Conversely, if the coding block is small or the value of the quantization parameter is small, the boundary value of the coding block can be set large. If the coding mode is inter, the boundary value of the coding block can be set small. All of these can also be set in reverse.

Figure 9c shows the number of pixels at the boundary of a coding block and the boundary value of the coding block to adjust the strength of deblocking. The explanation for this is the same as that described in FIGS. 9A and 9B, so it is omitted here.

As described above, the set block division map is used in the learning process to help the CNN filter operate as a strong deblocking filter.

Figure 10 is a diagram showing a flowchart of decoding an image using a CNN-based filter according to the present disclosure.

At least one of a quantization parameter map and a block division map and YUV to improve image quality are input to the CNN-based filter (1001). The quantization parameter map may be set to the same resolution as YUV, which will improve the image quality. The block division map may display different values for the divided boundaries of the block and the internal area of the block. The number and value of pixels representing the divided boundaries of blocks in the block partition map are determined by at least one of the size of the coding block, the value of the quantization parameter, the encoding mode, the number of pixels to be updated, and the number of pixels to be referenced for filtering. can be decided.

YUV with improved picture quality is output using the coefficients of the CNN-based filter learned using YUV, a quantization parameter map, and a block division map to improve picture quality as input and the original YUV as the final output (1003). When a hint such as a block mode map is additionally input to the CNN-based filter, the coefficients of the CNN-based filter are also learned by additionally inputting a hint.

The processes shown in FIG. 10 can be implemented as computer-readable codes on a computer-readable recording medium. Computer-readable recording media include all types of recording devices that store data that can be read by a computer system. In other words, computer-readable recording media include magnetic storage media (e.g. ROM, floppy disk, hard disk, etc.), optical read media (e.g. CD-ROM, DVD, etc.), and carrier waves (e.g. Internet It includes storage media such as transmission through . Additionally, computer-readable recording media can be distributed across networked computer systems so that computer-readable code can be stored and executed in a distributed manner.

Figure 11 is a diagram schematically showing the configuration of an apparatus for decoding an image according to the present disclosure.

The device for decoding the image may include an input unit 1101, a filter unit 1103, and an output unit 1105. Other configurations may be included, but descriptions of configurations not directly related to the present disclosure will be omitted.

The input unit 1101 receives at least one of a quantization parameter map and a block division map and YUV to improve image quality. The quantization parameter map may be set to the same resolution as YUV to improve the image quality, and the block division map may display different values of the divided boundaries of the block and the internal area of the block. The number and value of pixels representing the divided boundaries of blocks in the block division map may be determined by at least one of the size of the coding block, the value of the quantization parameter, and the encoding mode.

The filter unit 1103 applies at least one of the quantization parameter map and the block division map input to the input unit 1101 and the learned coefficients of the CNN-based filter to YUV that will improve image quality.

The output unit 1105 outputs a YUV with improved image quality generated by applying at least one of the input quantization parameter map and the block division map and the coefficients of the CNN-based filter learned to the YUV that will improve image quality.

In the present disclosure, the input unit 1101, the filter unit 1103, and the output unit 1105 are described separately, but they may be integrated and implemented as one configuration, or one configuration may be implemented by dividing them into several configurations.

The above description is merely an illustrative explanation of the technical idea of the present embodiment, and those skilled in the art will be able to make various modifications and variations without departing from the essential characteristics of the present embodiment. Accordingly, the present embodiments are not intended to limit the technical idea of the present embodiment, but rather to explain it, and the scope of the technical idea of the present embodiment is not limited by these examples. The scope of protection of this embodiment should be interpreted in accordance with the claims below, and all technical ideas within the equivalent scope should be interpreted as being included in the scope of rights of this embodiment.

Claims (16)

In an image decoding method using a CNN (Convolutional Neural Network)-based filter,
Inputting input data including a quantization parameter map, a block division map, and a first picture into the CNN-based filter; and
Generating a second picture by adding output data of the CNN-based filter to the first picture,
The quantization parameter map and the block division map are set to the same resolution as the first picture,
In the block division map, sample positions located in internal areas of blocks constituting the first picture have a first value, and sample positions adjacent to the boundaries of the blocks have values different from the first value,
The values of sample positions adjacent to the boundaries of the blocks are the deblocking strength determined by at least one of the size of the block, the value of the quantization parameter, the encoding mode, the number of samples to be updated by filtering, and the number of samples to be referenced for filtering. A video decoding method characterized by representing. According to paragraph 1,
A video decoding method, characterized in that the coefficients of the CNN-based filter are learned using training data including a quantization parameter map, a block division map, a third picture, and an original picture corresponding to the third picture. delete According to paragraph 1,
The input data further includes a block mode map indicating an encoding mode with the CNN-based filter. delete According to paragraph 1,
In the block division map, the range of sample positions adjacent to the boundaries of the blocks depends on at least one of the size of the coding block, the value of the quantization parameter, the encoding mode, the number of pixels to be updated by filtering, and the number of samples to be referenced for filtering. A video decoding method characterized by being determined by. delete According to paragraph 1,
An image decoding method, characterized in that the coefficients of the CNN-based filter are received from a device that encodes an image. In an image decoding device using a CNN (Convolutional Neural Network)-based filter,
An input unit that receives input data including a quantization parameter map, a block division map, and a first picture;
a filter unit providing the quantization parameter map, the block division map, and the first picture input to the input unit to the CNN-based filter; and
An output unit that adds output data of the CNN-based filter to the first picture and outputs a second picture,
The quantization parameter map and the block division map are set to the same resolution as the first picture,
In the block division map, sample positions located in internal areas of blocks constituting the first picture have a first value, and sample positions adjacent to the boundaries of the blocks have values different from the first value,
The values of sample positions adjacent to the boundaries of the blocks are the deblocking strength determined by at least one of the size of the block, the value of the quantization parameter, the encoding mode, the number of samples to be updated by filtering, and the number of samples to be referenced for filtering. A video decoding device characterized in that it represents. According to clause 9,
A video decoding device, characterized in that the coefficients of the CNN-based filter are learned using training data including a quantization parameter map, a block division map, a third picture, and an original picture corresponding to the third picture. delete According to clause 9,
The input data further includes a block mode map indicating an encoding mode. delete According to clause 9,
In the block division map, the range of sample positions adjacent to the boundaries of the blocks depends on at least one of the size of the coding block, the value of the quantization parameter, the encoding mode, the number of pixels to be updated by filtering, and the number of samples to be referenced for filtering. A video decoding device characterized in that it is determined by. delete According to clause 9,
A video decoding device, characterized in that the coefficients of the CNN-based filter are received from a video encoding device.

Images