CN112738523A

CN112738523A - Image coding and decoding method and device

Info

Publication number: CN112738523A
Application number: CN202011591252.8A
Authority: CN
Inventors: 张路
Original assignee: Xian Wanxiang Electronics Technology Co Ltd
Current assignee: Xian Wanxiang Electronics Technology Co Ltd
Priority date: 2020-12-29
Filing date: 2020-12-29
Publication date: 2021-04-30

Abstract

The present disclosure provides an image encoding and decoding method and apparatus, relates to the technical field of image processing, and can solve the problem of poor user experience caused by the prior art. The specific technical scheme is as follows: carrying out macro block division on a current frame image to generate N macro blocks; determining a color gradient macro block from the N macro blocks and determining a color gradient area according to the color gradient macro block; coding the color gradient area according to a first coding mode to generate a code stream of the color gradient area; coding the character macro blocks except the color gradient area according to a second coding mode and coding the picture macro blocks except the color gradient area according to a third coding mode to respectively generate a code stream of the character macro blocks and a code stream of the picture macro blocks; and generating a code stream of the current frame image according to the code stream of the color gradient area, the code stream of the character macro block and the code stream of the picture macro block, and sending the code stream of the current frame image to a decoding end. The invention is used for improving the user experience.

Description

Image coding and decoding method and device

Technical Field

The present disclosure relates to the field of image processing, and in particular, to an image encoding and decoding method and apparatus.

Background

At present, the coding of an image generally classifies macro blocks of the image into text macro blocks and picture macro blocks, and different types of macro blocks are coded by different coding modes. However, if different types of macro blocks are encoded by different encoding methods, if the types of the macro blocks of two adjacent portions in a color gradient region are different, the encoding methods are different, the boundaries of the macro blocks of the two adjacent portions will have visual defects after decoding and reconstruction, and the macro blocks of the two adjacent portions which should be displayed in a gradient manner are presented as boundaries with a mosaic pattern, so that a user can perceive the discontinuity between the colors of the macro blocks of the two adjacent portions, and the user experience is poor.

For example, as shown in fig. 1, fig. 1 is a color gradient area in an image to be coded, wherein the dotted line indicates the pixel values of 4 macro blocks of the color gradient area, and each macro block has a size of 16 × 16 pixels. If the prior art is adopted to encode the image to be encoded, the image is shown in fig. 2 after being decoded at the decoding end. The dotted line in fig. 2 indicates the color jump, and the macroblock in the upper left corner (labeled as No. 1) of the color is different from the other three macroblocks, and there is a color jump. As shown in fig. 2, there is a change band of a color staircase shape in the lower left and upper right blocks of the color gradient area, i.e. there is a color jump in the color gradient area after decoding. The color step-like change band should be displayed as a gradient color. The pixels of the 4 macro blocks in the area after decoding are the portions marked by the dotted lines, such as the oval enclosed portions, and the reason why the color jump exists between the macro block 1 and the

macro blocks

2 and 3 is that although the macro block 1 and the

macro blocks

2 and 3 are adjacent, the pixel value has obvious jump.

Disclosure of Invention

The embodiment of the disclosure provides an image coding and decoding method and device, which can solve the problems that in the prior art, macro blocks of two adjacent parts which should be gradually displayed appear to have a mosaic-type boundary after being decoded, so that a user can perceive discontinuity between macro block colors of the two adjacent parts, and user experience is poor. The technical scheme is as follows:

according to a first aspect of embodiments of the present disclosure, there is provided an image encoding method, the method including:

carrying out macro block division on a current frame image to generate N macro blocks, wherein N is an integer greater than 1;

determining a color gradient macro block from the N macro blocks and determining a color gradient area according to the color gradient macro block, wherein the color gradient area comprises at least 2 adjacent color gradient macro blocks;

coding the color gradient area according to a first coding mode to generate a code stream of the color gradient area;

coding the character macro blocks except the color gradient area according to a second coding mode and coding the picture macro blocks except the color gradient area according to a third coding mode to respectively generate a code stream of the character macro blocks and a code stream of the picture macro blocks;

and generating the code stream of the current frame image according to the code stream of the color gradually changing area, the code stream of the character macro block and the code stream of the picture macro block and sending the code stream of the current frame image to a decoding end.

The image coding method provided by the embodiment of the disclosure can divide a current frame image into macro blocks to generate N macro blocks; determining a color gradient macro block from the N macro blocks and determining a color gradient area according to the color gradient macro block, wherein the color gradient area comprises at least 2 adjacent color gradient macro blocks; coding the color gradient area according to a first coding mode to generate a code stream of the color gradient area; coding the character macro blocks except the color gradient area according to a second coding mode and coding the picture macro blocks except the color gradient area according to a third coding mode to respectively generate a code stream of the character macro blocks and a code stream of the picture macro blocks; the method comprises the steps of coding character macro blocks except the color gradient area according to a second coding mode, coding picture macro blocks except the color gradient area according to a third coding mode, generating code streams of the character macro blocks and the picture macro blocks respectively, coding the macro blocks in the color gradient area by adopting a first coding mode no matter which type the macro blocks are, and being capable of avoiding the problem that the macro blocks of two adjacent parts are presented with mosaic type boundaries after decoding and reconstruction if the types of the macro blocks of the two adjacent parts in one color gradient area are different, so that a user can perceive the discontinuity between the colors of the macro blocks of the two adjacent parts, and the user experience is poor.

In one embodiment, determining a color gradient macroblock from the N macroblocks comprises:

calculating a pixel difference value between a target macro block and a first macro block, wherein the first macro block is a macro block adjacent to the left side or the right side of the target macro block, and the target macro block is any one of the N macro blocks;

if the pixel difference value of the target macro block and the first macro block is larger than 0 and smaller than a preset threshold value, calculating the pixel difference value of the target macro block and a second macro block, wherein the second macro block is a macro block adjacent to the upper side or the lower side of the target macro block;

and if the pixel difference value of the target macro block and the second macro block is greater than 0 and smaller than a preset threshold value, the target macro block is a color gradient macro block.

In one embodiment, the calculating the pixel difference value of the target macroblock and the first macroblock includes:

and taking the difference value of the last column of pixels of the target macro block and the first column of pixels of the first macro block as the pixel difference value of the target macro block and the first macro block or taking the difference value of the last column of pixels of the first macro block and the first column of pixels of the target macro block as the pixel difference value of the target macro block and the first macro block.

In one embodiment, said determining a color gradient area from said color gradient macroblock comprises:

the method comprises the following steps of taking a rectangular macro block area formed by at least 2 adjacent color gradient macro blocks as a color gradient area and/or taking a rectangular macro block area formed by at least 2 adjacent color gradient macro blocks and a row and/or a column of non-color gradient macro blocks as a color gradient area.

According to a second aspect of the embodiments of the present disclosure, there is provided an image decoding method, including:

receiving a code stream of a current frame image sent by a coding end and respectively determining a code stream of a character macro block, a code stream of a picture macro block and a code stream of a color gradient area from the code stream of the current frame image;

decoding the code stream of the color gradient area according to a first decoding mode to generate a color gradient area, wherein the color gradient area comprises at least 2 macro blocks;

determining the boundary of every two adjacent macro blocks in the color gradient area and filtering the boundary to generate a filtered color gradient area;

decoding the code stream of the character macro block according to a second decoding mode and decoding the code stream of the picture macro block according to a third decoding mode to respectively generate the character macro block and the picture macro block;

and generating the current frame image according to the filtered color gradient area, the character macro block and the picture macro block.

The image decoding method provided by the embodiment of the disclosure can receive the code stream of the current frame image sent by the encoding end and respectively determine the code stream of the text macro block, the code stream of the picture macro block and the code stream of the color gradient area from the code stream of the current frame image; decoding the code stream of the color gradient area according to a first decoding mode to generate a color gradient area, wherein the color gradient area comprises at least 2 macro blocks; determining the boundary of every two adjacent macro blocks in the color gradient area and filtering the boundary to generate a filtered color gradient area; decoding the code stream of the character macro block according to a second decoding mode and decoding the code stream of the picture macro block according to a third decoding mode to respectively generate a character macro block and a picture macro block; the current frame image is generated according to the filtered color gradient area, the character macro block and the picture macro block, the boundary of every two adjacent macro blocks in the color gradient area can be determined and filtered, so that the color jump of the boundary of the adjacent macro blocks in the color gradient area is eliminated, a user can perceive the color continuity of the boundary of the adjacent macro blocks in the color gradient area, and the user experience is improved.

In one embodiment, the determining the boundary of every two adjacent macroblocks in the color gradient region comprises:

and taking adjacent M rows or adjacent M columns of pixels of every two adjacent macro blocks in the color gradient area as the boundary of the two adjacent macro blocks, wherein M is an integer larger than 1.

In one embodiment, the filtering the boundary includes:

if the two adjacent macro blocks are vertically adjacent, performing difference operation on a first row of pixels and a last row of pixels of M rows of adjacent pixels of the two adjacent macro blocks to obtain a first pixel difference value;

obtaining a pixel gradient value according to the first pixel difference value and M, wherein the pixel gradient value is a value obtained by dividing the first pixel difference value by M and then rounding;

and for each row of pixels in the M-1 rows of pixels except the first row of pixels, determining the pixel value of the row of pixels according to the pixel value of the previous row of pixels adjacent to the row of pixels and the pixel gradient value.

In one embodiment, the filtering the boundary includes:

if the two adjacent macro blocks are adjacent left and right, performing difference operation on a first row of pixels and a last row of pixels of M rows of adjacent pixels of the two adjacent macro blocks to obtain a second pixel difference value;

obtaining a pixel gradient value according to the second pixel difference value and M, wherein the pixel gradient value is a value obtained by dividing the second pixel difference value by M and then rounding;

and for each column of pixels in M-1 columns of pixels except the first column of pixels, determining the pixel value of the column of pixels according to the pixel value of the last column of pixels adjacent to the column of pixels and the pixel gradient value.

According to a third aspect of the embodiments of the present disclosure, there is provided an image encoding device including:

the macro block dividing module is used for carrying out macro block division on the current frame image to generate N macro blocks, wherein N is an integer greater than 1;

a color gradient area determination module, configured to determine a color gradient macro block from the N macro blocks and determine a color gradient area according to the color gradient macro block, where the color gradient area includes at least 2 adjacent color gradient macro blocks;

the color gradient area coding module is used for coding the color gradient area according to a first coding mode to generate a code stream of the color gradient area;

the character macro block coding module is used for coding the character macro blocks except the color gradient area according to a second coding mode and coding the picture macro blocks except the color gradient area according to a third coding mode to respectively generate a code stream of the character macro blocks and a code stream of the picture macro blocks;

and the current frame image code stream generating module is used for generating the code stream of the current frame image according to the code stream of the color gradient area, the code stream of the character macro block and the code stream of the picture macro block and sending the code stream of the current frame image to a decoding end.

In one embodiment, the color gradient region determination module is to:

According to a fourth aspect of the embodiments of the present disclosure, there is provided an image decoding apparatus comprising:

the current frame image code stream receiving module is used for receiving the code stream of the current frame image sent by the encoding end and respectively determining the code stream of the character macro block, the code stream of the picture macro block and the code stream of the color gradient area from the code stream of the current frame image;

the code stream decoding module of the color gradient area is used for decoding the code stream of the color gradient area according to a first decoding mode to generate the color gradient area, and the color gradient area comprises at least 2 macro blocks;

the boundary filtering processing module is used for determining the boundary of every two adjacent macro blocks in the color gradient area and filtering the boundary to generate a filtered color gradient area;

the character macro block code stream decoding module is used for decoding the code stream of the character macro block according to a second decoding mode and decoding the code stream of the picture macro block according to a third decoding mode to respectively generate a character macro block and a picture macro block;

and the current frame image generating module is used for generating the current frame image according to the filtered color gradient area, the character macro block and the picture macro block.

In one embodiment, the boundary filtering processing module is configured to:

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and together with the description, serve to explain the principles of the disclosure.

Fig. 1 is a schematic diagram of a current frame image to be encoded according to an embodiment of the present disclosure;

fig. 2 is a schematic diagram of a decoded current frame image provided by the prior art;

fig. 3 is a flowchart of an image encoding method provided by an embodiment of the present disclosure;

FIG. 4A is a first schematic diagram of a color gradient region provided by an embodiment of the present disclosure;

FIG. 4B is a second schematic diagram of a color gradient region provided by an embodiment of the present disclosure;

fig. 5 is a flowchart of an image decoding method provided by an embodiment of the present disclosure;

fig. 6A is a schematic diagram of two adjacent macroblock boundaries according to an embodiment of the disclosure;

fig. 6B is a schematic pixel diagram of a decoded color gradient region according to an embodiment of the disclosure;

FIG. 6C is a schematic diagram of a pixel in a filtered color gradient region according to an embodiment of the disclosure;

fig. 7 is a schematic diagram of a decoded current frame image according to an embodiment of the present disclosure;

FIG. 8 is a component diagram of an image decoding system provided by an embodiment of the present disclosure;

fig. 9 is a schematic structural diagram of an image encoding apparatus provided in an embodiment of the present disclosure;

fig. 10 is a schematic structural diagram of an image decoding apparatus according to an embodiment of the present disclosure.

Detailed Description

Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the exemplary embodiments below are not intended to represent all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.

Fig. 3 is a flowchart of an image encoding method provided by an embodiment of the present disclosure, and the method is applied to an encoding end. As shown in fig. 3, the method includes:

s301, carrying out macro block division on the current frame image to generate N macro blocks, wherein N is an integer larger than 1.

Illustratively, a current frame image is subjected to macroblock division, and N macroblocks and identification information of each macroblock in the N macroblocks are generated, wherein the identification information is an index value of the macroblock and is used for indicating the position of the macroblock in the current frame image. Taking 1920 × 1080 size image as an example, after macroblock division is performed on the 1920 × 1080 size image, 120 × 68 macroblocks are generated, that is, 120 macroblocks are generated in each row, 68 macroblocks are generated in each column, and each macroblock has a size of 16 × 16 pixels, that is, each macroblock includes 16 pixels (0 th to 15 th) in each row, and 16 pixels (0 th to 15 th) are generated in each column. The index values of 120 x 68 macroblocks are from 0 to 8159, respectively. The position of the row and column of the macroblock can be known according to the index value of the macroblock.

S302, determining a color gradient macro block from the N macro blocks and determining a color gradient area according to the color gradient macro block, wherein the color gradient area comprises at least 2 adjacent color gradient macro blocks.

The following is a description of how to determine a color gradient macro block from among N macro blocks.

Calculating a pixel difference value of a target macro block and a first macro block, wherein the first macro block is a macro block adjacent to the left side or the right side of the target macro block, and the target macro block is any one of the N macro blocks; if the pixel difference value of the target macro block and the first macro block is larger than 0 and smaller than a preset threshold value, calculating the pixel difference value of the target macro block and a second macro block, wherein the second macro block is a macro block adjacent to the upper side or the lower side of the target macro block; and if the pixel difference value of the target macro block and the second macro block is greater than 0 and smaller than a preset threshold value, the target macro block is a color gradient macro block.

For example, the difference between the last row of pixels of the target macroblock and the first row of pixels of the first macroblock may be used as the pixel difference between the target macroblock and the first macroblock or the difference between the last row of pixels of the first macroblock and the first row of pixels of the target macroblock may be used as the pixel difference between the target macroblock and the first macroblock.

Specifically, if the first macroblock is a macroblock adjacent to the right of the target macroblock, the difference between the last row of pixels of the target macroblock and the first row of pixels of the first macroblock is used as the pixel difference between the target macroblock and the first macroblock. If the first macro block is a macro block adjacent to the left side of the target macro block, the difference value between the last row of pixels of the first macro block and the first row of pixels of the target macro block is used as the pixel difference value between the target macro block and the first macro block.

Here, the description will be given taking an example in which the size of the macroblock is 16 × 16 pixels. If the first macro block is a macro block adjacent to the right of the target macro block, that is, the first macro block is on the right of the target macro block and adjacent to the target macro block, the difference between the 15 th row of pixels of the target macro block and the 0 th row of pixels of the first macro block is used as the pixel difference between the target macro block and the first macro block. Since each row of the macroblock has 16 rows, that is, each row has 16 pixels, the difference between the 0 th row pixel at the 15 th column of the target macroblock and the 0 th row pixel at the 0 th column of the first macroblock, the difference between the 1 st row pixel at the 15 th column of the target macroblock and the 1 st row pixel at the 0 th column of the first macroblock, the difference between the 2 nd row pixel at the 15 th column of the target macroblock and the 2 nd row pixel at the 0 th column of the first macroblock … … the difference between the 15 th row pixel at the 15 th column of the target macroblock and the 15 th row pixel at the 0 th column of the first macroblock is all taken as the pixel difference between the target macroblock and the first macroblock, and the difference between the 0 th row pixel at the 15 th column of the target macroblock and the 0 th column of the 0 th row pixel of the first macroblock, the difference between the 1 st row pixel at the 15 th column of the target macroblock and the 1 st row pixel at the 0 th column of the first macroblock, the difference between the 2 nd row pixel at the 15 th column of the 2 th row pixel of the target macroblock and the 2 nd row pixel at the 0 th column of the 0 nd row pixel of the first macroblock … … And when the difference values are both larger than 0 and smaller than the preset threshold value, the pixel difference value between the target macro block and the first macro block is larger than 0 and smaller than the preset threshold value.

Similarly, if the first macroblock is a macroblock adjacent to the left of the target macroblock, i.e. the first macroblock is on the left side of the target macroblock and adjacent to the target macroblock, the difference between the 15 th row of pixels of the first macroblock and the 0 th row of pixels of the target macroblock is used as the pixel difference between the target macroblock and the first macroblock. Since each column has 16 rows, that is, each column has 16 pixels, the difference between the 0 th row pixel at the 15 th column of the first macroblock and the 0 th row pixel at the 0 th column of the target macroblock, the difference between the 1 st row pixel at the 15 th column of the first macroblock and the 1 st row pixel at the 0 th column of the target macroblock, the difference … … between the 2 nd row pixel at the 15 th column of the first macroblock and the 2 nd row pixel at the 0 th column of the target macroblock, the difference between the 15 th row pixel at the 15 th column of the first macroblock and the 15 th row pixel at the 0 th column of the target macroblock is all regarded as the pixel difference between the target macroblock and the first macroblock, and the difference between the 0 th row pixel at the 15 th column of the first macroblock and the 0 th row pixel at the 0 th column of the target macroblock, the difference between the 1 st row pixel at the 15 th column of the first macroblock and the 1 st row pixel at the 0 th column of the target macroblock, and the difference between the 15 th row pixel at the 15 th column of the 2 nd row pixel at the 15 th column of the target macroblock and the 2 nd row pixel at the 0 th column of the target macroblock are all larger than the 0 th row pixel And when the difference value is smaller than the preset threshold value, the pixel difference value of the target macro block and the first macro block is larger than 0 and smaller than the preset threshold value.

How to determine the pixel difference value of the target macro block and the second macro block is described below. For example, the difference value between the last row of pixels of the target macroblock and the first row of pixels of the second macroblock may be used as the pixel difference value between the target macroblock and the second macroblock or the difference value between the last row of pixels of the second macroblock and the first row of pixels of the target macroblock may be used as the pixel difference value between the target macroblock and the second macroblock.

The specific calculation method of the pixel difference between the target macroblock and the second macroblock is similar to the calculation method of the pixel difference between the target macroblock and the first macroblock, and details are not repeated here.

The following includes for determining a color gradient area from the color gradient macro block:

exemplarily, a rectangular macroblock area consisting of at least 2 adjacent color gradient macroblocks is taken as a color gradient area and/or a rectangular macroblock area consisting of at least 2 adjacent color gradient macroblocks and a row and/or a column of non-color gradient macroblocks is taken as a color gradient area.

For example, as shown in fig. 4A and 4B, if the macro block 1/2/3 in fig. 4A is a color gradient macro block, or the macro block 1/2/3/4 is a color gradient macro block, a rectangle formed by the four macro blocks is a color gradient area.

In fig. 4B, the portions marked by the dotted lines are color gradation macroblocks, and the areas marked by X are non-color gradation macroblocks. The rectangular area of the macroblock consisting of the color gradient macroblock marked by the dotted line and the non-color gradient macroblock marked by X is the color gradient area. It should be noted that, in order to consider the accuracy of each macroblock pixel of the image during decoding and reconstruction at the decoding end, when the color gradient area is defined, at most, only one row and/or one column of non-color gradient macroblocks in the color gradient area is allowed to be included.

And S303, coding the color gradient area according to a first coding mode to generate a code stream of the color gradient area.

Illustratively, the color gradient area is encoded according to a huffman coding method, and a code stream of the color gradient area is generated. Illustratively, the color gradient area is encoded while the identification information of the color gradient area is encoded, and the identification information of the color gradient area is encoded by the index values of the first macro block (i.e. the macro block at the upper left corner) and the last macro block (i.e. the macro block at the lower right corner) in the color gradient area.

S304, coding the character macro blocks except the color gradient area according to a second coding mode, coding the picture macro blocks except the color gradient area according to a third coding mode, and respectively generating a code stream of the character macro blocks and a code stream of the picture macro blocks.

In this step, each of the other macro blocks except the color gradient area is identified and classified according to its content, and divided into a text macro block and a picture macro block, where the text macro block is a macro block with more details, and the picture macro block is a macro block with more gradual changes. The character macro blocks need to be displayed clearly, so the character macro blocks are coded by adopting a coding mode of palette quantization and huffman coding to generate code streams of the character macro blocks. The picture macro block is coded by Joint Photographic Experts Group (JPEG) coding mode information to generate a code stream of the picture macro block. Illustratively, the text macroblock and the picture macroblock are encoded separately, and at the same time, the index value of the text macroblock and the index value of the picture macroblock are encoded separately.

S305, generating the code stream of the current frame image according to the code stream of the color gradient area, the code stream of the character macro block and the code stream of the picture macro block, and sending the code stream of the current frame image to a decoding end.

In this step, the code stream of the color gradient area, the code stream of the text macro block and the code stream of the picture macro block are packed, the code stream of the current frame image is generated, and the code stream of the current frame image is sent to a decoding end. Illustratively, while the code stream of the color gradient area, the code stream of the text macro block and the code stream of the picture macro block are packed, the identification information of the color gradient area, the index value of the text macro block and the index value of the picture macro block are packed and sent to a decoding end.

How to decode the code stream of the current frame image to generate the current frame image is described below. Fig. 5 is a flowchart of an image decoding method provided by an embodiment of the present disclosure, and the method is applied to a decoding end. As shown in fig. 5, the method includes:

s501, receiving a code stream of a current frame image sent by a coding end and respectively determining a code stream of a character macro block, a code stream of a picture macro block and a code stream of a color gradient area from the code stream of the current frame image.

Illustratively, the code stream of the current frame image includes a code stream of a text macro block, a code stream of a picture macro block, a code stream of a color gradient area, an index value of the text macro block, an index value of the picture macro block, and identification information of the color gradient area. And after receiving the code stream of the current frame image sent by the encoding end, the decoding end respectively determines the code stream of the character macro block, the code stream of the picture macro block and the code stream of the color gradient area from the code stream of the current frame image.

S502, decoding the code stream of the color gradient area according to a first decoding mode to generate the color gradient area, wherein the color gradient area comprises at least 2 macro blocks.

Exemplarily, the code stream of the color gradient area is decoded according to a decoding mode corresponding to the huffman coding mode to generate the color gradient area, the color gradient area includes at least 2 macro blocks, the color gradient area is a macro block rectangular area formed by the at least 2 macro blocks, and the at least 2 macro blocks are color gradient macro blocks determined by a coding end.

S503, determining the boundary of every two adjacent macro blocks in the color gradient area, and performing filtering processing on the boundary to generate a filtered color gradient area.

A description will be given below of how to determine every two adjacent macroblocks in the color gradation area.

Illustratively, the adjacent M rows or M columns of pixels of every two adjacent macro blocks in the color gradient area are taken as the boundary of the two adjacent macro blocks, and M is an integer greater than 1.

Specifically, if the two adjacent macroblocks are vertically adjacent, the M adjacent rows of pixels of the two adjacent macroblocks are the boundaries of the two adjacent macroblocks. If the two adjacent macroblocks are left-right adjacent, the adjacent M rows of pixels of the two adjacent macroblocks are the boundaries of the two adjacent macroblocks.

Here, the description will be given taking an example in which the size of the macroblock is 16 × 16 pixels. The macroblock size is 16 × 16 pixels, that is, each macroblock includes 16 pixels (0 th to 15 th) in each row and 16 pixels (0 th to 15 th) in each column, then the boundaries of two adjacent macroblocks which are adjacent up and down are the adjacent 6 rows of pixels of the two adjacent macroblocks, that is, the pixels in the 13 th to 15 th rows of the upper macroblock and the pixels in the 0 th to 2 nd rows of the lower macroblock, and the boundaries of two adjacent macroblocks which are adjacent left and right are the adjacent 6 rows of pixels of the two adjacent macroblocks, that is, the pixels in the 13 th to 15 th rows of the left macroblock and the pixels in the 0 th to 2 nd rows of the right macroblock.

How the boundary is subjected to the filtering process will be described below.

In one embodiment, if the two adjacent macro blocks are adjacent up and down, performing difference operation on a first row of pixels and a last row of pixels of M rows of pixels adjacent to the two adjacent macro blocks to obtain a first pixel difference value; obtaining a pixel gradient value according to the first pixel difference value and M, wherein the pixel gradient value is a value obtained by dividing the first pixel difference value by M and then rounding; and for each row of pixels in the M-1 rows of pixels except the first row of pixels, determining the pixel value of the row of pixels according to the pixel value of the previous row of pixels adjacent to the row of pixels and the pixel gradient value.

Here, the description will be given taking an example in which the size of the macroblock is 16 × 16 pixels. If the two adjacent macro blocks are adjacent up and down, performing difference operation on the first row of pixels and the last row of pixels of the 6 adjacent rows of pixels of the two adjacent macro blocks to obtain a first pixel difference value, namely performing difference operation on the 13 th row of pixels of the left macro block and the 2 nd row of pixels of the right macro block to obtain a first pixel difference value. Because each row of the macro block has 16 rows, that is, each row has 16 pixels, the pixels in the two rows of each row are respectively subjected to difference operation to obtain a first pixel difference value of the pixels in the two rows of each row. And dividing the first pixel difference value of the two rows of pixels in each row by 6 to obtain the pixel gradient value of the two rows of pixels in each row. The first row of pixels of each column is kept constant in the boundary of two adjacent macroblocks, the second row of pixels of each column has a pixel value that is the pixel value of the first row of pixels of each column minus a pixel tapering value, and the third row of pixels of each column has a pixel value that is the pixel value of the second row of pixels of each column minus a pixel tapering value … …, and the sixth row of pixels of each column has a pixel value that is equal to the pixel value of the fifth row of pixels of each column minus a pixel tapering value.

It should be noted here that, the last row of pixels in each column in the boundary between two adjacent macroblocks may also be kept unchanged, the pixel value of the fifth row of pixels in each column is the pixel value of the last row of pixels in each column plus the pixel gradient value, and the pixel value of the fourth row of pixels in each column is the pixel value of the fifth row of pixels in each column plus the pixel gradient value … …, and the pixel value of the first row of pixels in each column is equal to the pixel value of the second row of pixels in each column plus the pixel gradient value.

In another embodiment, if the two adjacent macro blocks are adjacent left and right, performing difference operation on a first row of pixels and a last row of pixels of M rows of pixels adjacent to the two adjacent macro blocks to obtain a second pixel difference value; obtaining a pixel gradient value according to the second pixel difference value and M, wherein the pixel gradient value is a value obtained by dividing the second pixel difference value by M and then rounding; and for each column of pixels in the M-1 columns of pixels except the first column of pixels, determining the pixel value of the column of pixels according to the pixel value of the last column of pixels adjacent to the column of pixels and the pixel gradient value.

The filtering processing on the boundaries of the left and right adjacent macroblocks is similar to the filtering processing on the boundaries of the upper and lower adjacent macroblocks, and is not described herein again.

This will be explained with reference to fig. 6A to 6C. As shown in fig. 6A, two adjacent macroblock boundaries are marked with dashed lines, macroblock 1 and macroblock 2 are adjacent to each other on the left and right, and the boundaries of macroblock 1 and macroblock 2 are the 13 th column of pixels of macroblock 1 to the 2 nd column of pixels of macroblock 2. That is, the 13 th row of pixels of the macroblock 1 is the first row of pixels on the boundary of the macroblock 1 and the macroblock 2, the 2 nd row of pixels of the macroblock 2 is the last row of pixels on the boundary of the macroblock 1 and the macroblock 2, the two rows of pixels in each row are respectively subjected to difference, then divided by 6 to obtain the pixel gradient value of the two rows of pixels in each row, then the 13 th row of pixels of the macroblock 1 in each row is kept unchanged, and the 5 th row of pixels except the 13 th row of the macroblock 1 takes the pixel value of the row of pixels adjacent to the row of pixels in each row minus the pixel gradient value as the pixel value of the row of pixels in each row. For example, the pixel values of the adjacent 6 pixels in the 1 st row of the two adjacent macro blocks macro block 1 and macro block 2 before filtering are:

72 71 71|65 65 65

then (65-72)/6 is calculated to be-1.17, and the integer is 1, i.e. the pixel gradient value is 1. The pixel values after filtering the boundaries of the macro block 1 and the macro block 2 are:

72 71 70|69 68 67

it can be seen that there is a transition from 71 to 65 in the first row of pixels of the original neighboring macroblock, and after the smoothing process, the boundaries of the pixels are smoothed.

In fig. 6A, the 6 boundary pixels of each row adjacent to the

macro blocks

1 and 2 are processed by the above method, i.e., the boundaries between macro blocks can be smoothed.

Fig. 6B is a schematic pixel diagram of a decoded color gradient area according to an embodiment of the disclosure. As shown in fig. 6B, the color gradient area is composed of 4 macroblocks, and after decoding, the pixel values of the 4 macroblocks of the gradient area are as shown in fig. 6B, and there is a pixel jump between boundary 1 (the boundary between macroblock 1 and macroblock 2) and boundary 2 (the boundary between macroblock 1 and macroblock 3), for example, from 71 to 65. Fig. 6C is a schematic pixel diagram of a filtered color gradient area according to an embodiment of the disclosure. After the boundary of 4 adjacent macroblocks in the color gradient area is filtered, the pixel values of the 4 macroblocks are as shown in fig. 6C, the boundary pixel jump is eliminated, and the pixels at the boundary change smoothly.

S504, decoding the code stream of the character macro block according to a second decoding mode and decoding the code stream of the picture macro block according to a third decoding mode to respectively generate the character macro block and the picture macro block.

Illustratively, decoding the code stream of the text macro block according to a decoding mode corresponding to the coding mode of the palette quantization and the huffman coding to generate the text macro block; and decoding the code stream of the picture macro block according to a JPEG coding mode to generate the picture macro block.

And S505, generating the current frame image according to the filtered color gradient area, the character macro block and the picture macro block.

Exemplarily, the filtered color gradient area, the text macro block and the picture macro block are arranged according to the identification information of the color gradient area, the index value of the text macro block and the index value of the picture macro block, respectively, to generate the current frame image.

Fig. 7 is a schematic diagram of a decoded current frame image according to an embodiment of the present disclosure. As shown in fig. 7, in the current frame image generated by using the image decoding method provided by the embodiment of the present disclosure, step-like color jump is eliminated in the image portion of fig. 7, and the image is close to the original image before encoding. As shown in fig. 7, the part enclosed by the oval part is the pixels of the boundary of the adjacent macro block of the color gradient area, the pixel jump part of the boundary of the adjacent macro block of the color gradient area is smoothed, and the color jump of the macro block boundary of the color gradient area is eliminated.

The following describes the image encoding and decoding method provided in the embodiment of the present disclosure in further detail with reference to fig. 8. Fig. 8 is a component diagram of an image decoding system provided in an embodiment of the present disclosure. As shown in fig. 8, the functions of the components are as follows:

801: and the image to be coded acquisition module. The module collects a frame of image of the computer desktop in YUV format.

802: and a macro block identification and classification module. This module logically divides a full frame picture into macroblocks of size 16 x 16 pixels. And each macroblock is identified and classified according to its content. The module mainly divides macro blocks into two types, namely character blocks and picture blocks, wherein the character blocks are macro blocks with more details, and the picture blocks are macro blocks with more gentle changes. Since the text blocks need to be displayed clearly, the methods of palette quantization and huffman coding are adopted. And the picture blocks are coded by JPEG.

803: and a gradual change area identification module. While the macroblock partition is done 802, the module will search for a gradient color macroblock within the full frame. And after 802, analyzing the macro blocks with gradually changed colors in the full frame to form a rectangular gradually-changed area. A full frame may have multiple fade areas that will be placed into the encoded codestream. The working process of the module is as follows:

1. when the 802 module performs macroblock identification and classification, each macroblock is analyzed simultaneously to determine whether it is a macroblock with gradual change. The analysis method comprises the following steps:

the 16 × 16 pixels are analyzed from left to right to determine whether the color difference between every two adjacent macroblocks is greater than 0 and less than Delta, which may be a smaller value, for example, 2, according to the specific situation of the practical application. If both of the conditions are met, whether two adjacent pixels are both larger than 0 and smaller than Delta is analyzed from top to bottom, if both of the pixels are also met, the macro block is a gradual change macro block, otherwise, the macro block is not a gradual change macro block. After the module 402 performs full frame macroblock analysis, the module 803 has recorded the locations of all the gradient macroblocks in the full frame. The position recording method may use the index value of the macroblock, taking 1920 × 1080 pictures as an example, the macroblock index value is from 0 to 8159, where there are 120 macroblocks in each row and 68 macroblocks in each column. The position of the row and the column of the macro block can be obtained according to the index value.

2. After the 802 module is finished, the 803 module analyzes all the gradient macro blocks to generate a plurality of gradient areas. The generation method comprises the following steps:

(1) if two gradient macro blocks are adjacent horizontally or vertically, the gradient area includes the two macro blocks;

(2) if several gradual change macro blocks are adjacent in group, but they form a rectangle that also contains a row or a column of macro blocks that are not all gradual change macro blocks, the gradual change area will be defined as the rectangle, and the row or the column of non-full gradual change macro blocks will also be defined. This situation is illustrated in fig. 4A and 4B.

If macroblock 1/2/3 is a fade macroblock or macroblock 1/2/3/4 is a fade macroblock in fig. 4A, the fade area is a rectangle formed by the four macroblocks.

In fig. 4B, the portion marked by the dotted line is a gradation macroblock, and the region marked by X is a non-gradation macroblock. This case would then delimit the peripheral several non-gradient macroblocks also within the gradient area. It should be noted that, for the accuracy of reconstruction at the decoding end, when defining the gradient region, at most only one layer of non-gradient macro blocks is allowed to be covered in the rectangle.

804: and a text macro block coding module. All the character macro blocks are coded, and the coding method mainly comprises palette quantization and huffman coding.

805: and a picture macro block coding module. And coding all the picture macroblocks by JPEG.

806: and a transition area information generation module. The previous 803 module generates one or more fade areas, which are rectangles containing a plurality of macroblocks. The module 806 records these areas. For each transition area, the index numbers of the first macro block and the last macro block are recorded. And after the information of all the transition regions is encoded by using huffman, the information is the transition region information.

807: and a code stream generation module. And packing the generated character block code stream, the picture block code stream and the information data of the gradual change area to form a whole frame coding code stream.

The components at the decoding end are as follows:

808: and a code stream splitting module. The code stream is divided into a text block code stream, a picture block code stream and gradual change region information data. The first two are handed to the 809 module for decoding. The fade area information data is handed 810 to perform the fade area filtering.

810: and a gradual change region filtering module. The module receives the information of the gradual change area and carries out filtering processing on the boundaries of the macro blocks in the area. Fig. 6A shows a fade area composed of four macroblocks, and the following describes the boundary filtering method by taking this figure as an example.

First, define the macroblock boundary as: two macroblocks border 6 rows or 6 columns of pixels. Then in fig. 6A there are two macroblock boundaries as indicated by the dashed line markers. Taking the boundaries of the macro block 1 and the macro block 2 as an example, after the text and picture blocks are completely decoded, the 410 module analyzes the 13 th row of pixels of the macro block 1 and the 2 nd row of pixels of the macro block 2, makes a difference value for each pixel of the two rows in each row, then divides the difference value by 6 to obtain a gradual change value, then regards the 13 th row of the macro block 1 and the 2 nd row of the macro block 2 as a minimum value and a maximum value, keeps the pixel value of the 13 th row of the macro block 1 unchanged, and subtracts the gradual change value for the pixel of the last row of pixels adjacent to the row of pixels. For example, the adjacent 6 pixel values of the adjacent two macroblocks before filtering are:

72 71 71|65 65 65

then (72-65)/6 is calculated to be 1.17, which can be regarded as 1 in engineering calculation, or can be calculated in decimal number and then rounded to integer pixel value. The filtered value is then:

72 71 70|69 68 67

it can be seen that there is a jump of 71 to 65 between original macroblocks, and after the smoothing process, the boundaries of the pixels are smoothed.

In fig. 6A, 6 boundary pixels of each row adjacent to the

macro blocks

1 and 2 are processed by the above method, i.e., the boundaries between macro blocks can be smoothed. Finally the 811 module reconstructs the filtered image.

Fig. 6B shows a fade area consisting of four macroblocks, and it can be seen that there is a pixel transition at both boundary 1 and boundary 2, e.g., there is a 71 transition to 65 transition. Fig. 6C shows the situation after the filtering process, where the visible pixel transitions are eliminated and the boundary is smoothed.

The scheme provides a phenomenon that the step-shaped visual effect formed at the junctions of macro blocks of different types in the gradual color change area can be processed. When a computer synthesizes an image for coding, different coding modes are often used for different macro blocks, or the same type of macro blocks bring pixel loss due to quantization. It is not clearly acceptable in the non-gradient area, but in the gradient color area, the formed macroblock boundary is more sensitive to the human eye. According to the scheme, the gradual change area is identified when the macro blocks are classified, the boundaries of the macro blocks in the gradual change area are smoothed at the decoding end, and the visual experience of the gradual change area is enhanced.

Based on the image encoding method described in the embodiment corresponding to fig. 3, the following is an embodiment of the apparatus of the present disclosure, which can be used to execute an embodiment of the method of the present disclosure.

Fig. 9 is a schematic structural diagram of an image encoding apparatus according to an embodiment of the present disclosure, and as shown in fig. 9, the apparatus 90 includes:

a macroblock dividing module 901, configured to perform macroblock division on a current frame image to generate N macroblocks, where N is an integer greater than 1;

a color gradient area determining module 902, configured to determine a color gradient macro block from the N macro blocks and determine a color gradient area according to the color gradient macro block, where the color gradient area includes at least 2 adjacent color gradient macro blocks;

a color gradient area coding module 903, configured to code the color gradient area according to a first coding method, and generate a code stream of the color gradient area;

a text macro block encoding module 904, configured to encode the text macro blocks except the color gradient region according to a second encoding manner and encode the picture macro except the color gradient region according to a third encoding manner, so as to generate a code stream of the text macro blocks and a code stream of the picture macro blocks respectively;

a current frame image code stream generating module 905, configured to generate the current frame image code stream according to the code stream of the color gradient area, the code stream of the text macro block, and the code stream of the picture macro block, and send the current frame image code stream to a decoding end.

In one embodiment, the color gradient region determination module 903 is configured to:

The implementation process and technical effects of the image encoding apparatus provided in the embodiment of the present disclosure can be seen in the embodiment of fig. 3, which is not described herein again.

Based on the image decoding method described in the embodiment corresponding to fig. 5, the following is an embodiment of the apparatus of the present disclosure, which can be used to execute an embodiment of the method of the present disclosure.

Fig. 10 is a schematic structural diagram of an image decoding apparatus according to an embodiment of the present disclosure, and as shown in fig. 10, the apparatus 100 includes:

a current frame image code stream receiving module 1001, configured to receive a code stream of a current frame image sent by a coding end, and determine a code stream of a text macro block, a code stream of a picture macro block, and a code stream of a color gradient area from the code stream of the current frame image respectively;

the color gradient area code stream decoding module 1002 is configured to decode the code stream of the color gradient area according to a first decoding manner, and generate a color gradient area, where the color gradient area includes at least 2 macro blocks;

a boundary filtering processing module 1003, configured to determine a boundary between every two adjacent macroblocks in the color gradient area, and perform filtering processing on the boundary to generate a filtered color gradient area;

a text macro block code stream decoding module 1004, configured to decode the code stream of the text macro block according to a second decoding manner and decode the code stream of the picture macro block according to a third decoding manner, so as to generate a text macro block and a picture macro block respectively;

a current frame image generating module 1005, configured to generate the current frame image according to the filtered color gradient area, the text macro block, and the picture macro block.

In one embodiment, the boundary filtering processing module is configured to:

and taking the adjacent M rows or M columns of

pixels

1003 and 1003 of every two adjacent macro blocks in the color gradient area as the boundary of the two adjacent macro blocks, wherein M is an integer larger than 1.

In one embodiment, the boundary filtering processing module 1003 is configured to:

The implementation process and technical effects of the image decoding apparatus provided in the embodiment of the present disclosure can be seen in the embodiment of fig. 5, which is not described herein again.

It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program instructing relevant hardware, where the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.

Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This application is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.

Claims

1. An image encoding method, comprising:

2. The method of claim 1, wherein determining a color gradient macroblock from the N macroblocks comprises:

3. The method of claim 2, wherein calculating the pixel difference value of the target macroblock and the first macroblock comprises:

4. The method of claim 2, wherein said determining a color gradient area from said color gradient macroblock comprises:

5. An image decoding method, comprising:

6. The method of claim 5, wherein the determining the boundary of each two adjacent macroblocks in the color gradient region comprises:

7. The method of claim 6, wherein the filtering the boundary comprises:

8. The method of claim 6, wherein the filtering the boundary comprises:

9. An image encoding device characterized by comprising:

10. An image decoding apparatus, comprising: