WO2016197893A1

WO2016197893A1 - Image encoding and decoding method, image processing device, and computer storage medium

Info

Publication number: WO2016197893A1
Application number: PCT/CN2016/084992
Authority: WO
Inventors: 林涛; 李明; 吴平; 尚国强; 吴钊
Original assignee: 同济大学; 中兴通讯股份有限公司
Priority date: 2015-06-08
Filing date: 2016-06-06
Publication date: 2016-12-15

Abstract

Disclosed are an image encoding and decoding method, image processing device, and computer storage medium, comprising: when using one of a palette and a pixel-string encoding replication means to replicate the encoding of a currently encoded block, generating a new palette color according to the pixels of said currently encoded block; according to said new palette color and/or palette color candidate set shared by the palette and the pixel-string encoding replication means, generating the palette of said current encoded block; using said palette to replicate the encoding of the palette and the pixel string, and generate a video code stream comprising the replication means and replication parameters. The video code stream is parsed, and at least one of the following information is obtained: the replication means, replication parameters, and new palette color of the palette and pixel-string decoding replication; according to said new palette color and/or palette color candidate set shared by the palette and the pixel-string decoding replication means, generating a palette; using said palette to replicate the decoding of the palette and the pixel string.

Description

Image encoding and decoding method, image processing device, computer storage medium

Technical field

The present invention relates to digital video coding and decoding technologies, and in particular, to an image coding and decoding method, an image processing device, and a computer storage medium.

Background technique

With the development of televisions and monitors entering ultra-high definition (4K) and ultra-high definition (8K) resolutions, as well as the development and popularization of a new generation of cloud computing and information processing modes and platforms based on remote desktops, video images The need for data compression is also moving toward higher resolution and composite images containing camera-captured images and computer screen images. Ultra-high compression ratio and extremely high-quality data compression of video images have become an indispensable technology.

Taking full advantage of the characteristics of 4K/8K images and computer screen images, HEVC (High Efficiency Video Coding) is also the latest international video compression standard and several other international standards and domestic standards under development. A major goal of industry standards.

A notable feature of computer screen images is that there are often many similar or even identical pixel patterns (pi × el pattern) within the same frame image. For example, Chinese or foreign text that often appears in computer screen images is composed of a few basic strokes, and many similar or identical strokes can be found in the same frame image. Menus, icons, etc., which are common in computer screen images, also have many similar or identical patterns. Therefore, the existing image and video compression technologies usually adopt various copying methods, including at least the following copying methods:

1) Intra block copying means intraframe block matching or intraframe motion compensation or block matching or block copying.

2) Intra-frame micro-block copying means intra-frame micro-block matching or micro-block matching or micro-block copying.

3) Intra-frame lines (referred to as strips) are copied, that is, intra-frame matching or strip matching or strip copying.

4) Intra-frame string replication means intra-frame string matching or string matching or string copying or pixel string copying.

5) Palette index string copy is a palette or index string copy.

6) Fusion copy mode of index string copy and pixel string copy, referred to as index-pixel string fusion copy mode, also called palette-pixel string fusion copy mode or merged pixel string copy palette mode or fusion palette Pixel string copy mode or palette with pixel string copy mode.

7) Multiple palettes and pixel string copying methods combining multiple palettes and pixel string copying methods.

Among the various existing copying methods, the best compression effect is the multi-palette and pixel string copying method. However, in the prior art multi-palette and pixel string copying method, each palette and pixel string copying method has separate palette color candidate sets, if the codec block does not adopt a palette and a pixel. In the string copy mode, the palette color candidate set corresponding to the pixel string copy mode is not updated, and no new more suitable palette color is added to the corresponding palette color candidate set. This approach greatly reduces the compression efficiency of multiple palettes and pixel string replication.

Summary of the invention

To solve the above technical problem, an embodiment of the present invention provides an image encoding and decoding method, an image processing device, and a computer storage medium.

An image coding method provided by an embodiment of the present invention includes:

Generating a new palette color according to pixels of the current coding block when copying and encoding the current coding block using one of a palette and a pixel string copy coding method;

Generating a palette of the current coded block according to the palette color candidate set shared by the new palette color and/or color palette and the pixel string copy coding method;

Palette and pixel string copy coding is performed using the palette of the current coded block, and a video code stream including a copy mode and a copy parameter is generated.

In this embodiment, the coding block is an coding region of an image, and the coding block includes at least one of: a maximum coding unit (LCU), a coding tree unit (CTU), a coding unit (CU), and a sub-region of a CU. , prediction unit (PU), transform unit (TU).

In an embodiment of the invention, the video code stream further includes the new palette color.

In the embodiment of the present invention, the maximum number of palette colors that the maximum space of the palette color candidate set can accommodate is a predetermined constant.

In the embodiment of the present invention, the method further includes: the palette color candidate set performs one or more of the following operations according to a predetermined rule:

Update, palette color deletion, palette color position adjustment, palette color addition.

In the embodiment of the present invention, the palette color candidate set adds a new palette color and/or deletes an existing palette color according to a first-in, first-out rule.

In the embodiment of the present invention, the method further includes:

For a coded block that is encoded using one of the palette and pixel string copying methods, some or all of the new palette color of the encoded block is added to the shared palette color candidate set.

The image decoding method provided by the embodiment of the present invention includes:

Parsing the video stream to obtain at least one of the following: a copy mode of the palette and the pixel string copy decoding, a copy parameter, and a new palette color;

Generating a palette color candidate set shared by the new palette color and/or color palette and the pixel string copy decoding method, and generating a palette and pixel string copy decoding using the copy mode for the current decoded block. The palette used;

Palette and pixel string copy decoding is performed using the palette of the current decoded block.

In the embodiment of the present invention, the decoding block is a decoding area of an image, and the decoding block includes at least one of the following: an LCU, a CTU, a CU, a sub-area of the CU, a PU, and a TU.

In the embodiment of the present invention, the method further includes:

For a decoded block that is decoded using one of the palette and pixel string copying methods, some or all of the new palette color of the decoded block is added to the shared palette color candidate set.

An image processing device provided by an embodiment of the present invention includes:

a first generating unit, configured to generate a new palette color according to the pixel of the current encoding block when copying and encoding the current encoding block by using one of a color palette and a pixel string copy encoding method;

a second generating unit, configured to generate a palette of the current encoding block according to the palette color candidate set shared by the new palette color and/or color palette and the pixel string copy encoding manner;

And a coding unit configured to perform palette and pixel string copy coding using a palette of the current coded block, and generate a video code stream including a copy mode and a copy parameter.

An image processing device according to another embodiment of the present invention includes:

a parsing unit configured to parse the video stream to obtain at least one of the following: a copy mode of the palette and the pixel string copy decoding, a copy parameter, and a new palette color;

a third generating unit configured to generate the copy to the current decoded block according to the palette color candidate set shared by the new palette color and/or color palette and the pixel string copy decoding mode The color palette used for palette and pixel string copy decoding;

A decoding unit configured to perform palette and pixel string copy decoding using a palette of the current decoded block.

The computer storage medium provided by the embodiment of the present invention stores a computer program for performing the above image encoding and/or image decoding method.

In the technical solution of the embodiment of the present invention, encoding the image includes: when copying and encoding the current coding block by using one of a palette and a pixel string copy coding mode, generating a new color palette according to the pixel of the current coding block. Color; generating a palette of the current encoded block according to the palette color candidate set shared by the new palette color and/or palette and pixel string copy encoding; using the current encoding block The palette performs palette and pixel string copy encoding and generates a video stream that includes copy mode and copy parameters. Decoding the image includes parsing the video stream, and obtaining at least one of the following: a copy mode of the palette and the pixel string copy decoding, a copy parameter, and a new palette color; according to the new palette color and/or Or a palette color candidate set shared by the palette and the pixel string copy decoding method, generating a palette used for the current decoded block to perform palette and pixel string copy decoding using the copy mode; using the current decoding The palette of the block performs a palette and pixel string copy decoding. It can be seen that the embodiment of the present invention implements image compression of a plurality of copy mode sharing palette color candidate sets, and improves compression efficiency of multiple color palettes and pixel string copy modes.

DRAWINGS

In the drawings, which are not necessarily to scale, the Like reference numerals with different letter suffixes may indicate different examples of similar components. The drawings generally illustrate the various embodiments discussed herein by way of example and not limitation.

1 is a schematic flowchart 1 of an image encoding method according to an embodiment of the present invention;

2 is a schematic flowchart 1 of an image decoding method according to an embodiment of the present invention;

3 is a second schematic flowchart of an image encoding method according to an embodiment of the present invention;

4 is a second schematic flowchart of an image decoding method according to an embodiment of the present invention;

FIG. 5 is a first schematic structural diagram of an image processing apparatus according to an embodiment of the present invention; FIG.

FIG. 6 is a second schematic structural diagram of an image processing apparatus according to an embodiment of the present invention; FIG.

FIG. 7 is a third schematic structural diagram of an image processing apparatus according to an embodiment of the present invention.

detailed description

The embodiments of the present invention are described in detail below with reference to the accompanying drawings.

The natural form of a digital video signal of an image is a sequence of images. A frame of image is usually a rectangular area composed of several pixels, and a digital video signal is a sequence of video images composed of tens of frames to thousands of frames of images, sometimes simply referred to as a video sequence or sequence. Encoding a digital video signal encodes a frame by frame image. At any one time, the image of the frame being encoded is referred to as the current encoded image. Similarly, decoding a compressed video stream (also referred to as a bit stream) of a digital video signal is decoding a stream of one frame by one compressed image. At any one time, the image of the frame being decoded is referred to as the current decoded image. The current encoded image or the currently decoded image is collectively referred to as the current image.

In almost all international standards for video image coding, such as MPEG-1/2/4, H.264/AVC, and HEVC, when one frame of image is encoded (and correspondingly decoded), one frame of image is divided into blocks M. The sub-image of the M pixel is called a coding block (from the perspective of decoding, that is, a decoding block, collectively referred to as a codec block) or a coding unit (CU, Coding Unit), and the sub-image is performed one by one with the CU as a basic coding unit. coding. The size of the commonly used M is 4, 8, 16, 32, 64. Therefore, encoding the video image sequence is to encode one CU for each coding unit of each frame image, that is, CU. At any one time, the CU being coded is referred to as the current coded CU. Similarly, decoding the code stream of the video image sequence is also one by one for each CU of each frame image. The CUs decode and finally reconstruct the entire video image sequence. At any one time, the CU being decoded is referred to as the currently decoded CU. The current coding CU or the current decoding CU is collectively referred to as the current CU.

In order to adapt to the difference between the content and the nature of each part of the image in one frame, the most efficient coding is performed in a targeted manner. The size of each CU in one frame of image can be different, some are 8×8, some are 64×64, etc. . In order to enable seamlessly splicing CUs of different sizes, one frame of image is usually first divided into LCUs (Lorgest Coding Units) of the same size and having N×N pixels, and then each LCU is further divided into a tree structure. Multiple CUs of different sizes are not necessarily the same. Therefore, the LCU is also called a Coding Tree Unit (CTU). For example, a frame of image is first divided into 64x64 pixel LCUs of the same size (N=64). One of the LCUs is composed of three 32×32 pixel CUs and four 16×16 pixel CUs, so that seven CUs in a tree structure constitute one CTU. The other LCU is composed of two 32×32 pixel CUs, three 16×16 pixel CUs, and 20 8×8 pixel CUs. Such 25 CUs in a tree structure constitute another CTU. Encoding one frame of image is to sequentially encode one CU in one CTU. In the HEVC international standard, LCU is synonymous with CTU. A CU whose size is equal to the CTU is called a CU with a depth of zero. A CU obtained by dividing the CU up and down and left and right by a depth of 0 is called a CU having a depth of 1. A CU with a depth of 1 and a CU divided into four equal parts is called a CU with a depth of 2. A CU obtained by dividing the CU of the depth of 2 into four equal parts is called a CU having a depth of 3. At any one time, the CTU being coded is referred to as the current coded CTU. At any one time, the CTU being decoded is referred to as the current decoded CTU. The current coding CTU or the current decoding CTU is collectively referred to as the current CTU.

The CU can also be further divided into sub-areas. Sub-regions include, but are not limited to, prediction unit (PU), transform unit (TU), asymmetric partition (AMP) regions.

A color pixel usually consists of three components. The two most commonly used pixel color formats (pi × el color format) are GBR color formats consisting of green, blue, and red components and consist of one luma component and two chroma components. YUV color format. The color format commonly known as YUV actually includes multiple color formats, such as the YCbCr color format. Therefore, when encoding a CU, a CU can be divided into three component planes (G plane, B plane, R plane or Y plane, U plane, V plane), and the three component planes are respectively coded; The three component bundles of one pixel are combined into one 3-tuple, and the CUs composed of these 3-tuples are encoded as a whole. The arrangement of the former pixel and its components is called the planar format of the image (and its CU), and the arrangement of the latter pixel and its components is called the stacked format of the image (and its CU). Format). The GBR color format and the YUV color format of the pixel are both 3-component representation formats of the pixel.

In addition to the 3-component representation format of a pixel, another common representation format for a pixel is the palette index representation format. In the palette index representation format, the value of a pixel can also be represented by the index of the palette. The palette space stores the value or approximate value of the three components of the color of the pixel to be represented, and the address of the palette is referred to as the index of the color of the pixel stored in this address. An index can represent a component of a pixel's color, and an index can also represent three components of a pixel's color. The palette can be one or more. In the case of multiple palettes, a complete index is actually composed of the palette number (which one of the multiple palettes) and the index of the palette of the sequence number. The index representation format of a pixel is to represent this pixel with an index. If the pixels in an image region (such as a coded block or a decoded block) cannot all be represented by the palette color (ie, at least one pixel in the image region, the value of no three components is equal or approximately equal to the pixel) The palette color and its index), there is usually a special index called escape color in the palette, which is used to represent pixels that cannot be represented by normal palette colors. Therefore, if the index of one pixel is an index of the escape color, the pixel needs to express its color with another dedicated 3 components. The normal color and escape color in the palette are called palette colors, but the escape color is a virtual color. There is no physical space in the palette to store this color. There is only one special dedicated virtual index. . The index of the escape color is usually the last index of the palette. The index representation format of the pixel is also It is called the index color of the pixel (inde × ed color) or pseudo color, or often referred to directly as index pixel (inde × ed pi × el) or pseudo pixel (pseudo pi × el) or pixel index Or index. Indexes are sometimes referred to as indices. The representation of a pixel in its index representation format is also referred to as indexing or indexing.

Other commonly used pixel representation formats include CMYK representation formats and grayscale representation formats.

The YUV color format can be subdivided into several seed formats according to whether the chroma component is downsampled: a YUV 4:4:4 pixel color format consisting of 1 Y component, 1 U component, and 1 V component. The left and right adjacent pixels are composed of 2 Y components, 1 U component, and 1 V component in a YUV 4:2:2 pixel color format; the left and right adjacent 4 pixels arranged in a 2×2 spatial position are composed of YUV4:2:0 pixel color format consisting of 4 Y components, 1 U component, and 1 V component. A component is generally represented by a number of 8 to 16 bits. The YUV4:2:2 pixel color format and the YUV4:2:0 pixel color format are all downsampled for the YUV4:4:4 pixel color format. A pixel component is also referred to as a pixel sample (pi x el sample) or simply as a sample.

The most basic element when encoding or decoding can be one pixel, one pixel component, or one pixel index (ie, index pixel). A pixel or a pixel component or an index pixel, which is the most basic element of encoding or decoding, is collectively referred to as a pixel sample, sometimes referred to as a pixel value, or simply as a sample.

In the embodiment of the present invention, "pixel sample value", "pixel value", "sample value", "index pixel", and "pixel index" are synonyms, and depending on the context, whether "pixel" or "one pixel" is clearly indicated The "component" still means "index pixel" or both. If it is not clear from the context, then it means either of the three.

In an embodiment of the invention, a coded block or a decoded block (collectively referred to as a codec block) is an area consisting of a number of pixel values. The shape of the codec block may be a rectangle, a square, a parallelogram, a trapezoid, a polygon, a circle, an ellipse, and the like. The rectangle also includes the width or height A rectangle whose pixel value degenerates into a line (ie, a line segment or a line shape). In one frame of image, each codec block may have a different shape and size. In one frame of image, some or all of the codec blocks may overlap each other, or all codec blocks may not overlap each other. A codec block may be composed of "pixels", or may be composed of "components of pixels", or may be composed of "index pixels", or may be composed of a mixture of the three, or any of the three. Mixed composition. From the perspective of video image encoding or decoding, a codec block refers to an area in which encoding or decoding is performed in a frame image, including but not limited to at least one of the following: a maximum coding unit LCU, a coding tree unit CTU, and a coding unit CU. a sub-region of the CU, a prediction unit PU, and a transform unit TU.

A notable feature of computer screen images is that there are often many similar or even identical pixel patterns (pi × el pattern) within the same frame image. For example, Chinese or foreign text that often appears in computer screen images is composed of a few basic strokes, and many similar or identical strokes can be found in the same frame image. Menus, icons, etc., which are common in computer screen images, also have many similar or identical patterns. Therefore, the copying method adopted by image and video compression technology includes at least the following copying methods:

1) Intra block copying means intraframe block matching or intraframe motion compensation or block matching or block copying. The basic operation of block copy coding or decoding is to copy a current code block or a current decoding block (referred to as the current block) from the reconstructed reference pixel sample set by the same size as the current block (the same number of pixel samples). The reference block is referenced and the value of the reference block is assigned to the current block. The copy parameter of the block copy mode includes the displacement vector of the current block, indicating the relative position between the reference block and the current block. A current block has a displacement vector.

2) Intra-frame micro-block copying means intra-frame micro-block matching or micro-block matching or micro-block copying. In microblock copying, a current block (such as 8 × 8 pixel samples) is divided into several microblocks (such as microblocks of 4 × 2 pixel samples or microblocks of 8 × 2 pixel samples or 2 × 4 pixels) The microblock of the sample or the microblock of the 2×8 pixel sample), the basic operation of the microblock copy encoding or decoding is to encode each microblock in the current block or Decoding the microblock (referred to as the current microblock), copying a reference microblock from the reconstructed reference pixel sample set, and assigning the value of the reference microblock to the current microblock. The copy parameter of the microblock copy mode includes a displacement vector of the current microblock, indicating a relative position between the reference microblock and the current microblock. A current microblock has a displacement vector. How many displacement vectors are there in how many microblocks a current block is divided into.

3) Intra-frame lines (referred to as strips) are copied, that is, intra-frame matching or strip matching or strip copying. A strip is a microblock having a height of 1 or a width of 1, such as a microblock of 4x1 or 8x1 or 1x4 or 1x8 pixel samples. The basic operation of the strip copy encoding or decoding is to copy each reference strip or decoded strip (referred to as the current strip) in the current block from the reconstructed reference pixel sample set, and the value of the reference strip Assign to the current bar. Obviously, strip copying is a special case of microblock copying. The copy parameter of the strip copy mode includes the displacement vector of the current strip, indicating the relative position between the reference strip and the current strip. A current bar has a displacement vector. How many displacement vectors are there in how many bars a current block is divided into.

4) Intra-frame string replication means intra-frame string matching or string matching or string copying or pixel string copying. In pixel string copying, a current coded block or a current decoded block (referred to as the current block) is divided into several variable-length pixel sample strings. The string here refers to arranging the pixel samples in a two-dimensional area of an arbitrary shape into a string whose length is much larger than the width (for example, a string having a width of 1 pixel sample and a length of 37 pixel samples or a width of 2) A string of pixel samples of length 111 pixels, typically but not limited to a length that is an independent encoding or decoding parameter and a width that is a predetermined or derived parameter from other encoding or decoding parameters). The basic operation of string copy encoding or decoding is to copy each reference string from the reconstructed reference pixel sample set to each encoded string or decoded string (referred to as the current string) in the current block, and the value of the reference string Assign to the current string. The copy parameters of the string copy mode include the displacement vector and the copy length of the current string, that is, the copy size, which respectively represent the relative position between the reference string and the current string and the length of the current string, that is, the number of pixel samples. The length of the current string is also the length of the reference string. A current string has a displacement vector and a copy length. How many displacement vectors and how many copy lengths a current block is divided into. Displacement vector The quantity is also called the copy position, and its representation forms are: 2D coordinates, linear address, distance, pointer, index, number, mode number, and so on.

5) Palette index string copy is a palette or index string copy. In the palette encoding and the corresponding decoding method, first construct or acquire a palette, and then use some or all of the pixels of the current encoding block or the current decoding block (referred to as the current block) with the palette in the palette. The index of the color is expressed, and then the index is encoded and decoded, including but not limited to: dividing the index of a current block into several index strings of variable length, that is, performing index string copy encoding and decoding. The basic operation of the index string copy encoding or decoding is to copy each reference index string or index decoding string (referred to as the current index string) in the current block, and copy a reference index string from the indexed reconstructed reference pixel sample set. And assigning an index value of the reference index string to the current index string. The copy parameter of the index string copy mode includes the displacement vector and the copy length of the current index string, that is, the copy size, respectively indicating the relative position between the reference index string and the current index string and the length of the current index string, that is, the number of corresponding pixel samples. The length of the current index string is also the length of the reference index string. A current index string has a displacement vector and a copy length. How many displacement vectors and how many copy lengths a current index is divided into. The displacement vector is also called the copy position, and its representation forms are: 2D coordinates, linear address, distance, pointer, index, number, mode number, and so on. In the palette index string copy mode, the copied reference pixels are taken from the palette. Therefore, the index string is also called a palette color string or a palette pixel string or a palette string, and the index string copy mode is also called a palette copy mode or a palette mode. In the palette mode, strings are also called runs or strokes. Therefore, the index string is also called an index run or index run or simply a run or trip. The palette color of the current block is derived from the pixel color and/or palette color candidate set of the current block, and the palette color candidate set is composed of a portion of the palette color of the codec block that has completed the codec.

6) Fusion copy mode of index string copy and pixel string copy, referred to as index-pixel string fusion copy mode, also called palette-pixel string fusion copy mode or merged pixel string copy palette mode or fusion palette Pixel string copy mode or palette with pixel string copy mode. For one When the current coding block or the current decoding block (referred to as the current block) is encoded or decoded, part or all of the pixels adopt the pixel string copy mode, and some or all of the pixels adopt the index string copy mode, that is, the reference pixels are taken from the palette of the palette. the way.

7) Multiple palettes and pixel string copying methods combining multiple palettes and pixel string copying methods. Encoding and decoding are performed using at least two different color palettes and pixel string copying methods. Different color palettes and pixel string copying methods use different fusion schemes to fuse palette copying and pixel string copying. In one frame of image, part of the codec block adopts the palette and pixel string copy mode of the fusion scheme I, and part of the codec block adopts the palette and pixel string copy mode of the fusion scheme II, and some codec blocks adopt the fusion scheme III. The palette is copied with pixel strings, and so on. Different fusion schemes may differ in the following aspects: 1) the number of types of strings, 2) the range of values of parameters, 3) the range of values of one or several replicated parameters, and 4) the location of the displacement vector Range of values, 5) current index or current pixel position, 6) position of the current sample segment, 7) position of the reference index or reference pixel, 8) position of the reference sample segment; 9) copy shape.

Other methods of copying include rectangular copying, and a variety of copying methods.

The block in the block copy mode, the micro block in the micro block copy mode, the bar in the strip copy mode, the string in the string copy mode, the rectangle in the rectangular copy mode, and the pixel index string in the palette index mode are collectively referred to as A pixel sample segment, referred to as a sample segment. The basic constituent elements of a sample segment are pixel or pixel components or pixel indices. A sample segment has a copy parameter that represents the relationship between the current pixel sample segment and the reference pixel sample segment. Therefore, a sample segment is the smallest unit of a copy operation with the same replication relationship. A copy parameter includes a plurality of copy parameter components, and the copy parameter components include at least: displacement vector horizontal component, displacement vector vertical component, 1 dimensional displacement vector, linear address, relative linear address, index, palette linear address, relative index, toning Board relative linear address, copy length, copy width, copy height, rectangle width, rectangle length, unmatched pixels (also known as no reference pixels, ie non-replicated pixels that are not copied from elsewhere).

In various ways of copying, pixel samples or indexes need to be arranged in a certain order. The arrangement is also called the scanning method. The scanning method can be divided into the following types according to its path shape:

A) Horizontal Z-scan mode is also called horizontal raster scan mode. A pixel sample or index of a coded block or a decoded block (collectively referred to as a codec block) is arranged line by line and arranged in the same direction (all left to right or all right to left) in all rows. Rows and rows can be arranged from top to bottom or from bottom to top.

B) Vertical Z-scan mode is also called vertical raster scan mode. A pixel sample or index of a coded block or a decoded block (collectively referred to as a codec block) is arranged in columns and columns, and is arranged in the same direction (all from top to bottom or all from bottom to top) in all columns. Columns and columns can be arranged from left to right or from right to left.

C) Horizontal bow scan mode. A pixel sample or index of a coded block or a decoded block (collectively referred to as a codec block), arranged line by line, arranged in one direction (eg, from left to right) in an odd line and in another (opposite) direction in an even line (eg, from right to left). Rows and rows can be arranged from top to bottom or from bottom to top.

D) Vertical bow scanning method. A pixel sample or index of a coded block or a decoded block (collectively referred to as a codec block), arranged in columns and columns, arranged in one direction (eg, from top to bottom) in an odd column and in another in an even column (opposite ) Directions (eg, from bottom to top). Columns and columns can be arranged from left to right or from right to left.

In the string copy (index string copy or pixel string copy or index string-pixel string merge copy) mode, the basic copy shape has the following two types:

I) One-dimensional linear replication shape. The current string and the reference string are each a one-dimensional sample string formed in the order of a predetermined scanning manner in each codec block, and have equal lengths, but the two-dimensional regions formed by the two strings are not necessarily the same. Two-dimensional shape.

II) Two-dimensional conformal copy shape. The current strings are arranged in the order of the predetermined scanning mode within the current codec block. The reference string remains exactly the same two-dimensional shape as the current string, with equal length.

Each of the above basic replicated shapes can be subdivided into a plurality of replicated shapes according to a specific scanning manner, such as a vertical arcuate one-dimensional linear replica shape, a horizontal Z-shaped two-dimensional conformal replica shape, and the like.

It should be noted that “copying” is an operation of reconstruction and decoding, and the corresponding encoding operation is “matching”. Therefore, various copying methods such as block matching mode, microblock copying mode, line copying mode, pixel string copying mode, and index string copying mode are also called block matching mode, microblock matching mode, line matching mode, and pixel string matching mode. , index string matching method, and so on.

Among the various copying methods, the best compression effect is the multi-palette and pixel string copying method. However, in the multi-palette and pixel string copy mode, each palette and pixel string copy mode has separate palette color candidate sets, if one codec block does not adopt a palette and pixel string copy mode. The palette color candidate set corresponding to the pixel string copy mode is not updated, and no new more suitable palette color is added to the corresponding palette color candidate set. This approach greatly reduces the compression efficiency of multiple palettes and pixel string replication.

Based on this, an embodiment of the present invention provides an image encoding and decoding method and an image processing device. When encoding and decoding a codec block, some or all of the new palette color is placed in the shared palette color candidate set regardless of which palette and pixel string copying method is employed. When encoding and decoding subsequent codec blocks, various palette and pixel string copy modes generate respective palettes from the pixels of the current codec block and/or the shared palette color candidate set. The palette color candidate set is also known as the palette predictor or palette prediction set.

In the embodiment of the present invention, the maximum number of palette colors that can be accommodated in the maximum space of the palette color candidate set is a predetermined constant. In the embodiment of the present invention, the palette color candidate set is updated according to a predetermined rule. In the embodiment of the present invention, the palette color candidate set deletes an existing palette color according to a predetermined rule. In the embodiment of the present invention, the palette color candidate set adjusts the position of the palette color according to a predetermined rule, that is, changes its address or pointer. In an embodiment of the invention, the palette color candidate set adds a new palette color according to a predetermined rule. In the embodiment of the present invention, the palette color candidate set adds a new tone according to a first-in, first-out rule. Swatch color and / or delete existing palette colors. In the embodiment of the present invention, optionally, one of the plurality of palette and pixel string copying modes is a simple palette copying mode. In the embodiment of the present invention, any codec block is encoded and decoded by using one of a palette and a pixel string copy mode, and part or all of the newly generated palette color of the codec block is added to the shared palette. Color candidate set.

FIG. 1 is a schematic flowchart 1 of an image encoding method according to an embodiment of the present invention. In the embodiment of the present invention, when copying and encoding a current encoding block by using one of a palette and a pixel string copy encoding method, a pixel from a current encoding block is used. Generating a new palette color, generating a palette of current coded blocks from the palette color candidate set shared by the new palette color and/or palette and pixel string copy encoding, using the tone The swatch performs palette and pixel string copy coding and generates a video code stream containing at least a copy mode and a copy parameter; as shown in FIG. 1, the image coding method includes the following steps:

Step 101: When copying and encoding the current coding block by using one of a color palette and a pixel string copy coding mode, a new color palette color is generated according to the pixel of the current coding block.

Step 102: Generate a palette of the current coded block according to the palette color candidate set shared by the new palette color and/or color palette and the pixel string copy coding mode.

Step 103: Perform palette and pixel string copy coding using the palette of the current coding block, and generate a video code stream including a copy mode and a copy parameter.

In the foregoing solution of the embodiment of the present invention, the coding block is an coding region of an image, and the coding block includes at least one of: a maximum coding unit LCU, a coding tree unit CTU, a coding unit CU, a sub-region of a CU, and a prediction unit. PU, transform unit TU.

In the above solution of the embodiment of the present invention, the video code stream further includes the new palette color.

In the above solution of the embodiment of the present invention, optionally, the color palette and the pixel string copying manner have respective palettes.

In the above solution of the embodiment of the present invention, the maximum number of palette colors that the maximum space of the palette color candidate set can accommodate is a predetermined constant.

In the above solution of the embodiment of the present invention, the palette color candidate set performs one or more of the following operations according to a predetermined rule:

In an embodiment, the palette color candidate set adds a new palette color and/or deletes an existing palette color according to a first in first out rule.

In the above solution of the embodiment of the present invention, optionally, one of the plurality of palette and pixel string copying modes is a simple palette copying mode.

In the above solution of the embodiment of the present invention, part or all of the new palette color of the coding block is added to the sharing for the coding block coded by using one of the palette and the pixel string copy mode. Palette color candidate set.

FIG. 2 is a schematic flowchart 1 of an image decoding method according to an embodiment of the present invention. In the embodiment of the present invention, a video code stream is parsed, and at least one of the following information is obtained: a copy mode, a copy parameter, and a copying manner of a palette and a pixel string copy and decoding. Selecting a new palette color, from the new palette color and/or palette to the palette color copy set shared by the pixel string copy decoding mode, generating the copy mode for the current decoded block The palette used for the palette and pixel string copy decoding; as shown in FIG. 2, the image decoding method includes the following steps:

Step 201: Parse the video code stream, and obtain at least one of the following information: a copy mode of the palette and the pixel string copy decoding, a copy parameter, and a new palette color.

Step 202: Generate a palette and a pixel string using the copy mode for the current decoding block according to the palette color candidate set shared by the new palette color and/or color palette and the pixel string copy decoding mode. Copy the palette used for decoding.

Step 203: Perform a palette and pixel string copy solution using the palette of the current decoded block. code.

In the above solution of the embodiment of the present invention, the decoding block is a decoding area of an image, and the decoding block includes at least one of the following: an LCU, a CTU, a CU, a sub-area of the CU, a PU, and a TU.

In the above solution of the embodiment of the present invention, part or all of the new palette color of the decoding block is added to the shared decoding block for decoding using one of the palette and the pixel string copying mode. Palette color candidate set.

The technical solution of the embodiment of the present invention is applicable to encoding and decoding of a stacked format image. The technical solution of the embodiment of the present invention is also applicable to encoding and decoding of component plane format images.

FIG. 3 is a second schematic flowchart of an image encoding method according to an embodiment of the present invention. As shown in FIG. 3, in the process:

When copying and encoding the current coded block using one of the palette and the pixel string copy coding method, a new palette color is generated from the pixels of the current coded block, from the new palette color and/or color The palette color candidate set shared by the board and the pixel string copy coding method generates the current coding block a palette that uses the palette to perform palette and pixel string copy encoding and to generate a video stream containing at least copy mode, copy parameters. In an embodiment, the video stream contains a new palette color. In an embodiment, optionally, the palette and pixel string copy encoding methods have separate palettes. In an embodiment, the maximum number of palette colors that the maximum space of the palette color candidate set can accommodate is a predetermined constant. In an embodiment, the palette color candidate set is updated according to a predetermined rule. In an embodiment, the palette color candidate set deletes an existing palette color according to a predetermined rule. In an embodiment, the palette color candidate set adjusts the position of the palette color according to a predetermined rule, ie, changes its address or pointer. In an embodiment, the palette color candidate set adds a new palette color according to a predetermined rule. In an embodiment, the palette color candidate set adds a new palette color and/or deletes an existing palette color according to a first in first out rule. In an embodiment, optionally, one of the plurality of palette and pixel string copy coding modes is a simple palette copy coding mode. In an embodiment, any one of the coding blocks is encoded by using one of a palette and a pixel string copy coding method, and part or all of the newly generated palette color of the coding block is added to the shared palette. Color candidate set.

4 is a schematic flowchart 2 of an image decoding method according to an embodiment of the present invention. As shown in FIG. 4, in the process:

Parsing the video stream to obtain at least one of the following: a copy mode of the palette and pixel string copy decoding, a copy parameter, and an optional new palette color, from the new palette color and/or tone The palette color candidate set shared by the swatch and the pixel string copy decoding mode generates a palette used for the current decoded block to perform palette and pixel string copy decoding using the copy mode. In an embodiment, the video stream contains a new palette color. In an embodiment, optionally, the palette and pixel string copy decoding modes have separate palettes. In an embodiment, the maximum number of palette colors that the maximum space of the palette color candidate set can accommodate is a predetermined constant. In an embodiment, the palette color candidate set is performed according to a predetermined rule. Update. In an embodiment, the palette color candidate set deletes an existing palette color according to a predetermined rule. In an embodiment, the palette color candidate set adjusts the position of the palette color according to a predetermined rule, ie, changes its address or pointer. In an embodiment, the palette color candidate set adds a new palette color according to a predetermined rule. In an embodiment, the palette color candidate set adds a new palette color and/or deletes an existing palette color according to a first in first out rule. In an embodiment, optionally, one of the plurality of palette and pixel string copy decoding modes is a simple palette copy decoding mode. In an embodiment, the arbitrary decoding block is added to the shared palette color candidate by using one of the palette and the pixel string copy decoding mode, and adding part or all of the newly generated palette color of the decoded block. set.

FIG. 5 is a first schematic structural diagram of an image processing apparatus according to an embodiment of the present invention. The image processing apparatus can encode an image. As shown in FIG. 5, the image processing apparatus includes:

The first generating unit 51 is configured to generate a new palette color according to the pixel of the current encoding block when copying and encoding the current encoding block by using one of a palette and a pixel string copy encoding method;

a second generating unit 52, configured to generate a palette of the current encoding block according to the palette color candidate set shared by the new palette color and/or color palette and the pixel string copy encoding manner;

The encoding unit 53 is configured to perform palette and pixel string copy encoding using the palette of the current encoding block, and generate a video code stream including a copy mode and a copy parameter.

In an embodiment, the video stream contains a new palette color. In an embodiment, optionally, the palette and pixel string copy encoding methods have separate palettes. In an embodiment, the maximum number of palette colors that the maximum space of the palette color candidate set can accommodate is a predetermined constant. In an embodiment, the palette color candidate set is updated according to a predetermined rule. In an embodiment, the palette color candidate set deletes an existing palette color according to a predetermined rule. In an embodiment, the palette color candidate set adjusts the position of the palette color according to a predetermined rule, ie, changes its address or pointer. In an embodiment, the The palette color candidate set adds a new palette color according to a predetermined rule. In an embodiment, the palette color candidate set adds a new palette color and/or deletes an existing palette color according to a first in first out rule. In an embodiment, optionally, one of the plurality of palette and pixel string copy coding modes is a simple palette copy coding mode. In an embodiment, any coding block is encoded by using any one of a palette and a pixel string copy coding method, and part or all of the newly generated palette color of the coding block is added to the shared color. Board color candidate set.

It will be understood by those skilled in the art that the implementation functions of the units in the image processing apparatus shown in FIG. 5 can be understood by referring to the related description of the foregoing image encoding method. The functions of the units in the image processing apparatus shown in FIG. 5 can be realized by a program running on a processor, or can be realized by a specific logic circuit.

In practical applications, the functions implemented by each unit in the image processing device may be implemented by a central processing unit (CPU) or a microprocessor (Micro Processor Unit, MPU) located in the image processing device. Or a digital signal processor (DSP), or a Field Programmable Gate Array (FPGA).

FIG. 6 is a second schematic structural diagram of an image processing apparatus according to an embodiment of the present invention. The image processing apparatus can decode an image. As shown in FIG. 6, the image processing apparatus includes:

The parsing unit 61 is configured to parse the video code stream, and obtain at least one of the following information: a copy mode of the palette and the pixel string copy decoding, a copy parameter, and a new palette color;

The third generating unit 62 is configured to generate, according to the new palette color and/or the palette color candidate set shared by the palette and the pixel string copy decoding mode, to generate the current decoding block by using the copy mode. The palette used by the swatch and the pixel string to decode and decode;

The decoding unit 63 is configured to perform palette and pixel string copy decoding using the palette of the current decoded block.

In an embodiment, the video stream contains a new palette color. In an embodiment, optionally, the palette and pixel string copy decoding modes have separate palettes. In an embodiment, the maximum number of palette colors that the maximum space of the palette color candidate set can accommodate is a predetermined constant. In an embodiment, the palette color candidate set is updated according to a predetermined rule. In an embodiment, the palette color candidate set deletes an existing palette color according to a predetermined rule. In an embodiment, the palette color candidate set adjusts the position of the palette color according to a predetermined rule, ie, changes its address or pointer. In an embodiment, the palette color candidate set adds a new palette color according to a predetermined rule. In an embodiment, the palette color candidate set adds a new palette color and/or deletes an existing palette color according to a first in first out rule. In an embodiment, one of the palette and pixel string copy decoding modes is a simple palette copy decoding mode. In an embodiment, the arbitrary decoding block is added to the shared palette color candidate by using one of the palette and the pixel string copy decoding mode, and adding part or all of the newly generated palette color of the decoded block. set.

It will be understood by those skilled in the art that the implementation functions of the units in the image processing apparatus shown in FIG. 6 can be understood with reference to the related description of the aforementioned image decoding method. The functions of the units in the image processing apparatus shown in FIG. 6 can be realized by a program running on a processor, or can be realized by a specific logic circuit.

FIG. 7 is a third schematic structural diagram of an image processing apparatus according to an embodiment of the present invention. The image processing apparatus can encode and decode an image. As shown in FIG. 7, the image processing apparatus includes:

The first generating unit 71 is configured to generate a new palette color according to the pixel of the current encoding block when copying and encoding the current encoding block by using one of a palette and a pixel string copy encoding method;

a second generating unit 72, configured to generate a palette of the current encoding block according to the palette color candidate set shared by the new palette color and/or color palette and the pixel string copy encoding manner;

The encoding unit 73 is configured to perform palette and pixel string copy encoding using the palette of the current encoding block, and generate a video code stream including a copy mode and a copy parameter.

The parsing unit 74 is configured to parse the video code stream, and obtain at least one of the following information: a copy mode of the palette and the pixel string copy decoding, a copy parameter, and a new palette color;

The third generating unit 75 is configured to generate, according to the new palette color and/or the palette color candidate set shared by the palette and the pixel string copy decoding mode, to generate the current decoding block by using the copy mode. The palette used by the swatch and the pixel string to decode and decode;

Decoding unit 76 is configured to perform palette and pixel string copy decoding using the palette of the current decoded block.

Those skilled in the art will appreciate that embodiments of the present invention can be provided as a method, system, or computer program product. Accordingly, the present invention can take the form of a hardware embodiment, a software embodiment, or a combination of software and hardware. Moreover, the invention can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) including computer usable program code.

In the embodiment of the present invention, the image processing device is implemented in the form of a software function module. When sold or used as a standalone product, it can also be stored on a computer readable storage medium. Based on such understanding, the technical solution of the embodiments of the present invention may be embodied in the form of a software product in essence or in the form of a software product stored in a storage medium, including a plurality of instructions. A computer device (which may be a personal computer, server, or network device, etc.) is caused to perform all or part of the methods described in various embodiments of the present invention. The foregoing storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read only memory (ROM), a magnetic disk, or an optical disk. Thus, embodiments of the invention are not limited to any specific combination of hardware and software.

Correspondingly, an embodiment of the present invention further provides a computer storage medium, wherein a computer program for executing an image encoding method and/or an image decoding method according to an embodiment of the present invention is stored.

The present invention has been described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (system), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or FIG. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing device to produce a machine for the execution of instructions for execution by a processor of a computer or other programmable data processing device. Means for implementing the functions specified in one or more of the flow or in a block or blocks of the flow chart.

The computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture comprising the instruction device. The apparatus implements the functions specified in one or more blocks of a flow or a flow and/or block diagram of the flowchart.

These computer program instructions can also be loaded onto a computer or other programmable data processing device such that a series of operational steps are performed on the computer or other programmable device to produce the computer The implemented processing, such as instructions executed on a computer or other programmable device, provides steps for implementing the functions specified in one or more blocks of the flowchart or in a block or blocks of the flowchart.

The above is only the preferred embodiment of the present invention and is not intended to limit the scope of the present invention.

Industrial applicability

In the technical solution of the embodiment of the present invention, when copying and encoding the current coding block by using one of the color palette and the pixel string copy coding mode, a new color palette color is generated according to the pixel of the current coding block; Palette color and/or palette color palette color candidate set shared with pixel string copy encoding, generating a palette of the current encoding block; coloring using the palette of the current encoding block The board and the pixel string are copied and encoded, and a video stream including a copy mode and a copy parameter is generated. Parsing the video stream to obtain at least one of the following: a copy mode of the palette and the pixel string copy decoding, a copy parameter, and a new palette color; according to the new palette color and/or color palette a palette color candidate set shared by the pixel string copy decoding method, generating a palette used for the current decoded block to perform palette and pixel string copy decoding using the copy mode; using the palette of the current decoded block Perform palette and pixel string copy decoding. The image compression of the palette color candidate set is realized by multiple copy modes, and the compression efficiency of the multi-palette and pixel string copy mode is improved.

Claims

An image encoding method, the method comprising:

Generating a new palette color according to pixels of the current coding block when copying and encoding the current coding block using one of a palette and a pixel string copy coding method;

Generating a palette of the current coded block according to the palette color candidate set shared by the new palette color and/or color palette and the pixel string copy coding method;

Palette and pixel string copy coding is performed using the palette of the current coded block, and a video code stream including a copy mode and a copy parameter is generated.
The image encoding method according to claim 1, wherein the encoding block is an encoding region of an image, and the encoding block comprises at least one of: a maximum coding unit LCU, a coding tree unit CTU, a coding unit CU, a sub-CU Region, prediction unit PU, transform unit TU.
The image encoding method according to claim 1, wherein said video code stream further comprises said new palette color.
The image encoding method according to claim 1, wherein a maximum number of palette colors that the maximum space of the palette color candidate set can accommodate is a predetermined constant.
The image encoding method according to claim 1, wherein the method further comprises: the palette color candidate set performing one or more of the following operations according to a predetermined rule:

Update, palette color deletion, palette color position adjustment, palette color addition.
The image encoding method according to claim 1 or 5, wherein the palette color candidate set adds a new palette color and/or deletes an existing palette color according to a first-in, first-out rule.
The image encoding method according to claim 1, wherein the method further comprises:

For a coded block that is encoded using one of the palette and the pixel string copy mode, part or all of the new palette color of the coded block is added to the shared palette color. Selection.
An image decoding method, the method comprising:

Parsing the video stream to obtain at least one of the following: a copy mode of the palette and the pixel string copy decoding, a copy parameter, and a new palette color;

Generating a palette color candidate set shared by the new palette color and/or color palette and the pixel string copy decoding method, and generating a palette and pixel string copy decoding using the copy mode for the current decoded block. The palette used;

Palette and pixel string copy decoding is performed using the palette of the current decoded block.
The image decoding method according to claim 8, wherein the decoding block is a decoding region of an image, and the decoding block includes at least one of the following: an LCU, a CTU, a CU, a sub-region of a CU, a PU, and a TU.
The image decoding method according to claim 8, wherein a maximum number of palette colors that the maximum space of the palette color candidate set can accommodate is a predetermined constant.
The image decoding method according to claim 8, wherein the method further comprises: the palette color candidate set performing one or more of the following operations according to a predetermined rule:

Update, palette color deletion, palette color position adjustment, palette color addition.
The image decoding method according to claim 8 or 11, wherein the palette color candidate set adds a new palette color and/or deletes an existing palette color according to a first-in, first-out rule.
The image decoding method according to claim 8, wherein the method further comprises:

For a decoded block that is decoded using one of the palette and pixel string copying methods, some or all of the new palette color of the decoded block is added to the shared palette color candidate set.
An image processing device, the device comprising:

a first generating unit, configured to generate a new palette color according to the pixel of the current encoding block when copying and encoding the current encoding block by using one of a color palette and a pixel string copy encoding method;

a second generating unit, configured to generate a palette of the current encoding block according to the palette color candidate set shared by the new palette color and/or color palette and the pixel string copy encoding manner;

And a coding unit configured to perform palette and pixel string copy coding using a palette of the current coded block, and generate a video code stream including a copy mode and a copy parameter.
An image processing device, the device comprising:

a parsing unit configured to parse the video stream to obtain at least one of the following: a copy mode of the palette and the pixel string copy decoding, a copy parameter, and a new palette color;

a third generating unit, configured to perform color matching on the current decoding block by using the copy mode according to the new palette color and/or the palette color candidate set shared by the color palette and the pixel string copy decoding mode The palette used by the board and the pixel string to decode and decode;

A decoding unit configured to perform palette and pixel string copy decoding using a palette of the current decoded block.
A computer storage medium having stored therein computer executable instructions configured to perform the image encoding method and/or image decoding method of any of claims 1-13.