TW202209878A - Image coding method, image decoding method, and related apparatus - Google Patents

Abstract

An image coding method, an image decoding method, and related apparatuses. The image coding method comprises: determining intra-frame prediction filtering indication information of the current coding block; if it is determined, according to the intra-frame prediction filtering indication information, that the current coding block needs to use a first intra-frame prediction filtering mode, setting a first use identification bit of the first intra-frame prediction filtering mode of the current coding block to be allowed-to-use; writing the intra-frame prediction filtering indication information, the first intra-frame prediction filtering mode and the first use identification bit into a code stream; and performing prediction on the current coding block according to the first intra-frame prediction filtering mode to obtain a predicted block, coding the predicted block, and writing a coding result into the code stream. Options for operations, such as smoothing processing or local blurring, required for intra-frame prediction are provided, and for parts of an image texture that do not need to be too sharp, predicted pixels are smoother, and a predicted block is closer to an original image. Therefore, the coding efficiency can be improved.

Description

Image coding method, image decoding method and related device

The present application relates to the technical field of electronic devices, and in particular, to an image encoding method, an image decoding method, and related apparatuses.

Digital imaging capabilities can be incorporated into a wide range of devices, including digital televisions, digital live broadcasting systems, wireless broadcasting systems, personal digital assistants (PDAs), laptop or desktop computers, tablet computers, e-books Readers, digital cameras, digital recording devices, digital media players, video game devices, video game consoles, mobile or satellite radio phones, video conferencing devices, video streaming devices, etc.

Digital video devices implement video compression techniques such as those developed by the Moving Picture Experts Group (MPEG)-2, MPEG-4, ITU-TH.263, ITU-TH.264/MPEG-4 Part 10 Advanced Video Coding advanced video coding (AVC), the standards defined by the ITU-TH .265 high efficiency video coding (HEVC) standard, and those video compression techniques described in extensions of said standards, to be more efficient to transmit and receive digital video messages. Video devices can transmit, receive, encode, decode and/or store digital video information more efficiently by implementing these video codec techniques.

With the proliferation of online video, despite the continuous evolution of digital video compression technology, it still puts forward higher requirements for video compression ratio.

The embodiments of the present application provide an image encoding method, an image decoding method, and a related device, in order to provide options for intra-frame prediction in operations that require smoothing or local blurring, and for parts of image textures that do not need to be too sharp, Using this technique makes the predicted pixels smoother and the predicted block closer to the original image, which will ultimately improve the coding efficiency.

In a first aspect, an embodiment of the present application provides an image encoding method, including:

Divide the image, and determine the intra-frame prediction filtering instruction message of the current coding block, where the intra-frame prediction filtering instruction message includes a first instruction message and a second instruction message, and the first instruction message is used to indicate whether to allow the use of the first frame Intra prediction filtering mode, the second indication message is used to indicate whether to allow the use of the second intra prediction filtering mode, and the first intra prediction filtering mode is an intra prediction filtering IPF mode;

If it is determined according to the intra-frame prediction filtering instruction message that the current coding block needs to use the first intra-frame prediction filtering mode, the first use flag of the first intra-frame prediction filtering mode of the current coding block is set bit set to allow use;

transmitting the intra-prediction filter indication message, the first intra-prediction filter mode, and the first usage flag via a bitstream;

The prediction block of the current coding block and the residual block obtained after inverse transformation and inverse quantization are superimposed to obtain the reconstructed reconstructed block, which is used as the prediction reference block of the next coding block.

Compared with the prior art, the solution of the present application provides options for operations such as smoothing or local blurring for intra-frame prediction. For the part of the image texture that does not need to be too sharp, the use of this technology makes the predicted pixels smoother and the predicted blocks more accurate. Close to the original image, which will eventually improve the encoding efficiency.

In a second aspect, an embodiment of the present application provides an image decoding method, including:

Parse the bit stream, and determine the intra-frame prediction filter indication message and the first usage flag of the current decoding block, where the intra-frame prediction indication message includes a first indication message and a second indication message, and the first indication message is used to indicate whether to allow the use of the first intra-frame prediction filter mode, the second indication message is used to indicate whether to allow the use of the second intra-frame prediction filter mode, the first intra-frame prediction filter mode is the intra-frame prediction filter IPF mode, the The first use flag is the use flag of the first intra-frame prediction filtering mode;

According to the intra-frame prediction filtering instruction message and the first use flag, it is determined that the prediction block of the decoding block is obtained by using the first frame rate prediction filtering mode.

Compared with the prior art, the solution of the present application provides options for operations such as smoothing or local blurring for intra-frame prediction. For the part of the image texture that does not need to be too sharp, the use of this technology makes the predicted pixels smoother and the predicted blocks more accurate. Close to the original image, which will eventually improve the encoding efficiency.

In a third aspect, an embodiment of the present application provides an image encoding apparatus, including:

a dividing unit, configured to divide the image and determine the intra-frame prediction filtering indication information of the current coding block, the intra-frame prediction filtering indication information includes a first indication message and a second indication message, and the first indication message is used to indicate whether The first intra-frame prediction filtering mode is allowed to be used, and the second indication message is used to indicate whether the second intra-frame prediction filtering mode is allowed to be used, and the first intra-frame prediction filtering mode is an intra-frame prediction filtering IPF mode;

a determining unit, configured to set the first intra prediction filtering mode of the current coding block if it is determined according to the intra prediction filtering instruction message that the current coding block needs to use the first intra prediction filtering mode The first use flag is set to allow use;

a transmission unit, configured to transmit the intra-frame prediction filter indication message, the first intra-frame prediction filter mode and the first use identification bit through a bit stream;

and a superimposing unit, configured to determine, according to the intra-frame prediction filtering instruction message and the first usage flag, to use the first frame rate prediction filtering mode to obtain the prediction block of the decoding block.

In a fourth aspect, an embodiment of the present application provides an image decoding apparatus, including:

a parsing unit, configured to determine an intra-frame prediction filter indication message and a first use flag of the current decoding block, where the intra-frame prediction indication message includes a first indication message and a second indication message, and the first indication message is used to indicate whether to allow the use of the first intra-frame prediction filter mode, the second indication message is used to indicate whether to allow the use of the second intra-frame prediction filter mode, the first intra-frame prediction filter mode is the intra-frame prediction filter IPF mode, the The first use flag is the use flag of the first intra-frame prediction filtering mode;

A determining unit, configured to determine, according to the intra-frame prediction filtering instruction message and the first use flag, to obtain the prediction block of the decoding block by using the first frame rate prediction filtering mode.

In a fifth aspect, an embodiment of the present application provides an encoder, including: a processor and a memory coupled to the processor; the processor is configured to execute the method described in the first aspect.

In a sixth aspect, an embodiment of the present application provides a decoder, including: a processor and a memory coupled to the processor; the processor is configured to execute the method described in the second aspect above.

In a seventh aspect, an embodiment of the present application provides a terminal, the terminal includes: one or more processors, a memory, and a communication interface; the memory and the communication interface are connected to the one or more processors ; the terminal communicates with other devices through the communication interface, the memory is used to store computer program code, the computer program code includes instructions, when the one or more processors execute the instructions, the The terminal executes the method according to the first aspect or the second aspect.

In an eighth aspect, an embodiment of the present application provides a computer-readable storage medium, where an instruction is stored in the computer-readable storage medium, and when the instruction is executed on a computer, the computer is made to execute the above-mentioned first aspect or the second aspect the method described.

In a ninth aspect, an embodiment of the present application provides a computer program product including instructions, which, when the instructions are run on a computer, cause the computer to execute the method described in the first aspect or the second aspect.

In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention, but not to limit the present invention.

It will be understood that the terms "first", "second", etc., as used herein, may be used herein to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish a first element from another element. For example, a first user terminal may be referred to as a second user terminal, and similarly, a second user terminal may be referred to as a first user terminal without departing from the scope of the present invention. Both the first client and the second client are clients, but they are not the same client.

First, the terms used in the embodiments of this application are introduced.

For the division of images, in order to represent the video content more flexibly, the High Efficiency Video Coding standard (HEVC) technology defines a coding tree unit (CTU), a coding block (Coding Unit, CU) ), prediction unit (Prediction Unit, PU) and transform unit (Transform Unit, TU). CTU, CU, PU and TU are all image blocks.

Coding tree unit CTU, an image is composed of multiple CTUs, and a CTU usually corresponds to a square image area, including luma pixels and chroma pixels in this image area (or can only contain luma pixels, or also can only contain chroma pixels); the CTU also contains syntax elements that indicate how to divide the CTU into at least one coding block (coding unit, CU), and the method for decoding each coding block to obtain a reconstructed image. As shown in FIG. 1 , an image 1 is composed of a plurality of CTUs (including CTU A, CTU B, CTU C, etc.). The encoded information corresponding to a certain CTU contains the luminance and/or chrominance values of the pixels in the square image area corresponding to the CTU. In addition, the encoded information corresponding to a certain CTU may also include syntax elements indicating how to divide the CTU into at least one CU, and a method of decoding each CU to obtain a reconstructed image. An image area corresponding to one CTU may include 64×64, 128×128 or 256×256 pixels. In one example, a 64x64 pixel CTU contains a rectangular pixel lattice of 64 columns of 64 pixels, each pixel containing a luma component and/or a chrominance component. A CTU can also correspond to a rectangular image area or an image area of other shapes, and an image area corresponding to a CTU can also be an image area with a different number of pixels in the horizontal direction and the number of pixels in the vertical direction, for example, including 64 ×128 pixels.

Coding block CU, as shown in Figure 2, a coding tree unit CTU can be further divided into coding block CU, which usually corresponds to an A×B rectangular area in the image, including A×B luminance pixels or/and its corresponding color. Degree pixel, A is the width of the rectangle, B is the height of the rectangle, A and B can be the same or different, the value of A and B is usually an integer power of 2, such as 128, 64, 32, 16, 8, 4 . The width involved in the embodiments of the present application refers to the length along the X-axis direction (horizontal direction) in the two-dimensional rectangular coordinate system XoY shown in FIG. 1 , and the height refers to the two-dimensional rectangular coordinate system XoY shown in FIG. 1 . The length along the Y-axis direction (vertical direction) in the middle. The reconstructed image of a CU can be obtained by adding the predicted image and the residual image. The predicted image is generated through intra-frame prediction or inter-frame prediction, and can be specifically composed of one or more prediction blocks (PB), The residual image is generated by performing inverse quantization and inverse transform processing on the transform coefficients, and may be specifically composed of one or more transform blocks (transform blocks, TB). Specifically, a CU contains encoded information, and the encoded information includes information such as prediction mode and transform coefficients. According to the encoded information, the CU is subjected to corresponding decoding processes such as prediction, inverse quantization, and inverse transformation, and a reconstructed image corresponding to the CU is generated. The relationship between the coding tree unit and the coding block is shown in Figure 3.

The prediction unit PU is the basic unit of intra prediction and inter prediction. The motion information that defines the image block includes inter prediction direction, reference frame, motion vector, etc. The image block that is being encoded is called the current coding block (CCB), and the image block that is being decoded is called the current coding block (CCB). is the current decoding block (CDB), for example, when prediction processing is being performed on an image block, the current coding block or the current decoding block is the prediction block; when residual processing is being performed on an image block, the current coding block Or the current decoding block is a transform block. The picture in which the current coding block or the current decoding block is located is called the current frame. In the current frame, the image blocks located on the left or upper side of the current block (based on the coordinate system in Figure 1, the left side refers to the negative direction of the X-axis, and the upper side refers to the positive direction of the Y-axis) may be located in the current frame and have been encoded. /Decoding process, the reconstructed images are obtained, which are called reconstructed blocks; information such as coding mode and reconstructed pixels of the reconstructed blocks are available. A frame that has completed encoding/decoding before the current frame is encoded/decoded is called a reconstructed frame. When the current frame is a unidirectional prediction frame (P frame) or a bidirectional prediction frame (B frame), it has one or two reference frame lists respectively, and the two lists are called L0 and L1 respectively, and each list contains at least one reconstructed frame. frame, referred to as the reference frame for the current frame. The reference frame provides reference pixels for inter prediction of the current frame.

The transform unit TU processes the residuals of the original image block and the predicted image block.

Pixels (also known as pixel points) refer to pixels in an image, such as pixels in coding blocks, pixels in luminance component pixel blocks (also known as luminance pixels), and pixels in chrominance component pixel blocks points (aka chroma pixels), etc.

The sample (also known as pixel value, sample value) refers to the pixel value of a pixel point. The pixel value in the luminance component domain specifically refers to the brightness (ie grayscale value), and the pixel value in the chrominance component domain specifically refers to the color. The degree value (ie color and saturation), according to the different processing stages, the sample of a pixel specifically includes the original sample, the predicted sample and the reconstructed sample.

Direction description: horizontal direction, for example: along the X-axis direction and vertical direction in the two-dimensional rectangular coordinate system XoY as shown in Figure 1, for example: the negative direction along the Y-axis in the two-dimensional rectangular coordinate system XoY shown in Figure 1 .

In intra-frame prediction, a predicted image of the current block is generated according to the spatially adjacent pixels of the current block. An intra prediction mode corresponds to a method of generating a predicted image. The division of the intra prediction unit includes 2N×2N division and N×N division. The 2N×2N division method is to not divide the image block; the N×N division method is to divide the image block into four equal-sized blocks. sub-image block.

Generally, the digital image compression technology works on image sequences whose color coding method is YCbCr, also called YUV, and whose color format is 4:2:0, 4:2:2 or 4:4:4. Among them, Y represents the brightness (Luminance or Luma), that is, the grayscale value, Cb represents the blue chrominance component, Cr represents the red chrominance component, and U and V represent the chrominance (Chrominance or Chroma), which is used to describe color and saturation. Spend. In the color format, 4:2:0 means that every 4 pixels has 4 luminance components and 2 chrominance components (YYYYCbCr), and 4:2:2 means that every 4 pixels has 4 luminance components and 4 chrominance components. component (YYYYCbCrCbCr), and 4:4:4 represents full pixel display (YYYYCbCrCbCrCbCrCbCr), Figure 3 shows the distribution of each component in different color formats, where the circle is the Y component and the triangle is the UV component.

The intra-frame prediction part in the digital image coding and decoding mainly refers to the image information of the adjacent blocks of the current frame to predict the current coding unit block, calculates the residual between the predicted block and the original image block to obtain the residual information, and then transforms it with the original image block. Quantization and other processes are used to transmit the residual information to the decoding end. After the decoding end receives and parses the bit stream, the residual information is obtained through the steps of inverse transformation and inverse quantization, and the reconstructed image block is obtained by superimposing the residual information on the predicted image block predicted by the decoding end. In this process, intra prediction usually uses the respective angle mode and non-angle mode to predict the current coding block to obtain the prediction block, and according to the bit rate distortion information calculated from the prediction block and the original block, the optimal current coding unit The prediction mode is then transmitted to the decoding end through the bit stream. The decoding end parses the prediction mode, predicts the prediction image of the current decoding block, and superimposes the residual pixels transmitted through the bit stream to obtain the reconstructed image.

After the development of digital image coding and decoding standards over the past generations, the non-angle mode remains relatively stable, with average mode and flat mode; the angle mode continues to increase with the evolution of digital image coding standards. Take the international digital image coding standard H series as an example , the H.264/AVC standard has only 8 angle prediction modes and 1 non-angle prediction mode; H.265/HEVC is extended to 33 angle prediction modes and 2 non-angle prediction modes; and the latest general image coding standard H .266/VVC adopts 67 prediction modes, among which 2 non-angle prediction modes are reserved, and the angle mode is expanded from 33 kinds of H.265 to 65 kinds. Undoubtedly, with the increase of the angle mode, the intra-frame prediction will be more accurate, and it will be more in line with the needs of the current society for the development of high-definition and ultra-high-definition images. Not only the international standard, but the domestic digital audio and video coding standard AVS3 also continues to expand the angle mode and non-angle mode. The development of ultra-high-definition digital video has put forward higher requirements for intra-frame prediction, and cannot just rely on simply increasing the angle prediction mode. , extending the wide angle to commit coding efficiency. Therefore, the domestic digital audio and video coding standard AVS3 adopts the intra prediction filter technology (IPF, intra prediction filter). The correlation between the current coding units, the intra-frame prediction filtering technology improves the pixel prediction accuracy through point-to-point filtering, which can effectively enhance the spatial correlation, thereby improving the intra-frame prediction accuracy. The IPF technology takes the prediction mode from top right to bottom left in AVS3 as an example, as shown in Figure 4, where URB represents the boundary pixel of the adjacent block on the left side close to the current coding unit, and MRB represents the adjacent block on the upper side close to the current coding unit. The boundary pixels of , filter direction indicates the filtering direction. The direction of the prediction mode is from upper right to lower left, and the generated prediction value of the current coding unit mainly uses the reference pixels of the adjacent blocks in the upper MRB row, that is, the prediction pixels of the current coding unit do not refer to the reconstructed pixels of the adjacent blocks on the left. The current coding unit and the left reconstruction block are in a spatially adjacent relationship. If only the upper MRB pixels are referenced instead of the left URB pixels, the spatial correlation is likely to be lost, resulting in poor prediction effect.

IPF technology is applied to all prediction modes of intra-frame prediction, and is a filtering method to improve the accuracy of intra-frame prediction. IPF technology is mainly realized through the following processes:

a) The IPF technology judges the current prediction mode of the coding unit and divides it into horizontal angle prediction mode, vertical angle prediction mode and non-angle prediction mode;

b) According to different types of prediction modes, IPF technology uses different filters to filter the input pixels;

c) According to the different distances from the current pixel to the reference pixel, the IPF technology uses different filter coefficients to filter the input pixels;

The input pixels of the IPF technology are the predicted pixels obtained in each prediction mode, and the output pixels are the final predicted pixels after IPF.

The IPF technology has an allowable flag bit ipf_enable_flag, a binary variable. The value of '1' indicates that intra-frame prediction filtering can be used; the value of '0' indicates that intra-frame prediction filtering should not be used. At the same time, the IPF technology also uses the flag ipf_flag, a binary variable. The value of '1' indicates that intra-frame prediction filtering should be used; the value of '0' indicates that intra-frame prediction filtering should not be used. If there is no flag bit in the bit stream ipf_flag, the default value is 0.

The syntax element IPF_flag is as follows: ... if (IpfEnableFlag && (PartSize == 'SIZE_2Mx2N') && (! IsPcmMode[0])) { ipf_flag ae(v) } ...

The above-mentioned IPF technique classifies prediction modes 0, 1 and 2 as non-angular prediction modes, and uses the first three-tap filter to filter the predicted pixels;

Classify prediction mode 3 to prediction mode 18 and prediction mode 34 to prediction mode 50 as vertical class angle prediction modes, and use the first two-tap filter to filter the prediction pixels;

Prediction mode 19 to prediction mode 32, prediction mode 51 to prediction mode 65 are classified as horizontal-like angle prediction modes, and the prediction pixels are filtered using a second two-tap filter.

The above-mentioned first three-tap filter suitable for IPF technology, the filtering formula is as follows:

The above-mentioned first two-tap filter suitable for IPF technology, the filtering formula is as follows:

The above-mentioned second two-tap filter suitable for IPF technology, the filtering formula is as follows:

In the above formula, P'(x, y) is the final prediction value of the pixel located at the position (x, y) of the current chrominance prediction block, and f(x) and f(y) are the pixel values reconstructed with reference to the adjacent blocks on the left, respectively. The horizontal filter coefficient and the vertical filter coefficient of the reconstructed pixel of the adjacent block on the reference upper side, P(-1, y) and P(x,-1) are the left reconstruction pixel located in the y row and the upper reconstructed pixel located in the x column, P(-1, y) and P(x,-1) (x, y) is the original predicted pixel value in the current chrominance component prediction block. The values of x and y do not exceed the range of the width and height of the current coding unit block.

The values of the above-mentioned horizontal filter coefficients and vertical filter coefficients are related to the size of the current coding unit block and the distances from the predicted pixels in the current prediction block to the left reconstructed pixels and the top reconstructed pixels. The values of the above-mentioned horizontal filter coefficients and vertical filter coefficients are also related to the size of the current coding block, and are divided into different filter coefficient groups according to the size of the current coding unit block.

Table 1 presents the filter coefficients of the IPF technique.

Table 1 Intra-frame chroma prediction filter coefficients coding unit block size 4 8 16 32 64 The distance between the predicted sample and the reconstructed sample 1 3/8 11/16 5/8 9/16 13/16 2 1/14 25/64 27/64 27/64 11/16 3 1/32 7/32 19/64 21/64 37/16 4 0 1/8 13/64 1/4 31/16 5 0 1/16 9/64 3/16 13/32 6 0 1/32 1/14 9/64 11/32 7 0 1/64 1/16 7/64 9/32 8 0 1/64 3/64 5/64 15/64 9 0 0 1/32 1/16 13/64 10 0 0 1/64 3/64 11/64

5 shows schematic diagrams of three filtering situations for intra-frame prediction filtering, respectively only referring to the upper reference pixel to filter the predicted value in the current coding unit; only referring to the left reference pixel to filter the predicted value in the current coding unit; And both refer to the upper and left reference pixels to filter the prediction value in the current coding unit block.

FIG. 6 is a block diagram of an example image decoding system 1 described in the embodiments of the present application. As used herein, the term "video decoder" generally refers to both a video encoder and a video decoder. In this application, the term "image decoding" or "decoding" may generally refer to image encoding or image decoding. The video encoder 100 and the video decoder 200 of the video decoding system 1 are used to implement the cross-component prediction method proposed in this application.

As shown in FIG. 6 , the video decoding system 1 includes a source device 10 and a destination device 20 . Source device 10 generates encoded video data. Accordingly, the source device 10 may be referred to as an image encoding device. Destination device 20 may decode the encoded image data generated by source device 10 . Therefore, the destination device 20 may be referred to as an image decoding device. Various implementations of source device 10, destination device 20, or both may include one or more processors and memory coupled to the one or more processors. The memory may include, but is not limited to, RAM, ROM, EEPROM, flash memory, or any other medium that can be used to store the desired code in the form of instructions or data structures that can be accessed by a computer, as described herein.

Source device 10 and destination device 20 may include a variety of devices, including desktop computers, mobile computing devices, notebook (eg, laptop) computers, tablet computers, set-top boxes, telephone handhelds such as so-called "smart" phones, etc. computer, television, camera, display device, digital media player, video game console, car computer or the like.

Destination device 20 may receive encoded image data from source device 10 via link 30 . Link 30 may include one or more media or devices capable of moving encoded image data from source device 10 to destination device 20 . In one example, link 30 may include one or more communication media that enable source device 10 to transmit encoded image data directly to destination device 20 in real time. In this example, source device 10 may modulate the encoded image data according to a communication standard, such as a wireless communication protocol, and may transmit the modulated image data to destination device 20 . The one or more communication media may include wireless and/or wired communication media, such as radio frequency (RF) spectrum or one or more physical transmission lines. The one or more communication media may form part of a packet-based network, such as a local area network, a wide area network, or a global network (eg, the Internet). The one or more communication media may include routers, switches, base stations, or other equipment that facilitates communication from source device 10 to destination device 20 . In another example, the encoded data may be output from output interface 140 to storage device 40 .

The image encoding and decoding techniques of the present application can be applied to image encoding and decoding to support a variety of multimedia applications, such as over-the-air television broadcasting, cable television transmission, satellite television transmission, streaming image transmission (eg, via the Internet), storage in Encoding of image data on a data storage medium, decoding of image data stored on a data storage medium, or other applications. In some examples, the video decoding system 1 may be used to support one-way or two-way video transmission to support applications such as video data streaming, video playback, video broadcasting, and/or video telephony.

The image decoding system 1 illustrated in FIG. 6 is merely an example, and the techniques of this application may be applicable to image decoding setups (eg, image encoding or image decoding) that do not necessarily include any data communication between an encoding device and a decoding device. In other instances, data is retrieved from a local repository, data is streamed over a network, and so on. The image encoding device may encode the data and store the data to memory, and/or the image decoding device may retrieve the data from the memory and decode the data. In many instances, encoding and decoding is performed by devices that do not communicate with each other but only encode data to and/or retrieve data from memory and decode the data.

In the example of FIG. 6 , source device 10 includes video source 120 , video encoder 100 and output interface 140 . In some examples, output interface 140 may include a modulator/demodulator (modem) and/or a transmitter. Image source 120 may include an image capture device (eg, a camera), an image archive containing previously captured image data, an image feed interface for receiving image data from image content providers, and/or a computer for generating image data A graphics system, or a combination of such sources of image data.

Image encoder 100 may encode image data from image source 120 . In some examples, source device 10 transmits the encoded image data directly to destination device 20 via output interface 140 . In other examples, the encoded image data may also be stored on storage device 40 for later access by destination device 20 for decoding and/or playback.

In the example of FIG. 6 , destination device 20 includes input interface 240 , video decoder 200 and display device 220 . In some examples, input interface 240 includes a receiver and/or modem. Input interface 240 may receive encoded image data via link 30 and/or from storage device 40 . The display device 220 may be integrated with the destination device 20 or may be external to the destination device 20 . Generally, display device 220 displays decoded image data. The display device 220 may include various display devices, such as a liquid crystal display (LCD), a plasma display, an organic light emitting diode (OLED) display, or other types of display devices.

Although not shown in FIG. 6, in some aspects video encoder 100 and video decoder 200 may each be integrated with an audio encoder and decoder, and may include appropriate multiplexer-demultiplexer units or other Hardware and software to handle the encoding of both audio and video in a common data flow or separate data flows.

Image encoder 100 and image decoder 200 may each be implemented as any of a variety of circuits such as one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), Field Programmable Gate Array (FPGA), discrete logic, hardware, or any combination thereof. If the application is implemented in part in software, a device may store instructions for the software in a suitable non-volatile computer-readable storage medium and execute the instructions in hardware using one or more processors Thus, the technology of the present application is implemented. Any of the foregoing (including hardware, software, a combination of hardware and software, etc.) may be considered to be one or more processors. Each of image encoder 100 and image decoder 200 may be included in one or more encoders or decoders, either of which may be integrated as a combined encoder in the respective device /decoder (codec) part.

FIG. 7 is an exemplary block diagram of an image encoder 100 described in an embodiment of the present application. The video encoder 100 is used to output the video to the post-processing entity 41 . Post-processing entity 41 represents an example of an image entity that can process encoded image data from image encoder 100, such as a media aware network element (MANE) or a stitching/editing device. In some cases, post-processing entity 41 may be an instance of a network entity. In some image encoding systems, post-processing entity 41 and image encoder 100 may be parts of separate devices, while in other cases, the functionality described with respect to post-processing entity 41 may be implemented by the same device that includes image encoder 100 implement. In an example, post-processing entity 41 is an example of storage device 40 of FIG. 1 .

In the example of FIG. 7 , image encoder 100 includes prediction processing unit 108 , filter unit 106 , memory 107 , summer 112 , transformer 101 , quantizer 102 , and entropy encoder 103 . The prediction processing unit 108 includes an inter predictor 110 and an intra predictor 109 . For image block reconstruction, the image encoder 100 further includes an inverse quantizer 104 , an inverse transformer 105 and a summer 111 . Filter unit 106 represents one or more in-loop filters, such as deblocking filters, adaptive in-loop filters (ALF), and sample adaptive offset (SAO) filters. Although filter unit 106 is shown in FIG. 7 as an in-loop filter, in other implementations, filter unit 106 may be implemented as a post-loop filter. In an example, the image encoder 100 may further include an image data memory and a division unit (not shown in the figure).

The image encoder 100 receives image data and stores the image data in an image data memory. The dividing unit divides the image data into several image blocks, and these image blocks can be further divided into smaller blocks, such as image block division based on a quad-tree structure or a binary tree structure. Prediction processing unit 108 may select one of multiple possible decoding modes for the current image block, such as one of multiple intra decoding modes or one of multiple inter decoding modes. Prediction processing unit 108 may provide the resulting intra, inter decoded blocks to summer 112 to generate a residual block, and to summer 111 to reconstruct an encoded block for use as a reference image. Intra-predictor 109 within prediction processing unit 108 may perform intra-predictive encoding of the current image block relative to one or more adjacent encoded blocks in the same frame or slice as the current block to be encoded to remove spatial redundancy. Inter predictor 110 within prediction processing unit 108 may perform inter-predictive encoding of the current image block relative to one or more prediction blocks in one or more reference images to remove temporal redundancy. Prediction processing unit 108 provides information indicating the selected intra or inter prediction mode for the current image block to entropy encoder 103 for entropy encoder 103 to encode the information indicating the selected inter prediction mode.

After the prediction processing unit 108 generates the prediction block of the current image block via inter prediction/intra prediction, the image encoder 100 forms a residual image block by subtracting the prediction block from the current image block to be encoded. Summer 112 represents one or more components that perform this subtraction operation. Residual image data in the residual block may be included in one or more TUs and applied to transformer 101 . Transformer 101 transforms the residual image data into residual transform coefficients using a transform such as a discrete cosine transform (DCT) or a conceptually similar transform. Transformer 101 may convert residual image data from a pixel value domain to a transform domain, such as a frequency domain.

Transformer 101 may send the resulting transform coefficients to quantizer 102 . Quantizer 102 quantizes the transform coefficients to further reduce the bit rate. In some examples, quantizer 102 may then perform a scan of the matrix including the quantized transform coefficients. Alternatively, the entropy encoder 103 may perform scanning.

After quantization, the entropy encoder 103 entropy encodes the quantized transform coefficients. For example, entropy encoder 103 may perform Context Adaptive Variable Length Coding (CAVLC), Adaptive Binary Arithmetic Coding (CABAC) referenced above, Syntax-Based Adaptive Binary Arithmetic Coding (SBAC) above, Probability Interval Partitioning Entropy (PIPE) coding or another entropy coding method or technique. After entropy encoding by entropy encoder 103 , the encoded bitstream may be transmitted to image decoder 200 , or archived for later transmission or retrieval by image decoder 200 . The entropy encoder 103 may also entropy encode the syntax elements of the current image block to be encoded.

Inverse quantizer 104 and inverse transformer 105 apply inverse quantization and inverse transform, respectively, to reconstruct the residual block in the pixel domain, eg, for later use as a reference block for a reference image. Summer 111 adds the reconstructed residual block to the prediction block produced by inter predictor 110 or intra predictor 109 to produce a reconstructed image block. Filter unit 106 may be applied to reconstructed image blocks to reduce distortions, such as block artifacts. The reconstructed image block is then stored in the memory 107 as a reference block, which can be used by the inter-predictor 110 as a reference block for inter-predicting blocks in subsequent image frames or images.

The image encoder 100 divides the input image into a number of coding tree units, and each coding tree unit is further divided into a number of rectangular or square coding blocks. When the current coding block selects the intra-frame prediction mode for coding, the luminance component of the current coding block is subjected to calculation traversal of several prediction modes, and the optimal prediction mode is selected according to the bit rate distortion cost, and the chrominance components of the current coding block are The computation of several prediction modes traverses and selects the optimal prediction mode according to the bit rate distortion cost. After that, the residual between the original image block and the predicted block is calculated. The residual is subsequently transformed, quantized, and entropy encoded to form an output bit stream, and the other is subjected to inverse transformation, inverse quantization, and loop filtering to form a reconstructed sample. As reference information for subsequent image compression.

The current specific implementation of the IPF technology in the video encoder 100 is as follows.

The input digital image information is divided into a number of coding tree units at the coding end, and each coding tree unit is further divided into a number of rectangular or square coding units, and each coding unit performs an intra-frame prediction process to calculate a prediction block.

In the current coding unit,

①If the allowable flag bit of IPF is '1', perform all the following steps;

② If the allowable flag bit of IPF is '0', only steps a1), b1), f1 and g1) are performed.

a1) Intra prediction first traverses all prediction modes, calculates the predicted pixels in each intra prediction mode, and calculates the bit rate distortion cost according to the original pixels;

b1) According to the principle of minimum rate distortion cost of all the above prediction modes, select the optimal prediction mode of the current coding unit, and record the optimal prediction mode information and the corresponding bit rate distortion cost information;

c1) Traverse all intra-frame prediction modes again. This process starts the IPF technology, first calculates the predicted pixels in each intra-frame prediction mode, and obtains the prediction block of the current coding unit;

d1) Perform IPF on the prediction block of the current coding unit, select the corresponding filter according to the current prediction mode, and select the corresponding filter coefficient group according to the size of the current coding unit. The specific correspondence can be found in Table 1;

e1) Calculate the bit rate distortion cost information of each prediction mode according to the final predicted pixels and original pixels obtained by the IPF technology, and record the prediction mode and corresponding cost value of the minimum bit rate distortion cost information;

f1) If the IPF allows the flag bit to be '0', then transmit the prediction mode index recorded in b1) to the decoding end via the bit stream;

If the IPF allows the flag bit to be '1', then compare the minimum cost value recorded in b1) with the minimum cost value recorded in e1),

If the bit rate distortion cost in b1) is smaller, the prediction mode index code recorded in b1) is transmitted to the decoding end as the optimal prediction mode of the current coding unit through the bit stream, and the current coding unit identification position of the IPF is used If the flag position is no, it means that IPF technology is not used, and it is also transmitted to the decoding end through the bit stream;

If the bit rate distortion in e1) is smaller, use the prediction mode index code recorded in e1) as the optimal prediction mode of the current coding unit and transmit it to the decoding end via the bit stream, and use the IPF current coding unit identification position use flag The position is true, indicating that the IPF technology is used, and it is also transmitted to the decoding end through the bit stream.

g) After that, superimpose the predicted value and the residual information after transformation and quantization to obtain the reconstructed block of the current coding unit as the reference information of the subsequent coding unit.

Intra predictor 109 may also provide information indicating the selected intra prediction mode for the current coding block to entropy encoder 103 so that entropy encoder 103 encodes the information indicating the selected intra prediction mode.

FIG. 8 is an exemplary block diagram of an image decoder 200 described in an embodiment of the present application. In the example of FIG. 8 , image decoder 200 includes entropy decoder 203 , prediction processing unit 208 , inverse quantizer 204 , inverse transformer 205 , summer 211 , filter unit 206 , and memory 207 . Prediction processing unit 208 may include inter predictor 210 and intra predictor 209 . In some examples, image decoder 200 may perform a decoding process that is substantially the inverse of the encoding process described with respect to image encoder 100 from FIG. 7 .

During the decoding process, image decoder 200 receives from image encoder 100 an encoded image bitstream representing image blocks of an encoded image slice and associated syntax elements. The image decoder 200 may receive image data from the network entity 42, and optionally, may also store the image data in an image data memory (not shown in the figure). Image data memory may store image data, such as an encoded image bitstream, to be decoded by the components of image decoder 200 . The image data stored in the image data memory can be obtained, for example, from the storage device 40, from a local image source such as a camera, through wired or wireless network communication of the image data, or by accessing a physical data storage medium. The video data memory may act as a decoded picture buffer (CPB) for storing encoded video data from the encoded video bitstream.

Network entity 42 may be, for example, a server, a MANE, a video editor/splicer, or other such device for implementing one or more of the techniques described above. Network entity 42 may or may not include a video encoder, such as video encoder 100 . Before network entity 42 sends the encoded video bitstream to video decoder 200, network entity 42 may implement portions of the techniques described in this application. In some video decoding systems, network entity 42 and video decoder 200 may be part of separate devices, while in other cases, functionality described with respect to network entity 42 may be performed by the same device that includes video decoder 200 .

Entropy decoder 203 of image decoder 200 entropy decodes the bitstream to generate quantized coefficients and some syntax elements. Entropy decoder 203 forwards the syntax elements to prediction processing unit 208 . Image decoder 200 may receive syntax elements at the image slice level and/or the image block level. When an image slice is decoded as an intra-decoded (I) slice, the intra-predictor 209 of the prediction processing unit 208 is based on the signaled intra-prediction mode and the data to generate prediction blocks for image blocks of the current image slice. When a picture slice is decoded as an inter-decoded (ie, B or P) slice, the inter predictor 210 of the prediction processing unit 208 may determine, based on the syntax elements received from the entropy decoder 203, which method to use for the current The inter prediction mode in which the current image block of the video slice is decoded, based on the determined inter prediction mode, the current image block is decoded (eg, inter prediction is performed).

Inverse quantizer 204 inverse quantizes, ie dequantizes, the quantized transform coefficients provided in the bitstream and decoded by entropy decoder 203 . The inverse quantization process may include using quantization parameters calculated by the image encoder 100 for each image block in the image slice to determine the degree of quantization that should be applied, and likewise the degree of inverse quantization that should be applied. The inverse transformer 205 applies an inverse transform to the transform coefficients, such as an inverse DCT, an inverse integer transform, or a conceptually similar inverse transform process, to produce a residual block in the pixel domain.

After the inter predictor 210 generates a prediction block for the current image block or a sub-block of the current image block, the image decoder 200 combines the residual block from the inverse transformer 205 with the prediction block generated by the inter predictor 210 The corresponding predicted blocks are summed to obtain a reconstructed block, ie a decoded image block. Summer 211 represents the component that performs this summation operation. In-loop filters may also be used (in the decoding loop or after the decoding loop) to smooth pixel transitions or otherwise improve image quality when desired. Filter unit 206 may represent one or more in-loop filters, such as a deblocking filter, an adaptive in-loop filter (ALF), and a sample adaptive offset (SAO) filter. Although filter unit 206 is shown in FIG. 8 as an in-loop filter, in other implementations filter unit 206 may be implemented as a post-loop filter.

The image decoding method specifically performed by the image decoder 200 includes: after the input bit stream is parsed, inversely transformed and inversely quantized, the prediction mode index of the current coding block is obtained. If the prediction mode index of the chrominance component of the current coding block is the enhanced two-step cross-component prediction mode, select only the reconstructed samples from the adjacent pixels on the upper side or the left side of the current coding block according to the index value to calculate the linear model. The linear model calculates the reference prediction block of the chrominance component of the current coding block, down-sampling, and performs prediction and correction based on the correlation of the adjacent pixels of the border in the orthogonal direction for the down-sampled prediction block to obtain the final chrominance component. the final prediction block. One of the subsequent bit streams is used as reference information for subsequent image decoding, and one of the subsequent bit streams is post-filtered to output an image signal.

At present, the specific implementation of the IPF technology at the video decoder 200 is as follows.

The decoding end obtains the bit stream and parses to obtain the digital image sequence information, and parses to obtain the IPF permission flag of the current image sequence, the coding mode of the current decoding unit is the intra-frame prediction coding mode, and the IPF usage flag of the current decoding unit.

In the current decoding unit,

①If the allowable flag bit of IPF is '1', perform all the following steps;

②If the allowable flag bit of IPF is '0', only steps a2), b2) and e2) are performed:

a2) Obtain the bit stream information, analyze the residual information of the current decoding unit, and obtain the time domain residual information through the inverse transformation and inverse quantization process;

b2) Parse the bit stream and obtain the prediction mode index of the current decoding unit, and calculate the prediction block of the current decoding unit according to the adjacent reconstruction block and prediction mode index;

c2) Parse and obtain the usage flag of IPF, if the usage flag of IPF is '0', no additional operation is performed on the current prediction block; if the usage flag of IPF is '1', execute d2);

d2) Select the corresponding filter according to the prediction mode classification information of the current decoding unit, select the corresponding filter coefficient group according to the size of the current decoding unit, and then filter each pixel in the prediction block to obtain the final prediction block;

e2) The reconstructed block of the current decoding unit is obtained by superimposing the restored residual information of the prediction block, and output after post-processing;

It should be understood that other structural variations of the video decoder 200 may be used to decode the encoded video bitstream. For example, the image decoder 200 may generate an output image stream without being processed by the filter unit 206; or, for some image blocks or image frames, the entropy decoder 203 of the image decoder 200 does not decode quantized coefficients, Accordingly, processing by inverse quantizer 204 and inverse transformer 205 is not required.

Among the above intra-frame prediction technologies, the existing IPF technology can effectively improve the coding efficiency of intra-frame prediction, greatly enhance the spatial correlation of intra-frame prediction, and solve the problem of using only a single reference pixel row or column in the intra-frame prediction process. , while ignoring the effect of some pixels on the predicted value. However, when the intra-frame prediction process needs to be smoothed, neither the IPF technology nor the current intra-frame prediction mode can solve similar problems well. Unable to solve the smoothing problem inside the prediction block.

A prediction block calculated according to a single prediction mode usually shows a better prediction effect in an image with a clearer texture, and the residual error will therefore become smaller and less, and the coding efficiency will be improved. However, in image blocks with blurred texture, over-sharpened prediction may lead to increased and larger residual errors, poor prediction effect and reduced coding efficiency.

In view of the above problems, the embodiments of the present application propose an IPF technology based on smoothing processing for some image blocks that need smoothing processing, and directly filter the predicted blocks obtained according to the intra-frame prediction mode.

FIG. 9 is a schematic flowchart of an image encoding method in an embodiment of the present application, and the image encoding method may be applied to the source device 10 in the image decoding system 1 shown in FIG. 6 or the image encoder 100 shown in FIG. 7 . The flow shown in FIG. 9 is described by taking the execution subject as the video encoder 100 shown in FIG. 7 as an example. As shown in FIG. 9 , the cross-component prediction method provided by the embodiment of the present application includes:

Step S110: Divide the image, and determine the intra-frame prediction filtering instruction message of the current coding block, where the intra-frame prediction filtering instruction message includes a first instruction message and a second instruction message, and the first instruction message is used to indicate whether to allow the use of a first intra-frame prediction filter mode, the second indication message is used to indicate whether to allow the use of a second intra-frame prediction filter mode, and the first intra-frame prediction filter mode is an intra-frame prediction filter IPF mode;

Step S120, if it is determined according to the intra-frame prediction filtering instruction message that the current coding block needs to use the first intra-frame prediction filtering mode, then use the first intra-frame prediction filtering mode of the current coding block. - The use flag is set to allow use;

Step S130, transmitting the intra-frame prediction filtering instruction message, the first intra-frame prediction filtering mode, and the first usage flag through a bit stream;

Step S140, superimposing the prediction block of the current coding block and the residual block obtained after inverse transformation and inverse quantization to obtain a reconstructed reconstructed block, which is used as a prediction reference block of the next coding block.

The specific implementation of the technical solution 1 in the intra-frame prediction part of the coding end is as follows:

The encoder obtains the encoded information, including the intra-frame prediction filtering allowable flag and the intra-frame prediction smoothing filter (hereinafter referred to as IPS) allowable flag of the technical solution, etc. After obtaining the image information, the image is divided into several CTUs, and further It is divided into several CUs, and each independent CU performs intra-frame prediction;

During intra prediction,

①If the IPF allowable flag bit and the IPS allowable flag bit are both '1', perform all the following steps;

②If the IPF allowable flag bit is '1' and the IPS allowable flag bit is '0', only execute a3), b3), c3), d3), e3), and i3), j3);

③ If the IPF allowable flag bit is '0', and the IPS allowable flag bit is '1', and the current CU area is greater than or equal to 64 and less than 4096, only execute a3), b3), f3), g3), h3), and i3), j3);

④ If both the IPF allow flag bit and the IPS allow flag bit are '0', only execute a3), b3) and i3), j3):

a3) The current coding unit traverses all intra prediction modes, calculates the prediction block in each prediction mode, and calculates the bit rate distortion cost information of the current prediction mode according to the original pixel block;

b3) According to the principle of minimum rate distortion cost of all prediction modes above, select the optimal prediction mode of the current coding unit, and record the optimal prediction mode information and the corresponding bit rate distortion cost information;

c3) Perform a second traversal of all intra-frame prediction modes. This process enables the IPF technology. First, the predicted pixels in each intra-frame prediction mode are calculated to obtain the prediction block of the current coding unit;

d3) Perform IPF filtering on the prediction block of the current coding unit, select the corresponding filter according to the current prediction mode, and select the corresponding filter coefficient group according to the size of the current coding unit. The specific correspondence can be found in Table 1;

e3) Calculate the bit rate distortion cost information of each prediction mode according to the final predicted pixels and original pixels obtained by the IPF technology, and record the prediction mode and corresponding cost value of the minimum bit rate distortion cost information;

f3) Perform a third traversal of all intra-frame prediction modes. This process enables IPS technology. First, the predicted pixels in each intra-frame prediction mode are calculated to obtain the prediction block of the current coding unit;

g3) Perform two IPS on the prediction block of the current coding unit to obtain the final prediction block;

h3) Calculate the bit rate distortion cost information of each prediction mode according to the final predicted pixels and original pixels obtained by the above IPS technology, and record the prediction mode and corresponding cost value of the minimum bit rate distortion cost information;

i3) If the IPF allowable flag bit is '0' and the IPS allowable flag bit is '0', then the prediction mode index recorded in b) is transmitted to the decoding end via the bit stream;

If the IPF allowable flag bit is '1' and the IPS allowable flag bit is '0', then compare the minimum cost value recorded in b) with the minimum cost value recorded in e),

If the bit rate distortion cost in b3) is smaller, the prediction mode index code recorded in b3) is used as the optimal prediction mode of the current coding unit and transmitted to the decoding end via the bit stream, and the current coding unit of IPF uses the flag position '0', indicating that IPF technology is not used, and it is also transmitted to the decoding end through the bit stream;

If the bit rate distortion in e3) is smaller, then use the prediction mode index code recorded in e3) as the optimal prediction mode of the current coding unit and transmit it to the decoding end via the bit stream, and use the IPF current coding unit to use the flag position' 1', indicating that IPF technology is used, and it is also transmitted to the decoding end through the bit stream;

If the IPF allowable flag bit is '0' and the IPS allowable flag bit is '1', then compare the minimum cost value recorded in b) with the minimum cost value recorded in h),

If the bit rate distortion cost in b3) is smaller, the prediction mode index code recorded in b3) is transmitted to the decoding end via the bit stream as the optimal prediction mode of the current coding unit, and the IPS usage flag of the current coding unit is The position is '0', which means that the technology is not used, and it is also transmitted to the decoding end through the bit stream;

If the bit rate distortion in h3) is smaller, then use the prediction mode index code recorded in h3) as the optimal prediction mode of the current coding unit and transmit it to the decoding end via the bit stream, and use the IPS flag position of the current coding unit '1', indicating that this technology is used, and it is also transmitted to the decoding end through the bit stream;

If the IPF allow flag bit is '1' and the IPS allow flag bit is '1', then compare the minimum cost values recorded in b3), e3) and h3),

If the bit rate distortion cost in b3) is smaller, the prediction mode index encoding recorded in b) is transmitted to the decoding end via the bit stream as the optimal prediction mode of the current coding unit, and the IPS usage flag of the current coding unit is used. Bit and IPF use flag position '0', indicating that neither is used, and is also transmitted to the decoding end through the bit stream;

If the bit rate distortion in e3) is smaller, then use the prediction mode index encoding recorded in e) as the optimal prediction mode of the current coding unit and transmit it to the decoding end via the bit stream, and use the IPF use marker position of the current coding unit '1' and do not transmit the IPS identification bit, indicating that the IPF technology is used instead of the IPS technology, and it is also transmitted to the decoding end through the bit stream;

If the bit rate distortion in h3) is smaller, then use the prediction mode index encoding recorded in h) as the optimal prediction mode of the current coding unit and transmit it to the decoding end via the bit stream, and use the IPF of the current coding unit to identify the position '0' and the IPS use flag position is '1', indicating that the IPF technology is not used but the IPS technology is used, and is also transmitted to the decoding end through the bit stream.

j3) Then, superimpose the prediction block and the inversely transformed and inversely quantized residuals to obtain the reconstructed coding unit block, which is used as the prediction reference block of the next coding unit.

The technical solution 2 is specifically implemented in the intra-frame prediction part of the coding end as follows:

The encoder obtains the encoded information, including the intra-frame prediction filtering allowable flag and the intra-frame prediction smoothing filter (hereinafter referred to as IPS) allowable flag of the technical solution, etc. After obtaining the image information, the image is divided into several CTUs, and further It is divided into several CUs, and each independent CU performs intra-frame prediction;

During intra prediction,

①If the IPF allowable flag bit and the IPS allowable flag bit are both '1', perform all the following steps;

②If the IPF allowable flag bit is '1' and the IPS allowable flag bit is '0', only execute a4), b4), c4), d4), e4), and i4), j4);

③ If the IPF allowable flag bit is '0', and the IPS allowable flag bit is '1', and the current CU area is greater than or equal to 64 and less than 4096, only execute a4), b4), f4), g4), h4), and i4), j4);

④ If the IPF allowable flag bit and the IPS allowable flag bit are both '0', only execute a4), b4) and i4), j4):

a4) The current coding unit traverses all intra prediction modes, calculates the prediction block in each prediction mode, and calculates the bit rate distortion cost information of the current prediction mode according to the original pixel block;

b4) According to the principle of minimum rate distortion cost of all prediction modes above, select the optimal prediction mode of the current coding unit, and record the optimal prediction mode information and the corresponding bit rate distortion cost information;

c4) Perform a second traversal of all intra-frame prediction modes. This process enables the IPF technology. First, the predicted pixels in each intra-frame prediction mode are calculated to obtain the prediction block of the current coding unit;

d4) Perform IPF filtering on the prediction block of the current coding unit, select the corresponding filter according to the current prediction mode, and select the corresponding filter coefficient group according to the size of the current coding unit. The specific correspondence can be found in Table 1;

e4) Calculate the bit rate distortion cost information of each prediction mode according to the final predicted pixels and original pixels obtained by the IPF technology, and record the prediction mode and corresponding cost value of the minimum bit rate distortion cost information;

f4) Perform a third traversal of all intra-frame prediction modes, this process enables IPS technology, first calculates the predicted pixels in each intra-frame prediction mode, and obtains the prediction block of the current coding unit;

g4) Perform an IPS on the prediction block of the current coding unit to obtain the final prediction block;

h4) Calculate the bit rate distortion cost information of each prediction mode according to the final predicted pixels and original pixels obtained by the IPS technology, and record the prediction mode and corresponding cost value of the minimum bit rate distortion cost information;

i4) If the IPF allowable flag bit is '0' and the IPS allowable flag bit is '0', then the prediction mode index recorded in b) is transmitted to the decoding end via the bit stream;

If the IPF allowable flag bit is '1' and the IPS allowable flag bit is '0', then compare the minimum cost value recorded in b) with the minimum cost value recorded in e),

If the bit rate distortion cost in b4) is smaller, the prediction mode index code recorded in b4) is used as the optimal prediction mode of the current coding unit and transmitted to the decoding end through the bit stream, and the current coding unit of IPF is used as the flag position. '0', indicating that IPF technology is not used, and it is also transmitted to the decoding end through the bit stream;

If the bit rate distortion in e4) is smaller, use the prediction mode index code recorded in e4) as the optimal prediction mode of the current coding unit and transmit it to the decoding end via the bit stream, and use the IPF current coding unit to use the flag position' 1', indicating that IPF technology is used, and it is also transmitted to the decoding end through the bit stream;

If the IPF allowable flag bit is '0' and the IPS allowable flag bit is '1', then compare the minimum cost value recorded in b) with the minimum cost value recorded in h),

If the bit rate distortion cost in b4) is smaller, the prediction mode index code recorded in b4) is used as the optimal prediction mode of the current coding unit and transmitted to the decoding end via the bit stream, and the IPS usage flag of the current coding unit is used. The position is '0', which means that the technology is not used, and it is also transmitted to the decoding end through the bit stream;

If the bit rate distortion in h4) is smaller, then use the prediction mode index code recorded in h4) as the optimal prediction mode of the current coding unit and transmit it to the decoding end through the bit stream, and use the IPS flag position of the current coding unit. '1', indicating that this technology is used, and it is also transmitted to the decoding end through the bit stream;

If the IPF allow flag bit is '1' and the IPS allow flag bit is '1', then compare the minimum cost values recorded in b4), e4) and h4),

If the bit rate distortion cost in b4) is smaller, the prediction mode index code recorded in b) is transmitted to the decoding end as the optimal prediction mode of the current coding unit through the bit stream, and the IPS usage flag of the current coding unit is used. Bit and IPF use flag position '0', indicating that neither is used, and is also transmitted to the decoding end through the bit stream;

If the bit rate distortion in e4) is smaller, then use the prediction mode index encoding recorded in e) as the optimal prediction mode of the current coding unit and transmit it to the decoding end through the bit stream, and use the IPF of the current coding unit to identify the position '1' and do not transmit the IPS identification bit, indicating that the IPF technology is used instead of the IPS technology, and it is also transmitted to the decoding end through the bit stream;

If the bit rate distortion in h4) is smaller, then use the prediction mode index encoding recorded in h) as the optimal prediction mode of the current coding unit and transmit it to the decoding end via the bit stream, and use the IPF of the current coding unit to identify the position '0' and the IPS use flag position is '1', indicating that the IPF technology is not used but the IPS technology is used, and is also transmitted to the decoding end through the bit stream.

j4) After that, superimpose the prediction block and the residual after inverse transformation and inverse quantization to obtain the reconstructed coding unit block, which is used as the prediction reference block of the next coding unit.

Corresponding to the image encoding method shown in FIG. 9 , FIG. 10 is a schematic flowchart of the image encoding method in the embodiment of the present application, and the image encoding method can be applied to the purpose of the image decoding system 1 shown in FIG. 6 . The device 20 or the video decoder 200 shown in FIG. 8 . The flow shown in FIG. 10 is described by taking the execution subject as the video encoder 200 shown in FIG. 8 as an example. As shown in FIG. 10 , the cross-component prediction method provided by the embodiment of the present application includes:

Step S210, parse the bit stream to determine the intra-frame prediction filtering indication message and the first usage flag of the current decoding block, where the intra-frame prediction indication message includes a first indication message and a second indication message, the first indication message Used to indicate whether to allow the use of the first intra-frame prediction filtering mode, the second indication message is used to indicate whether to allow the use of the second intra-frame prediction filtering mode, the first intra-frame prediction filtering mode is the intra-frame prediction filtering IPF mode , the first use flag is the use flag of the first intra-frame prediction filtering mode;

Step S220, according to the intra-frame prediction filtering instruction message and the first use flag, determine to use the first frame rate prediction filtering mode to obtain the prediction block of the decoding block.

The specific process of intra-frame prediction at the decoding end of the technical solution 1 is as follows:

The decoder obtains the code bit stream, parses the bit stream to obtain the IPF permission flag and IPS permission flag of the current image sequence, parses the bit stream and performs inverse transformation and inverse quantization on the obtained residual information.

During the intra-frame prediction decoding process,

①If the allowable flag bit of IPF and the flag bit of IPS are both '1', perform all the following steps;

②If the allowable identification bit of IPF is '1' and the identification bit of IPS is '0', then only steps a5), b5), c5), d5) and g5) are performed;

3. If the allowable identification bit of IPF is '0' and the identification bit of IPS is '1', and the current CU area is greater than or equal to 65 and less than 5096, then only a5), b5), e5), f5) and g5) steps are performed ;

④If the allowable identification bit of IPF and the identification bit of IPS are all '0', then only a5), b5) and g5) steps are performed;

a5) Obtaining the bit stream and decoding to obtain residual information, and obtaining time-domain residual information through processes such as inverse transformation and inverse quantization;

b5) parse the bit stream to obtain the prediction mode of the current decoding unit, and calculate the prediction block according to the prediction mode of the current decoding unit and the adjacent reconstruction block;

c5) Parse and obtain the usage flag of IPF,

If the use flag of IPF is '0', no additional operation is performed on the current prediction block;

If the use flag of IPF is '1', then execute d5);

d5) selecting the corresponding filter according to the prediction mode classification information of the current decoding unit, selecting the corresponding filter coefficient group according to the size of the current decoding unit, then filtering each pixel in the prediction block to obtain the prediction block;

e5) Obtain the usage flag of IPF,

If the use identification bit of IPF is '1', then skip the remaining process of this step, and skip the step f5);

If the usage flag of the IPF is '0', parse and obtain the usage flag of the IPS.

If the use flag bit of IPS is '0', no additional operation is performed on the current prediction block;

If the use flag of IPS is '1', then execute f5);

f5) use the IPS device to filter the input prediction block twice to obtain the filtered current decoding unit prediction block;

g5) The reconstructed block of the current decoding unit is obtained by superimposing the restored residual information of the prediction block, which is output after post-processing;

The specific process of intra-frame prediction at the decoding end in the technical solution 2 is as follows:

The decoder obtains the bit stream, parses the bit stream to obtain the IPF permission flag and IPS permission flag of the current image sequence, parses the bit stream and performs inverse transformation and inverse quantization on the obtained residual information.

During the intra-frame prediction decoding process,

①If the allowable flag bit of IPF and the flag bit of IPS are both '1', perform all the following steps;

②If the allowable identification bit of IPF is '1' and the identification bit of IPS is '0', then only steps a6), b6), c6), d6) and g6) are performed;

3. If the allowable identification bit of IPF is '0' and the identification bit of IPS is '1', and the current CU area is greater than or equal to 66 and less than 6096, then only a6), b6), e6), f6) and g6) steps are performed ;

④If the allowable identification bit of IPF and the identification bit of IPS are all '0', then only a6), b6) and g6) steps are performed;

a6) Obtain the bit stream and decode to obtain residual information, and obtain time-domain residual information through processes such as inverse transformation and inverse quantization;

b6) parse the bit stream to obtain the prediction mode of the current decoding unit, and calculate the prediction block according to the prediction mode of the current decoding unit and the adjacent reconstruction block;

c6) Parse and obtain the usage flag of IPF,

If the use flag of IPF is '0', no additional operation is performed on the current prediction block;

If the use flag of IPF is '1', execute d6);

d6) selecting the corresponding filter according to the prediction mode classification information of the current decoding unit, selecting the corresponding filter coefficient group according to the size of the current decoding unit, then filtering each pixel in the prediction block to obtain the prediction block;

e6) Obtain the usage flag of IPF,

If the use identification bit of IPF is '1', then skip the remaining process of this step, and skip step f6);

If the usage flag of the IPF is '0', parse and obtain the usage flag of the IPS.

If the use flag bit of IPS is '0', no additional operation is performed on the current prediction block;

If the use flag of IPS is '1', then execute f6);

f6) use the IPS device to filter the input prediction block once to obtain the filtered current decoding unit prediction block;

g6) The reconstructed block of the current decoding unit is obtained by superimposing the restored residual information of the prediction block, which is output after post-processing;

The technical solution acts on the intra-frame prediction part in the coding and decoding framework. When using the IPS technology to filter the current coding unit or decoding unit, the current block needs to be filled first, and the steps are as follows:

a7) If the reference pixels on the left side and the top side outside the current prediction block are available, that is, there are reconstructed pixels on the left side and the top side, then the left column and the top row are filled with reconstructed pixels;

b7) If the reference pixels on the left or upper side outside the current prediction block are unavailable, that is, there are no reconstructed pixels on the left or upper side, the side without reconstructed pixels is filled with the row or column closest to that side of the current prediction block;

c7) Fill the right adjacent column outside the current prediction block with the predicted value of the rightmost column of the current prediction block;

d7) Fill the lower adjacent row outside the current prediction block with the lowest row prediction value of the current prediction block;

Fill the upper right corner pixel outside the current prediction block with the rightmost pixel filled on the upper side of the current prediction block, and use the filled rightmost pixel outside the current prediction block for the lower right pixel outside the current prediction block. Padding is performed, and the bottom-left pixel point outside the current prediction block is filled with the bottommost pixel point that has been filled with the padding amount on the left side outside the current prediction block.

FIG. 11A shows a filling schematic diagram of the prediction block, wherein Pred.Pixel represents the pixel of the prediction block, and Recon.Pixel represents the filled pixel.

FIG. 11B shows another filling schematic diagram of the prediction block, wherein Pred.Pixel represents the pixel of the prediction block, and Recon.Pixel represents the filled pixel.

The above IPS technology uses a simplified Gaussian convolution kernel to filter the prediction block. The filter has 9 taps and 9 different filter coefficients, as shown below:

filter_coefficients represents filter coefficients.

Filter each predicted pixel in the predicted block, and the filtering formula is as follows:

In the above formula, P' (x, y) is the final predicted value at the current coding unit (x, y), c ₁ , c ₂ and c ₃ are the coefficients in the above filters, respectively, in the above approximate Gaussian convolution In the kernel coefficient, c ₁ is 0075, c ₂ is 0.124, and c ₃ is 0.204. P(x, y) and others such as P(x-1, y-1) are the predicted values located at the current coding units (x, y) and (x-1, y-1), where the values of x and y are The value range does not exceed the width and height of the current coding unit block.

The convolution kernel coefficients used by the above IPS technology can be approximated as integers and all coefficients sum to the exponential power of 2, which can avoid both floating-point calculations and division operations of the computer, which greatly reduces the computational complexity, as shown below:

The sum of the above filter coefficients is 64, that is, the calculated predicted value needs to be shifted to the right by 6 bits.

In the above technical solution 2, the IPS technology uses a simplified Gaussian convolution kernel to filter the prediction block. The filter has a total of 25 taps and 6 different filter coefficients, as shown below:

Filter each predicted pixel in the predicted block, and the filtering formula is as follows:

In the above formula, P' (x, y) is the final predicted value at the current coding unit (x, y), c ₁ , c ₂ , c ₃ , c ₄ , c ₅ and c ₆ are the In the above approximate Gaussian convolution kernel coefficients, c ₁ is 0.0030, c ₂ is 0.0133, c ₃ is 0.0219, c ₄ is 0.0596, c ₅ is 0.0983, and c ₆ is 0.1621. P(x, y) and others such as P(x-1, y-1) are the predicted values located at the current coding units (x, y) and (x-1, y-1), where the values of x and y are The value range does not exceed the width and height of the current coding unit block.

The convolution kernel coefficients used by the above IPS technology can be approximated as integers and all coefficients sum to the exponential power of 2, which can avoid both floating-point calculations and division operations of the computer, which greatly reduces the computational complexity, as shown below:

The sum of the above filter coefficients is 1024, that is, the calculated predicted value needs to be shifted to the right by 10 bits.

In the above technical solution 2, the IPS technology can also use a simplified Gaussian convolution kernel to filter the prediction block. The filter has a total of 13 taps and 4 different filter coefficients, as shown below:

Filter each predicted pixel in the predicted block, and the filtering formula is as follows:

The filtering formula is as follows:

In the above formula, P' (x, y) is the final predicted value at the current coding unit (x, y), c ₁ , c ₂ , c ₃ and c ₄ are the coefficients in the above filter, respectively, in the above approximation Among the Gaussian convolution kernel coefficients, c ₁ is 13, c ₂ is 18, c ₃ is 25, and c ₄ is 32. P(x, y) and others such as P(x-1, y-1) are the predicted values located at the current coding units (x, y) and (x-1, y-1), where the values of x and y are The value range does not exceed the width and height of the current coding unit block.

The sum of the above filter coefficients is 256, that is, the calculated predicted value needs to be shifted to the right by 8 bits.

This technical solution will be applied to the coding and decoding part of intra-frame prediction, providing options for intra-frame prediction in operations that require smoothing or local blurring, and for parts where the image texture does not need to be too sharp, using this technology makes the predicted pixels smoother , the predicted block is closer to the original image, which will eventually improve the coding efficiency.

Technical solution 1 was tested on AVS' official simulation platform HPM7.0, and smooth filtering was performed on the intra-frame prediction block. The test results are shown in Table 2 and Table 3 under full intra-frame test conditions and random access conditions.

Table 2 All Intra test results Class Y U V 4K -0.61% -0.67% -0.84% 1080P -0.45% -0.78% -0.48% 720P -0.22% -0.08% -0.66% Average performance -0.42% -0.51% -0.66%

Table 3 Random Access test results Class Y U V 4K -0.25% -0.37% -0.57% 1080P -0.22% -0.41% -0.64% 720P -0.22% -0.01% -0.73% Average performance -0.23% -0.26% -0.65%

It can be seen from Table 2 and Table 3 that this scheme has a good performance improvement under both test conditions.

Under the AI test conditions, the luminance component has a BDBR saving of 0.42%, and the UV component has a BDBR saving of 0.51% and 0.66% respectively. It can be clearly seen that it has high performance and effectively improves the coding efficiency of the encoder.

From the perspective of various resolutions, this solution has a large encoding performance improvement on 4K resolution images, which will be beneficial to the development of ultra-high-definition images in the future, save more bit rate for ultra-high-resolution images, and save more multiple bandwidths.

This scheme proposes to perform smooth filtering on the prediction block calculated by the intra-frame prediction mode during the intra-frame prediction process, so as to improve the intra-frame prediction accuracy and effectively improve the coding efficiency. The details are as follows:

1. It is proposed to smooth the intra-coded prediction block;

2. It is proposed that in the case of IPS technology and IPF, when the allowable flag bits of the two technologies are both '1', IPF and IPS in the same coding unit will not be determined to be used at the same time;

3. It is proposed that when the encoder decides to use the IPF technology for the current coding unit, the IPS identification bit is not transmitted, and the decoder does not need to parse the IPS use identification bit;

When the encoder decides not to use the IPF technology for the current coding unit, it needs to transmit the identification bit of the IPS, and the decoder needs to parse the use identification bit of the IPS;

4. The IPS convolution kernel is proposed as a simplified 9-tap filtering Gaussian convolution kernel;

5. It is proposed to approximate the value of the convolution kernel of floating-point numbers, round the filter coefficients to avoid floating-point calculations, and use the shift operation instead of the division operation to save computing resources and reduce the complexity. ;

6. It is proposed that the 9-tap integer Gaussian convolution kernel filter coefficient is, the first filter coefficient is 5, the second filter coefficient is 8 and the third filter coefficient is 12, the predicted value after filtering needs to be shifted to the right by 6 bits;

7. Propose 25-tap and 13-tap filters.

The present application can also be further expanded from the following directions.

Extension 1: Replace the 9-tap Gaussian convolution kernel in this technical solution with a filter convolution kernel with more taps to achieve a better smoothing filtering effect.

Extended solution 2: The luminance component and the chrominance component in this technical solution are respectively used with independent identification bits to indicate whether the IPS is used or not.

Extended solution 3: The scope of use in this technical solution is limited, and IPS technology is not used for units with smaller prediction block areas, so as to reduce transmission identification bits and reduce computational complexity.

Extended solution 4: The scope of use in this technical solution is limited, and the prediction mode of the current coding unit is screened. If it is an average mode, the IPS technology is not used to reduce transmission identification bits and computational complexity.

An embodiment of the present application provides an image encoding apparatus, and the image encoding apparatus may be an image decoder or an image encoder. Specifically, the image encoding apparatus is configured to perform the steps performed by the image decoder in the above decoding method. The image encoding apparatus provided by the embodiments of the present application may include modules corresponding to corresponding steps.

In this embodiment of the present application, the image coding apparatus can be divided into functional modules according to the above method examples. For example, each functional module can be divided corresponding to each function, or two or more functions can be integrated into one processing module. middle. The above-mentioned integrated modules can be implemented in the form of hardware, or can be implemented in the form of software function modules. The division of modules in the embodiments of the present application is schematic, and is only a logical function division, and other division methods may be used in actual implementation.

In the case where each functional module is divided according to each function, FIG. 12 shows a possible schematic structural diagram of the image encoding apparatus involved in the above embodiment. As shown in FIG. 12 , the image encoding apparatus 12 includes a division unit 124 , a determination unit 121 , a transmission unit 122 , and a superimposition unit 123 .

The dividing unit 124 is configured to divide the image and determine the intra-frame prediction filtering instruction message of the current coding block, where the intra-frame prediction filtering instruction message includes a first indication message and a second indication message, and the first indication message is used to indicate Whether to allow the use of the first intra-frame prediction filtering mode, the second indication message is used to indicate whether to allow the use of the second intra-frame prediction filtering mode, and the first intra-frame prediction filtering mode is the intra-frame prediction filtering IPF mode;

The determining unit 121 is configured to filter the first intra-frame prediction of the current coding block if it is determined according to the intra-frame prediction filtering instruction message that the current coding block needs to use the first intra-frame prediction filtering mode The first use flag of the mode is set to allow use;

a transmission unit 122, configured to transmit the intra-frame prediction filter indication message, the first intra-frame prediction filter mode and the first use identification bit through a bit stream;

The superimposing unit 123 is configured to determine, according to the intra-frame prediction filtering instruction message and the first usage flag, to obtain the prediction block of the decoding block by using the first frame rate prediction filtering mode.

Wherein, all relevant contents of the steps involved in the above method embodiments can be cited in the functional descriptions of the corresponding functional modules, which will not be repeated here. Of course, the image encoding apparatus provided in the embodiment of the present application includes but is not limited to the above-mentioned modules. For example, the image encoding apparatus may further include a storage unit. The storage unit can be used to store the code and data of the image encoding device.

In the case of using an integrated unit, a schematic structural diagram of the image encoding apparatus provided by the embodiment of the present application is shown in FIG. 13 . In FIG. 13 , the image encoding device 13 includes: a processing module 130 and a communication module 131 . The processing module 130 is used to control and manage the actions of the image encoding device, for example, to perform the steps performed by the division unit 124, the determination unit 121, the transmission unit 122, and the superimposition unit 123, and/or to perform the techniques described herein. other processes. The communication module 131 is used to support the interaction between the image encoding device and other devices. As shown in FIG. 13 , the image encoding apparatus may further include a storage module 132, and the storage module 132 is used to store the program codes and data of the image encoding apparatus, for example, to store the contents stored in the above-mentioned storage unit.

The processing module 130 may be a processor or a controller, such as a central processing unit (Central Processing Unit, CPU), a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), ASIC, FPGA or other Program logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute the various exemplary logic blocks, modules and circuits described in connection with this disclosure. The processor may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and the like. The communication module 131 can be a transceiver, an RF circuit, a communication interface, or the like. The storage module 132 may be a memory.

Wherein, all relevant contents of the scenarios involved in the above method embodiments can be cited in the functional descriptions of the corresponding functional modules, which will not be repeated here. The above-mentioned image encoding apparatus can execute the above-mentioned image encoding method, and the image encoding apparatus may specifically be a video image encoding apparatus or other equipment having an image encoding function.

The present application also provides an image encoder, including a non-volatile storage medium, and a central processing unit, wherein the non-volatile storage medium stores an executable program, the central processing unit is connected to the non-volatile storage medium, and The executable program is executed to implement the image encoding method of the embodiment of the present application.

An embodiment of the present application provides an image decoding apparatus, and the image decoding apparatus may be an image decoder or an image decoder. Specifically, the image decoding apparatus is configured to perform the steps performed by the image decoder in the above decoding method. The image decoding apparatus provided by the embodiments of the present application may include modules corresponding to corresponding steps.

In this embodiment of the present application, the image decoding apparatus may be divided into functional modules according to the above method examples. For example, each functional module may be divided corresponding to each function, or two or more functions may be integrated into one processing module. middle. The above-mentioned integrated modules can be implemented in the form of hardware, or can be implemented in the form of software function modules. The division of modules in the embodiments of the present application is schematic, and is only a logical function division, and other division methods may be used in actual implementation.

In the case where each functional module is divided according to each function, FIG. 14 shows a possible schematic structural diagram of the image decoding apparatus involved in the above embodiment. As shown in FIG. 14 , the image decoding device 14 includes an analysis unit 142 and a determination unit 141 .

a parsing unit, configured to determine an intra-frame prediction filter indication message and a first use flag of the current decoding block, where the intra-frame prediction indication message includes a first indication message and a second indication message, and the first indication message is used to indicate whether to allow the use of the first intra-frame prediction filter mode, the second indication message is used to indicate whether to allow the use of the second intra-frame prediction filter mode, the first intra-frame prediction filter mode is the intra-frame prediction filter IPF mode, the The first use flag is the use flag of the first intra-frame prediction filtering mode;

A determining unit, configured to determine, according to the intra-frame prediction filtering instruction message and the first use flag, to obtain the prediction block of the decoding block by using the first frame rate prediction filtering mode.

Wherein, all relevant contents of the steps involved in the above method embodiments can be cited in the functional descriptions of the corresponding functional modules, which will not be repeated here. Of course, the image decoding apparatus provided in the embodiment of the present application includes but is not limited to the above-mentioned modules. For example, the image decoding apparatus may further include a storage unit. The storage unit can be used to store the code and data of the image decoding device.

In the case of using an integrated unit, a schematic structural diagram of an image decoding apparatus provided by an embodiment of the present application is shown in FIG. 13 . In FIG. 13 , the image decoding device 15 includes: a processing module 150 and a communication module 151 . The processing module 150 is used to control and manage the actions of the image decoding apparatus, eg, to perform the steps performed by the parsing unit 142, the determination unit 141, and/or other processes for performing the techniques described herein. The communication module 151 is used to support the interaction between the image decoding apparatus and other devices. As shown in FIG. 13 , the image decoding apparatus may further include a storage module 152, and the storage module 152 is used to store the program codes and data of the image decoding apparatus, for example, to store the contents stored in the above-mentioned storage unit.

The processing module 150 may be a processor or a controller, such as a central processing unit (CPU), a general-purpose processor, a digital signal processor (DSP), an ASIC, an FPGA, or other Program logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute the various exemplary logic blocks, modules and circuits described in connection with this disclosure. The processor may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and the like. The communication module 151 can be a transceiver, an RF circuit, a communication interface, or the like. The storage module 152 may be a memory.

Wherein, all relevant contents of the scenarios involved in the above method embodiments can be cited in the functional descriptions of the corresponding functional modules, which will not be repeated here. The above-mentioned image decoding apparatus may execute the above-mentioned image decoding method, and the image decoding apparatus may specifically be an image and image decoding apparatus or other equipment having an image decoding function.

The present application also provides an image decoder, including a non-volatile storage medium, and a central processing unit, wherein the non-volatile storage medium stores an executable program, the central processing unit is connected to the non-volatile storage medium, and The executable program is executed to implement the image decoding method of the embodiment of the present application.

The present application also provides a terminal, where the terminal includes: one or more processors, a memory, and a communication interface. The memory and the communication interface are coupled with one or more processors; the memory is used to store computer program codes, and the computer program codes include instructions. When one or more processors execute the instructions, the terminal executes the images of the embodiments of the present application. Encoding and/or image decoding methods. The terminal here can be an image display device, a smart phone, a portable computer, and other devices that can process images or play images.

Another embodiment of the present application further provides a computer-readable storage medium, the computer-readable storage medium includes one or more program codes, and the one or more programs include instructions, when the processor in the decoding device executes the program code , the decoding device executes the image encoding method and the image decoding method of the embodiments of the present application.

In another embodiment of the present application, there is also provided a computer program product, the computer program product includes computer-executable instructions, and the computer-executable instructions are stored in a computer-readable storage medium; at least one processor of the decoding device can be obtained from the computer The read storage medium reads the computer-executed instruction, and at least one processor executes the computer-executed instruction so that the terminal implements the image encoding method and the image decoding method of the embodiments of the present application.

In the above embodiments, it may be implemented in whole or in part through software, hardware, firmware or any combination thereof. When implemented using a software program, it may be in the form of a computer program product, in whole or in part. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on the computer, all or part of the processes or functions described in the embodiments of the present application are generated.

The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable device. The computer instructions may be stored on or transmitted from one computer readable storage medium to another computer readable storage medium, for example, the computer instructions may be downloaded from a website, computer, server or data center Transmission to another website, computer, server or data center via wired (eg coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (eg infrared, wireless, microwave, etc.).

The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, a data center, etc., which is integrated with one or more available media. The available media may be magnetic media (eg, floppy disk, hard disk, magnetic tape), optical media (eg, DVD), or semiconductor media (eg, Solid State Disk (SSD)), and the like.

Through the description of the above embodiments, those skilled in the art can clearly understand that, for the convenience and brevity of the description, only the division of the above functional modules is used as an example for illustration. The allocation is completed by different functional modules, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above.

In the several embodiments provided in this application, it should be understood that the disclosed apparatus and method may be implemented in other manners. For example, the device embodiments described above are only illustrative. For example, the division of the modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or elements may be Incorporation may either be integrated into another device, or some features may be omitted, or not implemented. On the other hand, the shown or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, indirect coupling or communication connection of devices or units, and may be electrical, mechanical or other forms.

The unit described as a separate component may or may not be physically separated, and a component shown as a unit may be one entity unit or multiple entity units, that is, it may be located in one place, or may be distributed to multiple different places. . Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist independently, or two or more units may be integrated into one unit. The above-mentioned integrated units can be implemented in the form of hardware, or can be implemented in the form of software functional units.

The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of the present application can be embodied in the form of software products in essence, or the parts that contribute to the prior art, or all or part of the technical solutions, which are stored in a storage medium , which includes several instructions for causing a device (which may be a single chip microcomputer, a chip, etc.) or a processor (processor) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage media include: flash drives, mobile hard drives, Read-Only Memory (ROM), Random Access Memory (RAM), disks or CDs, etc. that can store program codes medium.

The above are only specific embodiments of the present application, but the protection scope of the present application is not limited to this, and any changes or substitutions within the technical scope disclosed in the present application should be covered within the protection scope of the present application. . Therefore, the protection scope of the present application shall be subject to the protection scope of the claimed item.

1: Image 10: Source device 12: Image coding device 13: Image coding device 14: Image decoding device 15: Image decoding device 20: Purpose Device 30: Link 40: Storage device 41: Postprocessing entities 42: Network entities 100: Video encoder 101: Transformer 102: Quantizer 103: Entropy Encoder 104: Inverse Quantizer 105: Inverse Transformer 106: Filter unit 107: Memory 108: Prediction processing unit 109: Intra Predictor 110: Inter predictor 111: Summation 112: Summation 120: Image source 121: Determine unit 122: Transmission unit 123: Overlay unit 124: Division Units 130: Processing Modules 131: Communication module 132: Storage Module 140: Output interface 141: Determine unit 142: parsing unit 150: Processing Mods 151: Communication module 152: Storage Module 200: Video decoder 203: Entropy Decoder 204: Inverse Quantizer 205: Inverse Transformer 206: Filter unit 207: Memory 208: Prediction Processing Unit 209: Intra Predictor 210: Inter predictor 211: Summation 220: Display device 240: Input interface S110~S140: Steps S210~S220: Steps

In order to illustrate the embodiments of the present invention or the technical solutions in the prior art more clearly, the following briefly introduces the accompanying drawings that need to be used in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the For some embodiments of the invention, for those of ordinary skill in the art, other drawings can also be obtained according to these drawings without any creative effort.

1 is a schematic block diagram of a coding tree unit in an embodiment of the present application;

2 is a schematic block diagram of a CTU and a coding block CU in an embodiment of the present application;

3 is a schematic block diagram of a color format in an embodiment of the application;

4 is a schematic diagram of the IPF in the embodiment of the application;

5 is a schematic diagram of intra-frame prediction filtering in an embodiment of the present application;

6 is a schematic block diagram of an image decoding system in an embodiment of the present application;

7 is a schematic block diagram of an image encoder in an embodiment of the present application;

FIG. 8 is a schematic block diagram of an image decoder in an embodiment of the present application;

FIG. 9 is a schematic flowchart of an image encoding method in an embodiment of the present application;

10 is a schematic flowchart of an image decoding method in an embodiment of the present application;

11A is a schematic diagram of a filling of a prediction block in an embodiment of the present application;

11B is a schematic diagram of another filling of a prediction block in an embodiment of the present application;

12 is a block diagram of a functional unit of an image encoding apparatus in an embodiment of the application;

13 is a block diagram of another functional unit of the image encoding apparatus in the embodiment of the application;

14 is a block diagram of a functional unit of an image decoding apparatus in an embodiment of the application;

FIG. 15 is a block diagram of another functional unit of the image decoding apparatus according to the embodiment of the present application.

S110~S140: Steps

Claims (9)

An image encoding method, comprising: Divide the image, and determine the intra-frame prediction filtering instruction message of the current coding block, where the intra-frame prediction filtering instruction message includes a first instruction message and a second instruction message, and the first instruction message is used to indicate whether to allow the use of the first frame Intra prediction filtering mode, the second indication message is used to indicate whether to allow the use of the second intra prediction filtering mode, and the first intra prediction filtering mode is an intra prediction filtering IPF mode; If it is determined according to the intra-frame prediction filtering instruction message that the current coding block needs to use the first intra-frame prediction filtering mode, the first use flag of the first intra-frame prediction filtering mode of the current coding block is set bit set to allow use; transmitting the intra-prediction filter indication message, the first intra-prediction filter mode, and the first usage flag via a bitstream; The prediction block of the current coding block and the residual block obtained after inverse transformation and inverse quantization are superimposed to obtain the reconstructed reconstructed block, which is used as the prediction reference block of the next coding block. An image decoding method, comprising: Parse the bit stream, and determine the intra-frame prediction filter indication message and the first usage flag of the current decoding block, where the intra-frame prediction indication message includes a first indication message and a second indication message, and the first indication message is used to indicate whether to allow the use of the first intra-frame prediction filter mode, the second indication message is used to indicate whether to allow the use of the second intra-frame prediction filter mode, the first intra-frame prediction filter mode is the intra-frame prediction filter IPF mode, the The first use flag is the use flag of the first intra-frame prediction filtering mode; According to the intra-frame prediction filtering instruction message and the first use flag, it is determined that the prediction block of the decoding block is obtained by using the first frame rate prediction filtering mode. An image encoding device, comprising: a dividing unit, configured to divide the image and determine the intra-frame prediction filtering indication information of the current coding block, the intra-frame prediction filtering indication information includes a first indication message and a second indication message, and the first indication message is used to indicate whether The first intra-frame prediction filtering mode is allowed to be used, and the second indication message is used to indicate whether the second intra-frame prediction filtering mode is allowed to be used, and the first intra-frame prediction filtering mode is an intra-frame prediction filtering IPF mode; a determining unit, configured to set the first intra prediction filtering mode of the current coding block if it is determined according to the intra prediction filtering instruction message that the current coding block needs to use the first intra prediction filtering mode The first use flag is set to allow use; a transmission unit, configured to transmit the intra-frame prediction filter indication message, the first intra-frame prediction filter mode and the first use identification bit through a bit stream; and a superimposing unit, configured to determine, according to the intra-frame prediction filtering instruction message and the first usage flag, to use the first frame rate prediction filtering mode to obtain the prediction block of the decoding block. An image decoding device, comprising: a parsing unit, configured to determine an intra-frame prediction filter indication message and a first use flag of the current decoding block, where the intra-frame prediction indication message includes a first indication message and a second indication message, and the first indication message is used to indicate whether to allow the use of the first intra-frame prediction filter mode, the second indication message is used to indicate whether to allow the use of the second intra-frame prediction filter mode, the first intra-frame prediction filter mode is the intra-frame prediction filter IPF mode, the The first use flag is the use flag of the first intra-frame prediction filtering mode; A determining unit, configured to determine, according to the intra-frame prediction filtering instruction message and the first use flag, to obtain the prediction block of the decoding block by using the first frame rate prediction filtering mode. An encoder includes a non-volatile storage medium and a central processing unit, the non-volatile storage medium stores an executable program, the central processing unit is connected to the non-volatile storage medium, and when the central processing unit executes When the executable program is executed, the encoder performs the bidirectional inter prediction method as claimed in claim 1 or 2. A decoder includes a non-volatile storage medium and a central processing unit, the non-volatile storage medium stores an executable program, the central processing unit is connected to the non-volatile storage medium, and when the central processing unit executes When the executable program is executed, the decoder performs the bidirectional inter prediction method as claimed in claim 1 or 2. A terminal, the terminal comprises: one or more processors, a memory and a communication interface; the memory, the communication interface are connected with the one or more processors; the terminal communicates with the one or more processors through the communication interface other equipment communication, the memory is used to store computer program code, the computer program code includes instructions, When the one or more processors execute the instructions, the terminal executes the method as claimed in claim 1 or 2. A computer program product comprising instructions which, when run on a terminal, cause the terminal to perform the method as claimed in claim 1 or 2. A computer-readable storage medium comprising instructions which, when executed on a terminal, cause the terminal to perform the method as claimed in claim 1 or 2.

Images