CN116708794A

CN116708794A - Image component prediction method, encoder, decoder, and storage medium

Info

Publication number: CN116708794A
Application number: CN202310752393.0A
Authority: CN
Inventors: 霍俊彦; 万帅; 马彦卓
Original assignee: Guangdong Oppo Mobile Telecommunications Corp Ltd
Current assignee: Guangdong Oppo Mobile Telecommunications Corp Ltd
Priority date: 2019-03-18
Filing date: 2019-10-28
Publication date: 2023-09-05
Also published as: CN113273194A; WO2020186763A1; CN113676732A; EP3910951A4; US11368680B2; US20230388491A1; KR20210119514A; CN113676732B; EP4340360A2; US20210368167A1; EP3910951A1; US11778172B2; JP2022522487A; US20220248000A1; JP7383720B2

Abstract

The embodiment of the application discloses an image component prediction method, an encoder, a decoder and a storage medium, wherein the method comprises the following steps: determining adjacent reference pixels of a current block in an image; constructing a subset of adjacent reference pixels from the adjacent reference pixels; wherein the adjacent reference pixel subset includes a part of pixels in the adjacent reference pixels; calculating model parameters of a prediction model according to the adjacent reference pixel subsets; the prediction model comprises N prediction sub-models, the N prediction sub-models correspond to N groups of model parameters, the prediction sub-models are used for performing cross-component prediction processing on the image component to be predicted through the corresponding model parameters, and N is a positive integer greater than or equal to 2.

Description

Image component prediction method, encoder, decoder, and storage medium

The present application is a divisional application of application having a filing date of 2019, 10 month and 28 days, application number of 2019800887618, and entitled "image component prediction method, encoder, decoder, and storage medium".

Technical Field

The embodiment of the application relates to the technical field of image processing, in particular to an image component prediction method, an encoder, a decoder and a storage medium.

Background

With the improvement of the requirements of people on the video display quality, new video application forms such as high-definition and ultra-high-definition videos and the like are generated. The h.265/high efficiency video coding (High Efficiency Video Coding, HEVC) has failed to meet the rapidly evolving needs of video applications, and the joint video research group (Joint Video Exploration Team, jfet) has proposed the next generation video coding standard h.266/multi-function video coding (Versatile Video Coding, VVC) with a corresponding Test Model being the VVC reference software Test Model (VTM).

In VTM, a prediction model-based image component prediction method has been integrated, by which a chrominance component can be predicted from a luminance component of a current coding block. However, in the process of constructing the prediction model, especially when the prediction model is a nonlinear model or a multi-model (for example, the prediction model is composed of a plurality of linear models), the number of sample points used for model parameter derivation is large, and the calculation complexity and the memory bandwidth are high; also, abnormal sample points may exist in the sample points, resulting in inaccurate prediction model construction.

Disclosure of Invention

The embodiment of the application provides an image component prediction method, an encoder, a decoder and a storage medium, which not only can reduce the calculation complexity, but also can improve the accuracy of a prediction model by reducing the number of pixels in an adjacent reference pixel set, thereby improving the prediction accuracy of an image component to be processed.

The technical scheme of the embodiment of the application can be realized as follows:

in a first aspect, an embodiment of the present application provides an image component prediction method, applied to an encoder or a decoder, including:

determining adjacent reference pixels of a current block in an image;

constructing a subset of adjacent reference pixels from the adjacent reference pixels; wherein the adjacent reference pixel subset includes a part of pixels in the adjacent reference pixels;

calculating model parameters of a prediction model according to the adjacent reference pixel subsets; the prediction model comprises N prediction sub-models, the N prediction sub-models correspond to N groups of model parameters, the prediction sub-models are used for performing cross-component prediction processing on the image component to be predicted through the corresponding model parameters, and N is a positive integer greater than or equal to 2.

In a second aspect, an embodiment of the present application provides an image component prediction method, applied to an encoder or a decoder, including:

determining a first reference pixel set of a first image component of a current block in an image, wherein the first reference pixel set contains pixels adjacent to the current block;

constructing N first reference pixel subsets by using a first reference pixel set, wherein the first reference pixel subsets comprise partial pixels in the first reference pixel set, and N is equal to the number of prediction models;

And respectively calculating model parameters of N prediction models by using the N first reference pixel subsets, wherein the prediction models are used for mapping a first image component value of the current block into a predicted value of a second image component of the current block, and the second image component is different from the first image component.

In a third aspect, an embodiment of the present application provides an encoder, including: a first determination unit, a first construction unit and a first calculation unit, wherein,

a first determining unit configured to determine adjacent reference pixels of a current block in an image;

a first construction unit configured to construct a subset of adjacent reference pixels from the adjacent reference pixels; wherein the adjacent reference pixel subset comprises part of pixels in the adjacent reference pixels;

a first calculation unit configured to calculate model parameters of a predictive model from the subset of adjacent reference pixels; the prediction model comprises N prediction sub-models, the N prediction sub-models correspond to N groups of model parameters, the prediction sub-models are used for performing cross-component prediction processing on the image component to be predicted through the corresponding model parameters, and N is a positive integer greater than or equal to 2.

In a fourth aspect, embodiments of the present application provide an encoder comprising a first memory and a first processor, wherein,

a first memory for storing a computer program capable of running on the first processor;

a first processor for performing the method according to the first or second aspect when the computer program is run.

In a fifth aspect, an embodiment of the present application provides a decoder including a second determination unit, a second construction unit, and a second calculation unit, wherein,

a second determining unit configured to determine adjacent reference pixels of a current block in the image;

a second construction unit configured to construct a subset of adjacent reference pixels from the adjacent reference pixels; wherein the adjacent reference pixel subset comprises part of pixels in the adjacent reference pixels;

a second calculation unit configured to calculate model parameters of a predictive model from the subset of adjacent reference pixels; the prediction model comprises N prediction sub-models, the N prediction sub-models correspond to N groups of model parameters, the prediction sub-models are used for performing cross-component prediction processing on the image component to be predicted through the corresponding model parameters, and N is a positive integer greater than or equal to 2.

In a sixth aspect, an embodiment of the present application provides a decoder including a second memory and a second processor, wherein,

a second memory for storing a computer program capable of running on the second processor;

a second processor for performing the method according to the first or second aspect when the computer program is run.

In a seventh aspect, an embodiment of the present application provides a computer storage medium storing an image component prediction program, which when executed by a first processor or a second processor, implements the method according to the first or second aspect.

The embodiment of the application provides an image component prediction method, an encoder, a decoder and a storage medium, which are used for determining adjacent reference pixels of a current block in an image; constructing a subset of adjacent reference pixels from the adjacent reference pixels; wherein the adjacent reference pixel subset comprises part of pixels in the adjacent reference pixels; calculating model parameters of a prediction model according to the adjacent reference pixel subsets; the prediction model comprises N prediction sub-models, the N prediction sub-models correspond to N groups of model parameters, the prediction sub-models are used for performing cross-component prediction processing on image components to be predicted through the corresponding model parameters, and N is a positive integer greater than or equal to 2; in this way, as the screening process is carried out on the adjacent reference pixels of the current block, unimportant reference pixel points or abnormal reference pixel points can be removed, so that the number of pixels in the adjacent reference pixel set is reduced, the number of pixels in the adjacent reference pixel subset is smaller, the calculation complexity and the memory bandwidth are reduced, and the accuracy of the prediction model is improved; in addition, cross-component prediction processing is carried out on the image component to be processed through at least two predictor models, so that the prediction accuracy of the image component to be processed is further improved, and meanwhile, the prediction efficiency of the video image is improved.

Drawings

Fig. 1 is a schematic block diagram of a video coding system according to an embodiment of the present application;

fig. 2 is a schematic block diagram of a video decoding system according to an embodiment of the present application;

fig. 3 is a flowchart of an image component prediction method according to an embodiment of the present application;

FIG. 4 is a schematic diagram of a structure for selecting a subset of adjacent reference pixels on a current block according to an embodiment of the present application;

FIG. 5 is a schematic diagram illustrating another embodiment of selecting a subset of adjacent reference pixels on a current block;

FIG. 6 is a schematic diagram of grouping adjacent reference pixels of a coding block according to an embodiment of the present application;

FIG. 7 is a flowchart of another image component prediction method according to an embodiment of the present application;

FIG. 8 is a schematic diagram of an encoder according to an embodiment of the present application;

FIG. 9 is a schematic diagram of a specific hardware structure of an encoder according to an embodiment of the present application;

fig. 10 is a schematic diagram of a decoder according to an embodiment of the present application;

fig. 11 is a schematic diagram of a specific hardware structure of a decoder according to an embodiment of the present application.

Detailed Description

For a more complete understanding of the nature and the technical content of the embodiments of the present application, reference should be made to the following detailed description of embodiments of the application, taken in conjunction with the accompanying drawings, which are meant to be illustrative only and not limiting of the embodiments of the application.

In video images, a first image component, a second image component and a third image component are generally adopted to characterize Coding Blocks (CBs); wherein the three image components are a luminance component, a blue chrominance component and a red chrominance component, respectively, and in particular, the luminance component is generally represented by the symbol Y, the blue chrominance component is generally represented by the symbol Cb or U, and the red chrominance component is generally represented by the symbol Cr or V; thus, the video image may be represented in YCbCr format or YUV format.

In the embodiment of the present application, the first image component may be a luminance component, the second image component may be a blue chrominance component, and the third image component may be a red chrominance component, but the embodiment of the present application is not particularly limited.

In the next generation video coding standard h.266, in order to further improve coding performance and coding efficiency, extended improvement is performed on inter-component prediction (Cross-component Prediction, CCP), and Cross-component linear model prediction (Cross-component Linear Model Prediction, CCLM) is proposed. In h.266, the CCLM can realize not only prediction of the first image component to the second image component but also prediction of the first image component to the third image component or between the third image component to the first image component, and even prediction of the second image component to the third image component or between the third image component to the second image component. The following description will take prediction from the first image component to the second image component as an example, but the technical solution of the embodiment of the present application may be applied to prediction of other image components as well.

Among VTMs, there are currently two predictive modes of CCLM: one is a prediction mode of a single model CCLM; the other is a predictive mode of a multi-Model CCLM (MMLM), also known as MMLM. As the name suggests, the prediction mode of the single model CCLM is only one prediction model to implement the prediction from the first image component to the second image component; the prediction mode of MMLM is common to multiple prediction models to implement the prediction from the first image component to the second image component. For example, in the prediction mode of MMLM, reference pixel points adjacent to the coding block may be grouped into adjacent reference pixel sets, and the adjacent reference pixel sets may be divided into two groups, each of which may be separately used as a training set for deriving model parameters in the prediction model, i.e., each group may be capable of deriving a set of model parameters α and β.

In order to ensure the accuracy of the model parameters used by each of the prediction modes of the MMLM, the reference pixel set constructed for deriving the model parameters needs to be more accurate. Based on the above, the embodiment of the application provides an image component prediction method, which is implemented by determining adjacent reference pixels of a current block in an image; constructing a subset of adjacent reference pixels from the adjacent reference pixels; wherein the adjacent reference pixel subset comprises part of pixels in the adjacent reference pixels; calculating model parameters of the prediction model according to the adjacent reference pixel subsets; the prediction model comprises N prediction sub-models, the N prediction sub-models correspond to N groups of model parameters, the prediction sub-models are used for performing cross-component prediction processing on image components to be predicted through the corresponding model parameters, and N is a positive integer greater than or equal to 2; in this way, as the screening process is carried out on the adjacent reference pixels of the current block, unimportant reference pixel points or abnormal reference pixel points can be removed, so that the number of pixels in the adjacent reference pixel set is reduced, the number of pixels in the adjacent reference pixel subset is smaller, the calculation complexity and the memory bandwidth are reduced, and the accuracy of the prediction model is improved; in addition, cross-component prediction processing is carried out on the image component to be processed through at least two predictor models, so that the prediction accuracy of the image component to be processed is further improved, and meanwhile, the prediction efficiency of the video image is improved.

Embodiments of the present application will be described in detail below with reference to the accompanying drawings.

Referring to fig. 1, there is shown an example of a block diagram of a video coding system according to an embodiment of the present application; as shown in fig. 1, the video coding system 100 includes a transform and quantization unit 101, an intra estimation unit 102, an intra prediction unit 103, a motion compensation unit 104, a motion estimation unit 105, an inverse transform and inverse quantization unit 106, a filter control analysis unit 107, a filtering unit 108, a coding unit 109, a decoding image buffer unit 110, and the like, wherein the filtering unit 108 can implement deblocking filtering and Sample Adaptive 0 offset (sao) filtering, and the coding unit 109 can implement header information coding and Context-based Adaptive binary arithmetic coding (Context-based Adaptive Binary Arithmatic Coding, CABAC). For an input original video signal, a video Coding block can be obtained through Coding Tree Unit (CTU) division, and then residual pixel information obtained after intra-frame or inter-frame prediction is transformed by a transformation and quantization Unit 101, including transforming the residual information from a pixel domain to a transformation domain, and quantizing the obtained transformation coefficient to further reduce a bit rate; the intra estimation unit 102 and the intra prediction unit 103 are used for intra prediction of the video coding block; in particular, the intra-estimation unit 102 and the intra-prediction unit 103 are used to determine an intra-prediction mode to be used to encode the video encoding block; the motion compensation unit 104 and the motion estimation unit 105 are used to perform inter-prediction encoding of the received video coding block relative to one or more blocks in one or more reference frames to provide temporal prediction information; the motion estimation performed by the motion estimation unit 105 is a process of generating a motion vector that can estimate the motion of the video encoding block, and then the motion compensation unit 104 performs motion compensation based on the motion vector determined by the motion estimation unit 105; after determining the intra prediction mode, the intra prediction unit 103 is further configured to provide the selected intra prediction data to the encoding unit 109, and the motion estimation unit 105 also transmits the calculated determined motion vector data to the encoding unit 109; in addition, the inverse transform and inverse quantization unit 106 is used for reconstructing the video coding block, reconstructing a residual block in the pixel domain, removing blocking artifacts by the filter control analysis unit 107 and the filtering unit 108, and adding the reconstructed residual block to a predictive block in the frame of the decoded image buffer unit 110 to generate a reconstructed video coding block; the encoding unit 109 is configured to encode various encoding parameters and quantized transform coefficients, and in a CABAC-based encoding algorithm, context content may be based on neighboring encoding blocks, and may be configured to encode information indicating the determined intra prediction mode, and output a code stream of the video signal; and the decoded picture buffer unit 110 is for storing reconstructed video coding blocks for prediction reference. As video image encoding proceeds, new reconstructed video encoding blocks are generated, and the reconstructed video encoding blocks are stored in the decoded image buffer unit 110.

Referring to fig. 2, an example of a block diagram of a video decoding system according to an embodiment of the present application is shown; as shown in fig. 2, the video decoding system 200 includes a decoding unit 201, an inverse transform and inverse quantization unit 202, an intra prediction unit 203, a motion compensation unit 204, a filtering unit 205, a decoded image buffer unit 206, and the like, wherein the decoding unit 201 can implement header information decoding and CABAC decoding, and the filtering unit 205 can implement deblocking filtering and SAO filtering. After the input video signal is subjected to the coding process of fig. 1, outputting a code stream of the video signal; the code stream is input into the video decoding system 200, and first passes through the decoding unit 201 to obtain decoded transform coefficients; processing by the inverse transform and inverse quantization unit 202 for the transform coefficients to generate a residual block in the pixel domain; the intra prediction unit 203 may be used to generate prediction data for a current video decoded block based on the determined intra prediction mode and data from a previously decoded block of a current frame or picture; the motion compensation unit 204 is a unit that determines prediction information for a video decoding block by parsing the motion vector and other associated syntax elements and uses the prediction information to generate a predictive block of the video decoding block being decoded; forming a decoded video block by summing the residual block from the inverse transform and inverse quantization unit 202 with a corresponding predictive block generated by the intra prediction unit 203 or the motion compensation unit 204; the decoded video signal is passed through a filtering unit 205 to remove blocking artifacts, which may improve video quality; the decoded video blocks are then stored in a decoded image buffer unit 206, and the decoded image buffer unit 206 stores reference images for subsequent intra prediction or motion compensation, and is also used for output of video signals, i.e. the restored original video signals.

The image component prediction method in the embodiment of the application is mainly applied to the intra-frame prediction unit 103 part shown in fig. 1 and the intra-frame prediction unit 203 part shown in fig. 2, and is specifically applied to the CCLM prediction part in intra-frame prediction. That is, the image component prediction method in the embodiment of the present application may be applied to both a video encoding system and a video decoding system, or even to both a video encoding system and a video decoding system, but the embodiment of the present application is not particularly limited. When the method is applied to the intra prediction unit 103 part in a video coding system, "current block" specifically refers to the current coded block in intra prediction; when the method is applied to the intra prediction unit 203 part in a video decoding system, "current block" specifically refers to the current decoded block in intra prediction.

Based on the above application scenario examples of fig. 1 or fig. 2, referring to fig. 3, a flowchart of an image component prediction method provided by an embodiment of the present application is shown. As shown in fig. 3, the method may include:

s301: determining adjacent reference pixels of a current block in an image;

it should be noted that the video image may be divided into a plurality of image blocks, and each of the image blocks to be currently encoded may be referred to as an encoding block. Wherein each encoding block may include a first image component, a second image component, and a third image component; and the current block is a coding block in the video image, which is currently to be predicted for the first image component, the second image component or the third image component. When the first image component is required to be predicted through the prediction model, the image component to be predicted is the first image component; when the second image component is required to be predicted through the prediction model, the image component to be predicted is the second image component; when the third image component needs to be predicted through the prediction model, the image component to be predicted is the third image component.

It should be further noted that, when the left neighboring region, the left lower neighboring region, the upper neighboring region, and the upper right neighboring region are all effective regions, the neighboring reference pixels may be composed of neighboring reference pixel points in the left neighboring region and the upper neighboring region of the current block, may be composed of neighboring reference pixel points in the left neighboring region and the lower left neighboring region of the current block, and may be composed of neighboring reference pixel points in the upper neighboring region and the upper right neighboring region of the current block, which is not particularly limited.

In some embodiments, optionally, for S301, the determining the adjacent reference pixel of the current block in the image may include:

acquiring reference pixels adjacent to at least one edge of the current block; wherein at least one edge of the current block includes at least one of: uplink, right uplink, left column and left downlink;

and obtaining the adjacent reference pixels according to the acquired reference pixels.

It should be noted that, at least one edge of the current block may include at least one of the following: up (also referred to as an upper side), right up (also referred to as an upper right side), left column (also referred to as a left side), or left down (also referred to as a lower left side).

Optionally, in some embodiments, if at least one edge of the current block is a left edge and/or an upper edge, for S301, the determining the neighboring reference pixels of the current block in the image may include:

acquiring a reference pixel point adjacent to at least one edge of the current block; wherein the at least one edge comprises a left side edge of the current block and/or an upper side edge of the current block;

It should be noted that the at least one edge of the current block may include a left edge of the current block and/or an upper edge of the current block; that is, at least one side of the current block may refer to an upper side of the current block, may refer to a left side of the current block, and may even refer to an upper side and a left side of the current block.

Thus, when the left side neighboring region and the upper side neighboring region are all effective regions, the neighboring reference pixels at this time may be composed of reference pixel points neighboring the left side of the current block and reference pixel points neighboring the upper side of the current block, and when the left side neighboring region is an effective region and the upper side neighboring region is an ineffective region, the neighboring reference pixels at this time may be composed of reference pixel points neighboring the left side of the current block; when the left neighboring region is an invalid region and the upper neighboring region is an effective region, the neighboring reference pixels at this time may be composed of reference pixel points neighboring to the upper side of the current block.

Optionally, if at least one edge of the current block is an adjacent column formed by a left side edge and a left lower side edge, and/or an adjacent row formed by an upper side edge and an upper right side edge, for S301, the determining an adjacent reference pixel of the current block in the image may include:

acquiring reference pixel points in a reference row or a reference column adjacent to the current block; wherein the reference row is composed of rows adjacent to the upper side edge and the upper right side edge of the current block, and the reference column is composed of columns adjacent to the left side edge and the lower left side edge of the current block;

It should be noted that, the reference row adjacent to the current block may be formed by a row adjacent to the upper side edge and the upper right side edge of the current block, and the reference column adjacent to the current block may be formed by a column adjacent to the left side edge and the lower left side edge of the current block; the reference row or reference column adjacent to the current block may refer to a reference row adjacent to an upper side of the current block, may refer to a reference column adjacent to a left side of the current block, and may even refer to a reference row or reference column adjacent to other sides of the current block. For convenience of description, in the embodiment of the present application, the reference row adjacent to the current block will be described by taking the above reference row example adjacent to the side, and the reference column adjacent to the current block will be described by taking the reference column adjacent to the left side as an example.

The reference pixel points in the reference row adjacent to the current block may include reference pixel points adjacent to an upper side edge and an upper right side edge (also referred to as adjacent reference pixel points corresponding to the upper side edge and the upper right side edge), where the upper side edge represents an upper side edge of the current block, and the upper right side edge represents a side length of the current block, which is horizontally extended to the right of the upper side edge of the current block and is the same as the height of the current block; the reference pixel points in the reference column adjacent to the current block may further include reference pixel points adjacent to a left side edge and a left lower side edge (also referred to as adjacent reference pixel points corresponding to the left side edge and the left lower side edge), where the left side edge represents the left side edge of the current block, and the left lower side edge represents a side length of the current block, which is vertically extended downward from the left side edge, and is the same as the width of the current decoding block; however, the embodiment of the present application is not particularly limited.

Thus, when the left side neighboring region and the left lower side neighboring region are effective regions, the neighboring reference pixels at this time may be composed of reference pixel points in the reference column neighboring the current block; when the upper side neighboring region and the upper right side neighboring region are effective regions, the neighboring reference pixels at this time may be composed of reference pixel points in a reference line neighboring the current block.

S302: constructing a subset of adjacent reference pixels from the adjacent reference pixels; wherein the adjacent reference pixel subset includes a part of pixels in the adjacent reference pixels;

it should be noted that, the neighboring reference pixels are the reference pixels corresponding to the prediction model constructed in the current related technical scheme. In general, in order to avoid the transmission of model parameters from the video encoding system to the video decoding system, adjacent reference pixels may be formed using reference pixel points adjacent to one or more of four sides, i.e., an upper side, an upper right side, a left side, a lower left side, of the current block, to derive model parameters. However, the complexity of multi-model construction using so many reference pixels as sample points is high, and some reference pixels having outliers may also degrade the quality of the prediction model.

That is, in the adjacent reference pixels, there may be some unimportant reference pixels (such as those having poor correlation) or some abnormal reference pixels, and in order to ensure accuracy of model parameter derivation, the reference pixels need to be removed, so as to obtain an adjacent reference pixel subset; in this way, according to the adjacent reference pixel subsets, the accuracy of the prediction model can be ensured, so that the prediction efficiency of the image component to be processed is high.

The adjacent reference pixel subsets comprise a preset number of reference pixel points. Here, the preset number may be N, N being a positive integer greater than 1. In practical application, the number of N may be 4, but the embodiment of the present application is not limited specifically.

In some embodiments, for S302, constructing a subset of adjacent reference pixels from the adjacent reference pixels may include:

determining a candidate location of the candidate pixel based on at least one edge of the current block;

and selecting reference pixels corresponding to the candidate positions from the adjacent reference pixels, and forming the selected reference pixels into the adjacent reference pixel subset.

Further, the determining the candidate location of the candidate pixel based on at least one edge of the current block may include:

and determining the candidate position based on the pixel position corresponding to the reference pixel adjacent to at least one side of the current block.

and determining the candidate position based on the image component intensity value corresponding to the reference pixel adjacent to at least one edge of the current block.

and determining the candidate position based on the pixel position and the image component intensity value corresponding to the reference pixel adjacent to at least one side of the current block.

It should be noted that, the image component intensities may be represented by image component values, such as luminance values, chrominance values, and the like; here, the larger the image component value, the higher the image component intensity is indicated. The reference pixel selected by the embodiment of the application can be selected by the candidate position of the candidate pixel; the candidate position may be determined according to a pixel position, or may be determined according to an intensity value (such as a luminance value, a chrominance value, etc.) of an image component, which is not specifically limited in the embodiment of the present application.

S303: calculating model parameters of a prediction model according to the adjacent reference pixel subsets; the prediction model comprises N prediction sub-models, the N prediction sub-models correspond to N groups of model parameters, the prediction sub-models are used for performing cross-component prediction processing on the image component to be predicted through the corresponding model parameters, and N is a positive integer greater than or equal to 2.

It should be noted that, according to the adjacent reference pixel subsets, multiple sets of model parameters may be obtained by calculation; according to each group of model parameters, a predictor model can be constructed; thus, if N sets of model parameters are calculated, N predictor models may be obtained.

It should also be noted that the prediction model may be a nonlinear model or a complex model. The complex model may be a nonlinear model in a nonlinear form such as a quadratic curve, or may be a multi-model composed of a plurality of linear models. In this way, for the complex model, since the non-important reference pixel points or the partially abnormal reference pixel points are already removed from the adjacent reference pixel subsets, the N groups of model parameters determined according to the adjacent reference pixel subsets are more accurate, thereby improving the accuracy of the complex model and the prediction accuracy of the image components to be processed.

The embodiment provides an image component prediction method, which is implemented by determining adjacent reference pixels of a current block in an image; constructing a subset of adjacent reference pixels from the adjacent reference pixels; wherein the adjacent reference pixel subset comprises part of pixels in the adjacent reference pixels; calculating model parameters of a prediction model according to the adjacent reference pixel subsets; the prediction model comprises N prediction sub-models, the N prediction sub-models correspond to N groups of model parameters, the prediction sub-models are used for performing cross-component prediction processing on image components to be predicted through the corresponding model parameters, and N is a positive integer greater than or equal to 2; in this way, as the screening process is carried out on the adjacent reference pixels of the current block, unimportant reference pixel points or abnormal reference pixel points can be removed, so that the number of pixels in the adjacent reference pixel set is reduced, the number of pixels in the adjacent reference pixel subset is smaller, the calculation complexity and the memory bandwidth are reduced, and the accuracy of the prediction model is improved; in addition, cross-component prediction processing is carried out on the image component to be processed through at least two predictor models, so that the prediction accuracy of the image component to be processed is further improved, and meanwhile, the prediction efficiency of the video image is improved.

Further, in the obtained adjacent reference pixels, some unimportant reference pixels or abnormal reference pixels may be included, and these reference pixels may affect calculation of model parameters, so as to affect accuracy of the prediction model. At this time, a part of reference pixel points can be selected from adjacent reference pixels to form an adjacent reference pixel subset, and model parameters are calculated according to the adjacent reference pixel subset.

It should be noted that, the portion of the reference pixel points selected in the embodiment of the present application may be selected by the pixel position corresponding to the reference pixel, or may be selected according to the image component intensity value (such as the luminance value, the chrominance value, etc.) corresponding to the reference pixel, which is not particularly limited in the embodiment of the present application. The method comprises the steps of selecting proper reference pixel points by screening adjacent reference pixels through pixel positions corresponding to the reference pixels or through image component intensity values corresponding to the reference pixels, and further forming an adjacent reference pixel subset; in this way, the model parameters derived from the subset of adjacent reference pixels are more accurate, so that the prediction model constructed from the model parameters can also be more accurate.

In some embodiments, the determining the candidate location of the candidate pixel based on at least one edge of the current block may include:

determining the number of preset candidate pixels; wherein the preset candidate pixel number represents the number of pixels sampled from reference pixels adjacent to at least one edge of the current block;

determining the candidate position according to the preset candidate pixel number and the length of at least one edge of the current block; wherein the length of at least one edge of the current block is equal to the number of pixels contained in at least one edge of the current block.

It should be noted that, the preset candidate pixel number indicates a preset number of pixels to be sampled, that is, the number of pixels included in the adjacent reference pixel subset. Taking the pixel position as an example, after determining the preset candidate pixel number, calculating the candidate position of the candidate pixel according to the side length of at least one side and the preset candidate pixel number; and selecting proper reference pixel points from adjacent reference pixels according to the candidate positions to form an adjacent reference pixel subset. Therefore, the model parameters calculated according to the adjacent reference pixel subsets are more accurate, so that the constructed prediction model can be more accurate, the prediction accuracy of the image component to be predicted is further improved, and the prediction efficiency of the video image is improved.

Further, for determining the candidate position, a first sampling interval may be calculated, and then sampling processing is performed on the at least one edge according to the first sampling interval, so as to determine the candidate position of the candidate pixel corresponding to the at least one edge. Thus, in some embodiments, the determining the candidate location of the candidate pixel based on at least one edge of the current block may include:

calculating a first sampling interval according to the preset candidate pixel number and the length of at least one edge of the current block;

a reference point is determined from at least one edge of the current block, and the candidate position is determined according to the first sampling interval.

It should be noted that, the reference point may be a midpoint of the at least one edge, or may be a first reference pixel position of the at least one edge that is far to the left, or may be a first reference pixel position of the at least one edge that is far to the right, or may even be another reference pixel position of the at least one edge.

Specifically, a midpoint of the at least one edge may be determined according to the length of the at least one edge, and then the midpoint of the at least one edge is taken as the reference point. The reference point may be a midpoint of the at least one edge, or a first reference pixel position of the at least one edge that is far to the left of the midpoint, or a first reference pixel position of the at least one edge that is far to the right of the midpoint, or even other reference pixel positions of the at least one edge.

It should be noted that, considering that the importance of the reference pixel point adjacent to at least one edge of the current block is related to the corresponding position, in order to enable the reference pixel point in the adjacent reference pixel subset to represent the characteristic of the whole adjacent edge, the reference pixel point located at the center of the edge needs to be selected as far as possible to reject the point with relatively low importance (such as the reference pixel points at the edges of both sides of the edge). In the embodiment of the present application, if the upper side of the current block is taken as an example for explanation, the position of the first reference pixel point with the middle position being far to the right or far to the left can be taken as the datum point of the side; if the left side of the current block is taken as an example, the first reference pixel point position with the middle position being lower or upper can be taken as the datum point of the side.

Besides, before determining the reference point, a preset number of reference pixel points corresponding to the end position of one side of the current block may be deleted, or initial offset is performed on the side from the end position according to a preset offset, the offset reference pixel point position is used as a starting point to obtain a new side, and then an intermediate position corresponding to the new side is used as the reference point; correspondingly, a preset number of reference pixel points corresponding to the initial position of one side of the current block can be deleted first, or initial offset is performed on the side from the initial position according to a preset offset, the offset reference pixel point position is used as a starting point, a new side is obtained, and then the middle position corresponding to the new side is used as a reference point.

In practical application, since the side length of the left side or the upper side of the current block is an integer multiple of 2, the middle position of the left side or the upper side of the current block is between two points. In the example of fig. 4, the first pixel point to the left of the middle position is taken as the midpoint of the edge; however, in the embodiment of the present application, the first pixel point with the middle position being far to the right may be used as the midpoint of the edge, as shown in fig. 5. In fig. 4, the first pixel point (e.g. 3 in fig. 4) with the middle position far to the left is taken as the midpoint of the edge, and since the preset sampling number is 2, the positions (e.g. gray points in fig. 4) of the reference pixels to be selected can be determined to be 1 and 5, and the reference pixels corresponding to the positions can be selected according to the positions of the reference pixels, so that adjacent reference pixel subsets can be formed. Therefore, in the embodiment of the present application, for the upper side edge of the current block, the first pixel point with the right middle position may be taken as the midpoint of the edge, or the first pixel point with the left middle position may be taken as the midpoint of the edge, which is not particularly limited; in addition, for the left side edge of the current block, the first pixel point with the lower middle position may be taken as the midpoint of the edge, or the first pixel point with the upper middle position may be taken as the midpoint of the edge, and the embodiment of the present application is not limited specifically.

The present application will be described with reference to the following embodiments, but the present application is not limited to the embodiments, and the image component prediction method according to the present application is also applicable to the left side of the current block, even the right side of the reconstructed block or the lower side of the reconstructed block.

It will be appreciated that if the existence of reference pixel points adjacent to the left or upper side of the current block is not considered, the construction of the second reference pixel set may also be performed according to equations (1) and (2),

Δ＝length/(N ₂ /2) (1)

shift＝Δ/2 (2)

where Δ denotes the sampling interval, length denotes the number of a row of reference pixel points adjacent to the upper side of the current block or the number of a column of reference pixel points adjacent to the left side of the current block, N ₂ Representing the number of reference pixel points (generally, left and top sidesEach is one half, but the embodiment of the present application is not limited in detail, shift indicates the position of the start point of the selected reference pixel point. Here, when the middle position of the left side or the upper side of the current block is between two points, if the first pixel point whose middle position is right is taken as the midpoint of the side, then the start point position shift=Δ/2; at this time, if the first pixel point whose intermediate position is far to the left is taken as the midpoint of the edge, the start point position shift=Δ/2-1.

Illustratively, taking the upper side as shown in FIG. 4 as an example, length is equal to 8, N ₂ Equal to 4, assuming that the left side and the upper side are each one half, i.e., the preset number of samples of the upper side is 2, Δ=length/(N) is calculated according to equations (1) and (2), respectively ₂ First, the position of the reference pixel point to be selected, for example, 1 and 5, can be determined, and then the corresponding reference pixel point can be selected to form the adjacent reference pixel subset. Here, it should be noted that the preset number of samples corresponding to the left side and the preset number of samples corresponding to the upper side may be the same or different, which is not particularly limited in the embodiment of the present application.

It should be further noted that, according to the preset number of candidate pixels and the length of an edge of the current block, the first sampling interval corresponding to the edge may be calculated. In addition, since the side lengths of the left side edge or the upper side edge of the current block are all integral multiples of 2, the middle positions of the left side edge or the upper side edge of the current block are located between two points, the calculated midpoint value is a non-integer, and the calculated reference pixel point position is also a non-integer; however, if the side length of the left side or the upper side of the current block is not an integer multiple of 2, the middle position of the left side or the upper side of the current block is not located between two points, and at this time, the calculated midpoint value is an integer, and the calculated reference pixel point position is also an integer; that is, the calculated midpoint value may be an integer or a non-integer; correspondingly, the calculated reference pixel point position can be an integer or a non-integer; the embodiment of the present application is not particularly limited.

In this way, when the calculated midpoint value is an integer, correspondingly, the calculated reference pixel point position is also an integer, and at this time, the calculated reference pixel point position can be directly used as a candidate position; when the calculated midpoint value is a non-integer, correspondingly, the calculated reference pixel point position is also a non-integer, and the candidate position can be determined by rounding to a large value or rounding to a small value.

Further, in some embodiments, after calculating the first sampling interval, the method may further comprise:

adjusting the first sampling interval to obtain a second sampling interval;

the candidate locations are determined at the second sampling interval based on the fiducial points.

It should be noted that, after the first sampling interval is calculated, the first sampling interval may be further fine-tuned, for example, the first sampling interval is increased by 1 or decreased by 1, so as to obtain the second sampling interval. For example, the first sampling interval is 4, and then the adjusted second sampling interval may be 3 or 5. In the embodiment of the present application, for the adjustment of the first sampling interval, an adjustment with a small amplitude (for example, 1 is added or 1 is subtracted) may be performed, but for the specific setting of the adjustment amplitude, the embodiment of the present application is not particularly limited.

Further, in some embodiments, after the second sampling interval is obtained, the method may further include:

and determining candidate positions corresponding to one side of the datum point according to the first sampling interval based on the datum point, and determining candidate positions corresponding to the other side of the datum point according to the second sampling interval.

That is, after determining the reference point of at least one edge of the current block, uniform sampling may be performed at the first sampling interval or the second sampling interval; the non-uniform sampling can be performed according to the first sampling interval and the second sampling interval, and the candidate positions determined after the sampling can be symmetrically distributed on two sides of the datum point or asymmetrically distributed on two sides of the datum point; the embodiment of the present application is not particularly limited.

Further, since the reference pixel points located at the middle position of at least one side are related to the to-be-predicted image component of the current block in the adjacent reference pixels in a comparison manner, the reference pixel points with the continuous preset sampling number near the middle position can be used as the to-be-selected reference pixel point positions, and the method can be called a middle position continuous point taking scheme. Specifically, assuming that the reference pixel point positions on a row/column adjacent to the upper side or the left side of the current block are numbered from 0, the number of adjacent reference pixel points in the adjacent reference pixel subset formed in the present embodiment and the corresponding reference pixel point positions to be selected are shown in table 1. At this time, the positions of the reference pixel points with the continuous preset sampling number near the middle position can be used as candidate positions, so that adjacent reference pixel subsets are formed.

TABLE 1

Side length of at least one side of current block	Candidate locations	Presetting candidate pixel number
			2	0,1	2
4	1,2	2
			8	2,3,4 (or 3,4, 5)	3
16	6,7,8,9	4
			32	13,14,15,16,17,18,19,20	8

Further, for the screening process of the adjacent reference pixels, the jump point process can be further performed on the reference pixel points of at least one edge, namely, the unimportant reference pixel points or the reference pixel points with the abnormality are skipped (can also be regarded as the deletion process), so as to obtain the adjacent reference pixel subset; the filtering process can be performed on the rest of the reference pixel points after the skipping of the partial reference pixel points of at least one side so as to obtain the adjacent reference pixel subsets. Thus, in some embodiments, the determining the candidate location of the candidate pixel based on at least one edge of the current block may include:

determining a preset skipped pixel number K corresponding to at least one edge of the current block, wherein K is a positive integer greater than or equal to 1;

starting from the initial position and/or the tail end position of at least one edge of the current block, determining the positions corresponding to K pixel points to be skipped;

continuously skipping K pixel points to be skipped from the starting position and/or the end position of at least one edge of the current block based on the positions corresponding to the K pixel points to be skipped, so as to obtain at least one new edge of the current block;

And determining the candidate position based on at least one new edge of the current block and the preset candidate pixel number.

It should be noted that the preset skipped pixel number indicates a preset number of pixels to be deleted or skipped. In addition, the start position of at least one side represents the leftmost edge position of the upper side of the current block or the leftmost edge position of the left side of the current block, and the end position of at least one side represents the rightmost edge position of the upper side of the current block or the bottommost edge position of the left side of the current block.

It should be further noted that the value of K may be the number of preset reference pixel points, for example, 1, 2, or 4; or the current block is obtained by calculation according to the side length of the current block and the preset proportion corresponding to the current block; however, in practical application, the setting is still performed according to practical situations, and the embodiment of the present application is not limited specifically. The preset proportion corresponding to the upper side of the current block may be represented by a first preset proportion, the preset proportion corresponding to the left side of the current block may be represented by a second preset proportion, and the values of the first preset proportion and the second preset proportion may be the same or different.

Thus, assuming that from the starting position of at least one edge, if at least one edge is the upper side edge of the current block (may also be referred to as the reference line of the current block), the positions corresponding to the K pixels to be skipped may be determined from the leftmost edge position of at least one edge; if at least one edge is the left edge of the current block (also referred to as the reference column of the current block), determining the positions corresponding to K pixels to be skipped from the uppermost edge position of the at least one edge; assuming that the position starts from the tail end of at least one edge, if the at least one edge is the upper side of the current block, the positions corresponding to K pixels to be skipped can be determined from the rightmost edge position of the at least one edge; if at least one edge is the left edge of the current block, determining the positions corresponding to K pixels to be skipped from the position of the lowest edge of the at least one edge; in practical application, the setting is performed according to practical situations, and the embodiment of the application is not particularly limited.

After determining the positions corresponding to the K pixels to be skipped, assuming that starting from the initial position of at least one side, if at least one side is the upper side of the current block, starting from the leftmost edge position of the upper side, determining the positions corresponding to the K continuous pixels to be skipped to the right, and then continuously skipping the K pixels to be skipped to obtain a new upper side; at this time, according to the side length of the new upper side and the preset candidate pixel number, the candidate position corresponding to the new upper side can be determined, so that the selected candidate pixels form an adjacent reference pixel subset; if at least one edge is the left edge of the current block, determining the positions corresponding to the continuous K pixels to be skipped downwards from the uppermost edge position of the left edge, and then continuously skipping the K pixels to be skipped to obtain a new left edge; at this time, the candidate position corresponding to the new left side edge can be determined according to the side length of the new left side edge and the preset candidate pixel number, so that the selected candidate pixels form an adjacent reference pixel subset. Or, assuming that from the end position of at least one edge, if at least one edge is the upper edge of the current block, starting from the rightmost edge position of the upper edge, determining positions corresponding to continuous K pixels to be skipped to the left, and then continuously skipping the K pixels to be skipped to obtain a new upper edge; at this time, according to the side length of the new upper side and the preset candidate pixel number, the candidate position corresponding to the new upper side can be determined, so that the selected candidate pixels form an adjacent reference pixel subset; if at least one edge is the left edge of the current block, starting from the lowest edge position of the left edge, determining positions corresponding to continuous K pixels to be skipped upwards, and then continuously skipping the K pixels to be skipped to obtain a new left edge; at this time, the candidate position corresponding to the new left side edge can be determined according to the side length of the new left side edge and the preset candidate pixel number, so that the selected candidate pixels form an adjacent reference pixel subset.

Thus, the embodiment of the application uses part of pixels (i.e. the adjacent reference pixel subset) in the adjacent reference pixels obtained by the adjacent reference pixel points of the current block to derive the model parameters corresponding to the complex model (such as a nonlinear model or a multi-model). Because unimportant reference pixel points or abnormal reference pixel points exist in the obtained subset (namely the adjacent reference pixel subset), the obtained subset has fewer reference pixel points, so that the calculation complexity and the memory bandwidth are reduced, the accuracy of a complex model is improved, and the purposes of improving the prediction accuracy of the image components to be processed and the prediction efficiency of the video image can be achieved.

Further, after determining the adjacent reference pixel subset, model parameters of the prediction model may be calculated according to the adjacent reference pixel subset to construct the prediction model; the prediction model may include N predictor models, where N is a positive integer greater than or equal to 2. Thus, in some embodiments, for S303, the calculating model parameters of the prediction model according to the adjacent reference pixel subset may include:

constructing N first adjacent reference pixel subsets according to the adjacent reference pixel subsets;

Calculating N groups of model parameters corresponding to the N predictor models based on the N first adjacent reference pixel subsets; wherein each predictor model corresponds to a set of model parameters.

Further, constructing N first adjacent reference pixel subsets from the adjacent reference pixel subsets may include:

determining at least one threshold from the subset of adjacent reference pixels;

the subset of adjacent reference pixels is divided into the N first subsets of adjacent reference pixels according to the at least one threshold.

It should be noted that the threshold is a basis for classifying the reference pixel points included in the adjacent reference pixel subset, and is also a classification basis for the first image component reconstruction value of the current block. In addition, the threshold is also a set value for indicating that a plurality of predictor models are established, and the magnitude of the threshold is related to the first image component reconstruction value corresponding to all pixel points in the current block.

Specifically, the threshold value may be obtained by calculating the average value of the first image component reconstruction values corresponding to all the pixel points in the current block, or may be obtained by calculating the median value of the first image component reconstruction values of all the pixel points in the current block, which is not particularly limited in the embodiment of the present application.

In the embodiment of the present application, first, according to the first image component reconstruction values corresponding to all the pixel points in the current block and equation (3), a Mean value Mean may be calculated:

wherein Mean represents the average value of the reconstructed brightness values corresponding to all the pixel points in the current block, ΣRec _L [i,j]And representing the sum of the reconstructed brightness values corresponding to all the pixel points in the current block, and M represents the sampling number of the reconstructed brightness values corresponding to all the pixel points in the current block.

And secondly, directly taking the calculated Mean as a threshold value, and dividing the second reference pixel set into two groups by using the threshold value, so that two predictor models can be built, but the embodiment of the application is not limited to only building the two predictor models.

In this way, after the Mean is calculated according to the first image component reconstruction values corresponding to all the pixel points in the current block, if two predictor models need to be built, the Mean can be directly used as a threshold value, and the reference pixel points in the adjacent reference pixel subsets can be divided into two groups according to the threshold value, so that the two predictor models can be built later; if three predictor models need to be built, then (first image component minimum reconstruction value+mean+1) > >1 may be used as a first threshold value, and (first image component maximum reconstruction value+mean+1) > >1 may be used as a second threshold value, according to which the reference pixel points in the second reference pixel set may be divided into three groups, indicating that three predictor models may be built subsequently.

Further, in some embodiments, after said calculating model parameters of the prediction model from said subset of adjacent reference pixels, the method may further comprise:

determining a first image component reconstruction value corresponding to each pixel in the current block;

selecting one predictor model from the N predictor models;

calculating a second image component predicted value corresponding to each pixel in the current block according to the selected predictor model and the first image component reconstructed value corresponding to each pixel in the current block; wherein the image component to be predicted is a second image component.

Further, the selecting one predictor model from the N predictor models may include:

comparing a first image component reconstruction value corresponding to each pixel in the current block with the at least one threshold;

and selecting a predictor model corresponding to each pixel in the current block from the N predictor models according to the comparison result.

It should be noted that, at least two sets of reference pixel subsets may be obtained by grouping adjacent reference pixel subsets according to at least one threshold, and then determining at least two sets of model parameters. Specifically, the Mean value Mean may be directly used as a threshold value, and then a first image component adjacent reference value corresponding to each reference pixel point in the adjacent reference pixel subsets is compared with the Mean value Mean, for example, the reference pixel points with the first image component adjacent reference value greater than Mean form a first reference pixel subset, and the reference pixel points with the first image component adjacent reference value not greater than Mean form a second reference pixel subset, so as to obtain two groups of reference pixel subsets; determining a first group of model parameters alpha 1 and beta 1 according to the reference pixel points in the first reference pixel subset; and determining a second set of model parameters alpha 2 and beta 2 from the reference pixel points in the second subset of reference pixels.

It should be further noted that after at least two predictive sub-models are established, the first image component reconstruction value corresponding to each pixel point in the current block may be compared with at least one threshold value, and the predictive sub-model corresponding to each pixel point is selected from the at least two predictive sub-models according to the comparison result; and then, carrying out prediction processing on the image component to be predicted according to the selected predictor model, thereby obtaining a predicted value of the image component to be predicted.

Illustratively, assuming that a first image component neighboring reference value corresponding to a reference pixel point neighboring the current block is represented by L (n), a second image component neighboring reference value corresponding to a reference pixel point neighboring the current block is represented by C (n), and a Threshold value is represented by Threshold; fig. 6 is a schematic diagram of grouping adjacent reference pixels of a current block according to an embodiment of the present application. In fig. 6, with the threshold value as a demarcation point, all reference pixel points adjacent to the current block may be divided into two parts, such as a first reference pixel subset and a second reference pixel subset. Taking the Mean value Mean of the first image component reconstruction values of all pixel points in the current block as a Threshold value, namely threshold=mean; thus, if L (n) is less than or equal to Threshold, the reference pixel point corresponding to L (n) can be judged as a first reference pixel subset, and then a first image component adjacent reference value L (m) and a second image component adjacent reference value C (m) corresponding to all the reference pixel points in the first reference pixel subset are obtained; if L (n) > Threshold, the reference pixel point corresponding to L (n) can be determined as the second reference pixel subset, so as to obtain a first image component adjacent reference value L (k) and a second image component adjacent reference value C (k) corresponding to all the reference pixel points in the second reference pixel subset.

In this way, after the first image component adjacent reference value L (m) and the second image component adjacent reference value C (m) corresponding to all the reference pixel points in the first reference pixel subset are obtained, they can be used as separate training sets, so as to train out the first group of model parameters α1 and β1; after the first image component adjacent reference value L (k) and the second image component adjacent reference value C (k) corresponding to all the reference pixel points in the second reference pixel subset are obtained, the first image component adjacent reference value L (k) and the second image component adjacent reference value C (k) can also be used as separate training sets, so that a second group of model parameters alpha 2 and beta 2 are trained; specifically, from L (m), C (m), and equation (4), a first set of model parameters α1 and β1 are calculated, as follows,

from L (k), C (k) and equation (5), a second set of model parameters, alpha 2 and beta 2, are calculated, as follows,

after obtaining the first set of model parameters α1 and β1 and the second set of model parameters α2 and β2, respectively, a first predictor model Pred may also be built according to equation (6) _1C [i,j]And a second predictor model Pred _2C [i,j]As shown below, the present invention provides,

wherein M represents the number of reference pixel points in the first reference pixel subset, K represents the number of reference pixel points in the second reference pixel subset, i and j represent the position coordinates of each pixel point in the current block, i represents the horizontal direction, and j represents the vertical direction; threshold represents a Threshold value, which may be obtained by averaging the first image component reconstruction values corresponding to all the pixel points in the current block; rec _L [i,j]Indicating that the position coordinates of the current block are [ i, j ]]A first image component reconstruction value corresponding to a pixel point of (a); pred _1C [i,j]Indicating that the position coordinates of the current block are [ i, j ]]A second image component predicted value, pred, obtained by using the first predictor model for the pixel points of (2) _2C [i,j]Indicating that the position coordinates of the current block are [ i, j ]]And a second image component predicted value obtained by using the second predictor model.

In particular, for complex models (e.g. comprising two or more linear models), the prediction mode of the MMLM may be used at this time for the prediction processing of the image component to be predicted. For the current block, the pixels of different position coordinates can select the predictor model pairs corresponding to different model parameters to construct the predicted value. For example, assume that there are two linear models and there are two pixels m and n in the current block, the two pixels having two different reconstructed luminance values and the two reconstructed luminance values being within two different intervals.

Thus, the pixel m can obtain a chromaticity predicted value using the prediction model shown in the formula (7),

C' _m ＝α1×Y _m +β1 (7)

wherein α1 and β1 are a set of model parameters, Y _m Is the brightness reconstruction value corresponding to the pixel point m, C' _m Is the chroma prediction value corresponding to the pixel point m.

Pixel n may use the prediction model shown in equation (8) to obtain a chrominance prediction value,

C' _n ＝α2×Y _n +β2 (8)

wherein α2 and β2 are a set of model parameters, Y _n Is the brightness reconstruction value corresponding to the pixel point n, C' _n Is the chroma prediction value corresponding to the pixel point n.

Besides, the embodiment of the application can also construct a more complex prediction model (such as a more complex nonlinear model or a more complex multimode model) for realizing the purpose of cross-component prediction or cross-component prediction, for example, the chroma value can be predicted according to the acquired reconstructed luma value, the luma value can be predicted according to the acquired reconstructed chroma value, and even the prediction can be performed among different chroma values.

The embodiment provides an image component prediction method, which elaborates the specific implementation of the foregoing embodiment, and by adopting the technical scheme of the foregoing embodiment, it can be seen that, as the adjacent reference pixels of the current block are screened, unimportant reference pixel points or abnormal reference pixel points can be removed, so that the number of samples required for deriving the model parameters in cross-component prediction is reduced, the computational complexity and the memory bandwidth are reduced, the prediction model is optimized, and the accuracy of the prediction model is improved; in addition, the method is mainly used for optimizing the deduction process of model parameters in a complex model (a nonlinear model or a multi-model), and the prediction processing of the image component to be predicted is realized by utilizing at least two predictor models, so that the prediction accuracy of the image component to be predicted is improved, and meanwhile, the prediction efficiency of the video image is improved.

Referring to fig. 7, a flowchart of another image component prediction method according to an embodiment of the present application is shown. As shown in fig. 7, the method may include:

s701: determining a first reference pixel set of a first image component of a current block in an image, wherein the first reference pixel set contains pixels adjacent to the current block;

s702: constructing N first reference pixel subsets by using a first reference pixel set, wherein the first reference pixel subsets comprise partial pixels in the first reference pixel set, and N is equal to the number of prediction models;

s703: and respectively calculating model parameters of N prediction models by using the N first reference pixel subsets, wherein the prediction models are used for mapping a first image component value of the current block into a predicted value of a second image component of the current block, and the second image component is different from the first image component.

It should be noted that the video image may be divided into a plurality of image blocks, and each of the image blocks to be currently encoded may be referred to as an encoding block. Wherein each encoding block may include a first image component, a second image component, and a third image component; and the current block is a coding block in the video image, which is currently to be predicted for the first image component, the second image component or the third image component. When the first image component is required to be predicted through the prediction model, the image component to be predicted is the first image component; when the second image component is required to be predicted through the prediction model, the image component to be predicted is the second image component; when the third image component needs to be predicted through the prediction model, the image component to be predicted is the third image component. In the embodiment of the present application, the image component to be predicted is described by taking the second image component as an example, and the first image component value of the current block can be mapped to the predicted value of the second image component of the current block through the prediction model.

It should also be noted that the prediction model may be a nonlinear model or a complex model. The complex model may be a nonlinear model in a nonlinear form such as a quadratic curve, or may be a multi-model composed of a plurality of linear models. In this way, for a complex model, since there may be some unimportant reference pixel points (such as poor correlation of the reference pixel points) or some abnormal reference pixel points in the adjacent reference pixels of the current block, in order to ensure accuracy of model parameter derivation, the reference pixel points need to be removed, so as to construct a reference pixel subset, so that model parameters determined according to the reference pixel subset are more accurate, not only is accuracy of the complex model improved, but also prediction accuracy of image components to be processed is improved.

The embodiment provides an image component prediction method, which comprises the steps of determining a first reference pixel set of a first image component of a current block in an image, wherein the first reference pixel set comprises pixels adjacent to the current block; constructing N first reference pixel subsets by using a first reference pixel set, wherein the first reference pixel subsets comprise partial pixels in the first reference pixel set, and N is equal to the number of the prediction models; respectively calculating model parameters of N prediction models by utilizing the N first reference pixel subsets, wherein the prediction models are used for mapping a first image component value of the current block into a predicted value of a second image component of the current block, and the second image component is different from the first image component; in this way, as the screening process is carried out on the adjacent reference pixels of the current block, unimportant reference pixel points or abnormal reference pixel points can be removed, so that the number of pixels in the adjacent reference pixel set is reduced, the number of pixels in the adjacent reference pixel subset is smaller, the calculation complexity and the memory bandwidth are reduced, and the accuracy of the prediction model is improved; in addition, cross-component prediction processing is carried out on the image component to be processed through at least two predictor models, so that the prediction accuracy of the image component to be processed is further improved, and meanwhile, the prediction efficiency of the video image is improved.

Further, in some embodiments, for S702, constructing N first reference pixel subsets using the first reference pixel set may include:

determining at least one threshold from the first set of reference pixels;

the first set of reference pixels is divided into the N first subsets of reference pixels according to the at least one threshold.

The value of N is a positive integer greater than or equal to 2.

It should be further noted that the threshold is a basis for classifying the reference pixel points included in the adjacent reference pixel subset, and is also a classification basis for the reconstructed value of the first image component of the current block. In addition, the threshold is also a set value for indicating that a plurality of predictor models are established, and the magnitude of the threshold is related to the first image component reconstruction value corresponding to all pixel points in the current block. Specifically, the threshold value may be obtained by calculating the average value of the first image component reconstruction values corresponding to all the pixels in the current block, or may be obtained by calculating the median value of the first image component reconstruction values of all the pixels in the current block.

In some embodiments, for S701, the determining the first set of reference pixels with the first image component of the current block in the image may include:

obtaining the first reference pixel set according to part or all of pixels on the edge of the current block; wherein the edge of the current block comprises at least one of: upper adjacent rows, upper right adjacent rows, left adjacent columns, and lower left adjacent columns.

It should be noted that, at least one edge of the current block may include at least one of the following: up (also referred to as an upper side), right up (also referred to as an upper right side), left column (also referred to as a left side), or left down (also referred to as a lower left side). Wherein the first set of reference pixels may be composed based on some or all of the pixels on these sides.

Further, since there may be some unimportant reference pixel points (such as poor correlation of the reference pixel points) or some abnormal reference pixel points in the adjacent reference pixels of the current block, the adjacent reference pixels need to be screened at this time, and the screened candidate pixels form the first reference pixel set. Thus, in some embodiments, the obtaining the first reference pixel set according to some or all pixels on the edge of the current block may include:

Determining candidate positions of the candidate pixels on the edge of the current block;

and obtaining the first reference pixel set according to the pixels of the candidate positions.

Further, the determining the candidate location of the candidate pixel on the edge of the current block may include:

and determining the candidate position according to the pixel position corresponding to the pixel on the edge of the current block.

and determining the candidate position according to a first image component value corresponding to the pixel on the edge of the current block.

and determining the candidate position according to the pixel position corresponding to the pixel on the edge of the current block and the first image component value corresponding to the pixel on the edge of the current block.

It should be noted that, the first image component value is mainly used to characterize the image component intensity of the pixel, such as a luminance value, a chrominance value, and the like; here, the larger the image component value, the higher the image component intensity is indicated.

It should be further noted that, the reference pixel selected in the embodiment of the present application may be selected by a candidate position of the candidate pixel; the candidate position may be determined according to the pixel position, may be determined according to a first image component value (such as a luminance value, a chrominance value, etc.), or may be determined jointly according to the pixel position and the first image component value, which is not limited in particular. That is, the part of the reference pixel points selected in the embodiment of the present application may be selected by the pixel position corresponding to the reference pixel, or may be selected according to the intensity value (such as the luminance value, the chrominance value, etc.) of the image component corresponding to the reference pixel; selecting proper reference pixel points by screening adjacent reference pixels through pixel positions corresponding to the reference pixels or through image component intensity values corresponding to the reference pixels, and further forming a first reference pixel set; in this way, after the first reference pixel set is divided into N first reference pixel subsets, the model parameters derived from the N first reference pixel subsets are more accurate, so that the prediction model constructed according to the model parameters can also be more accurate.

Further, for determining the candidate position, a first sampling interval may be calculated, and then sampling processing is performed on the at least one edge according to the first sampling interval, so as to determine the candidate position of the candidate pixel corresponding to the at least one edge. Thus, in some embodiments, said determining the candidate location of the candidate pixel on the edge of the current block may comprise:

determining a preset candidate pixel number, wherein the preset candidate pixel number indicates the pixel number selected from the edge of the current block;

determining the candidate position according to the preset pixel number and the length of the edge of the current block; wherein the length of the edge of the current block is equal to the number of pixels on the edge of the current block.

It should be noted that, the preset candidate pixel number indicates a preset number of pixels to be sampled, that is, the number of pixels included in the adjacent reference pixel subset. Taking the pixel position as an example, after determining the preset candidate pixel number, calculating the candidate position of the candidate pixel according to the side length of at least one side and the preset candidate pixel number; and selecting proper reference pixel points from adjacent reference pixels according to the candidate positions to form an adjacent reference pixel subset.

Further, for determining the candidate position, a first sampling interval may be calculated, and then sampling processing is performed on the at least one edge according to the first sampling interval, so as to determine the candidate position of the candidate pixel corresponding to the at least one edge. Thus, in some embodiments, after said determining the number of preset candidate pixels, the method may further comprise:

and calculating a first sampling interval according to the length of the edge of the current block and the preset candidate pixel number.

Further, after said calculating the first sampling interval, the method may further comprise:

and adjusting the first sampling interval to obtain a second sampling interval.

and determining a reference point on the edge of the current block, and determining the candidate position at the first sampling interval from the reference point.

It should be noted that, after the first sampling interval is calculated, fine adjustment may be performed on the first sampling interval, for example, an operation of adding 1 or subtracting 1 from the first sampling interval is performed, so as to obtain the second sampling interval. For example, the first sampling interval is 4, and then the adjusted second sampling interval may be 3 or 5. In the embodiment of the present application, for the adjustment of the first sampling interval, an adjustment with a small amplitude (for example, 1 is added or 1 is subtracted) may be performed, but for the specific setting of the adjustment amplitude, the embodiment of the present application is not particularly limited.

Further, in some embodiments, the determining the candidate location of the candidate pixel on the edge of the current block may include:

and determining a datum point on the edge of the current block, and determining candidate positions on two sides of the datum point at the first sampling interval.

and determining a reference point on the edge of the current block, and determining the candidate position at the second sampling interval from the reference point.

And determining a datum point on the edge of the current block, and determining candidate positions on two sides of the datum point at the second sampling interval.

and determining a datum point on the edge of the current block, determining a candidate position corresponding to one side of the datum point at the first sampling interval, and determining a candidate position corresponding to the other side of the datum point at the second sampling interval.

It should be noted that, after determining the reference point of at least one edge of the current block, uniform sampling may be performed according to the first sampling interval or the second sampling interval; the non-uniform sampling can be performed according to the first sampling interval and the second sampling interval, and the candidate positions determined after the sampling can be symmetrically distributed on two sides of the datum point or asymmetrically distributed on two sides of the datum point; the embodiment of the present application is not particularly limited.

Further, for the screening process of the adjacent reference pixels, the reference pixel points of at least one side can be skipped, i.e. the reference pixel points which are not important or have abnormality are skipped (can also be regarded as deleting process), so as to obtain the adjacent reference pixel subset; the filtering process can be performed on the rest of the reference pixel points after the skipping of the partial reference pixel points of at least one side so as to obtain the adjacent reference pixel subsets. Thus, in some embodiments, the method may further comprise:

Determining a preset skipped pixel number K of the edge of the current block, wherein K is a non-negative integer;

setting a kth pixel position as the reference point from an end position of an edge of the current block; wherein the end position of the edge of the current block is the head end pixel position or the end pixel position of the edge of the current block.

Further, after determining the N first reference pixel subsets, model parameters of the prediction model may be calculated according to the N first reference pixel subsets to construct the prediction model; the prediction model may include N predictor models, where N is a positive integer greater than or equal to 2. Thus, in some embodiments, after calculating model parameters of the N prediction models using the N first reference pixel subsets, respectively, the method may further include:

determining a first image component reconstruction value corresponding to a pixel in the current block;

selecting one prediction model from the N prediction models;

and calculating a second image component predicted value corresponding to the pixel in the current block by using the selected prediction model and the first image component reconstructed value corresponding to the pixel in the current block.

Further, the selecting one prediction model from the N prediction models may include:

comparing a first image component reconstruction value corresponding to a pixel in the current block with the at least one threshold;

and selecting a prediction model used by the pixels in the current block from the N prediction models according to the comparison result.

After the N prediction models are established, the first image component reconstruction value corresponding to each pixel point in the current block may be compared with at least one threshold value, and according to the comparison result, one prediction model corresponding to each pixel point is selected from the N prediction models; and then, carrying out prediction processing on the image component to be predicted according to the selected prediction model, thereby obtaining a predicted value of the image component to be predicted.

In addition, in the embodiment of the present application, when the image component prediction method is applied to the encoder side, an adjacent reference pixel subset may be configured according to the adjacent reference pixels, and then model parameters of a prediction model are calculated according to the adjacent reference pixel subset, and the calculated model parameters are written into the code stream; the code stream is transmitted from the encoder side to the decoder side; correspondingly, when the image component prediction method is applied to a decoder side, model parameters of a prediction model can be directly obtained by analyzing a code stream; alternatively, on the decoder side, an adjacent reference pixel subset may be constructed from the adjacent reference pixels, and then model parameters of a prediction model may be calculated from the adjacent reference pixel subset, so as to construct a prediction model, and the prediction model may be used to perform cross-component prediction processing on at least one image component of the current block.

The embodiment provides an image component prediction method, which details the specific implementation of the foregoing embodiment, and by adopting the technical scheme of the foregoing embodiment, it can be seen that, due to screening processing on the adjacent reference pixels of the current block, unimportant reference pixel points or abnormal reference pixel points can be removed, so that the number of pixels in the adjacent reference pixel set is reduced, the number of pixels in the adjacent reference pixel subset is less, the computational complexity and the memory bandwidth are reduced, and the accuracy of the prediction model is improved; in addition, cross-component prediction processing is carried out on the image component to be processed through at least two predictor models, so that the prediction accuracy of the image component to be processed is further improved, and meanwhile, the prediction efficiency of the video image is improved.

Based on the same inventive concepts as the previous embodiments, referring to fig. 8, a schematic diagram of the composition structure of an encoder 80 according to an embodiment of the present application is shown. The encoder 80 may include: a first determination unit 801, a first construction unit 802 and a first calculation unit 803, wherein,

the first determining unit 801 is configured to determine adjacent reference pixels of a current block in an image;

The first construction unit 802 is configured to construct a subset of adjacent reference pixels from the adjacent reference pixels; wherein the adjacent reference pixel subset includes a part of pixels in the adjacent reference pixels;

the first calculation unit 803 is configured to calculate model parameters of a prediction model from the adjacent reference pixel subsets; the prediction model comprises N prediction sub-models, the N prediction sub-models correspond to N groups of model parameters, the prediction sub-models are used for performing cross-component prediction processing on the image component to be predicted through the corresponding model parameters, and N is a positive integer greater than or equal to 2.

In the above-described scheme, referring to fig. 8, the encoder 80 may further include a first acquisition unit 804 configured to acquire reference pixels adjacent to at least one edge of the current block; wherein at least one edge of the current block includes at least one of: uplink, right uplink, left column and left downlink; and obtaining the adjacent reference pixels according to the acquired reference pixels.

In the above-described scheme, referring to fig. 8, the encoder 80 may further include a first selection unit 805, wherein,

the first determining unit 801 is further configured to determine a candidate position of the candidate pixel based on at least one edge of the current block;

The first selecting unit 805 is configured to select a reference pixel corresponding to the candidate position from the adjacent reference pixels, and form the selected reference pixel into the adjacent reference pixel subset.

In the above aspect, the first determining unit 801 is further configured to determine the candidate position based on a pixel position corresponding to a reference pixel adjacent to at least one edge of the current block.

In the foregoing aspect, the first determining unit 801 is further configured to determine the candidate position based on an image component intensity value corresponding to a reference pixel adjacent to at least one edge of the current block.

In the above aspect, the first determining unit 801 is further configured to determine the candidate position based on a pixel position and an image component intensity value corresponding to a reference pixel adjacent to at least one edge of the current block.

In the above aspect, the first determining unit 801 is further configured to determine a preset candidate pixel number; wherein the preset candidate pixel number represents the number of pixels sampled from reference pixels adjacent to at least one edge of the current block; determining the candidate position according to the preset candidate pixel number and the length of at least one edge of the current block; wherein the length of at least one edge of the current block is equal to the number of pixels contained in at least one edge of the current block.

In the above aspect, the first calculating unit 803 is further configured to calculate a first sampling interval according to the preset number of candidate pixels and a length of at least one edge of the current block;

the first determining unit 801 is further configured to determine a reference point from at least one edge of the current block, and determine the candidate position according to the first sampling interval.

In the above-mentioned scheme, referring to fig. 8, the encoder 80 may further include a first adjusting unit 806 configured to adjust the first sampling interval to obtain a second sampling interval;

the first determining unit 801 is further configured to determine the candidate position at the second sampling interval based on the reference point.

In the above aspect, the first determining unit 801 is further configured to determine, based on the reference point, a candidate position corresponding to one side of the reference point at the first sampling interval, and determine a candidate position corresponding to the other side of the reference point at the second sampling interval.

In the above aspect, the first determining unit 801 is further configured to determine a preset skipped pixel number K corresponding to at least one edge of the current block, where K is a positive integer greater than or equal to 1; starting from the initial position and/or the tail end position of at least one edge of the current block, determining the positions corresponding to K pixel points to be skipped; continuously skipping the K pixel points to be skipped from the starting position and/or the end position of at least one edge of the current block based on the positions corresponding to the K pixel points to be skipped, so as to obtain at least one new edge of the current block; and determining the candidate position based on at least one new edge of the current block and the preset candidate pixel number.

In the above aspect, the first constructing unit 802 is configured to construct N first adjacent reference pixel subsets according to the adjacent reference pixel subsets;

the first calculating unit 803 is configured to calculate N groups of model parameters corresponding to the N predictor models based on the N first adjacent reference pixel subsets; wherein each predictor model corresponds to a set of model parameters.

In the above-described scheme, referring to fig. 8, the encoder 80 may further include a first dividing unit 807, in which,

the first determining unit 801 is further configured to determine at least one threshold value from the adjacent reference pixel subset;

the first dividing unit 807 is configured to divide the adjacent reference pixel subset into the N first adjacent reference pixel subsets according to the at least one threshold.

In the above aspect, the first determining unit 801 is further configured to determine a first image component reconstruction value corresponding to each pixel in the current block;

the first selecting unit 805 is further configured to select one predictor model from the N predictor models;

the first calculating unit 803 is further configured to calculate a second image component prediction value corresponding to each pixel in the current block according to the selected predictor model and the first image component reconstruction value corresponding to each pixel in the current block; wherein the image component to be predicted is a second image component.

In the above scenario, referring to fig. 8, the encoder 80 may further include a first comparing unit 808 configured to compare a first image component reconstruction value corresponding to each pixel in the current block with the at least one threshold value;

the first selecting unit 805 is further configured to select, according to the result of the comparison, a predictor model corresponding to each pixel in the current block from the N predictor models.

It will be appreciated that in embodiments of the application, a "unit" may be a part of a circuit, a part of a processor, a part of a program or software, etc., and may of course also be a module, or may be non-modular. Furthermore, the components in the present embodiment may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional modules.

The integrated units, if implemented in the form of software functional modules, may be stored in a computer-readable storage medium, if not sold or used as separate products, and based on such understanding, the technical solution of the present embodiment may be embodied essentially or partly in the form of a software product, which is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) or processor to perform all or part of the steps of the method described in the present embodiment. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a magnetic disk, or an optical disk, or other various media capable of storing program codes.

Accordingly, an embodiment of the present application provides a computer storage medium storing an image component prediction program which, when executed by at least one processor, implements the method of any of the preceding embodiments.

Based on the above-described composition of the encoder 80 and the computer storage medium, referring to fig. 9, a specific hardware structure example of the encoder 80 provided by the embodiment of the present application may include: a first communication interface 901, a first memory 902, and a first processor 903; the various components are coupled together by a first bus system 904. It is appreciated that the first bus system 904 is used to enable connected communications between these components. The first bus system 904 includes a power bus, a control bus, and a status signal bus in addition to a data bus. But for clarity of illustration, the various buses are labeled as a first bus system 904 in fig. 9. Wherein, the liquid crystal display device comprises a liquid crystal display device,

the first communication interface 901 is configured to receive and send signals in a process of receiving and sending information with other external network elements;

a first memory 902 for storing a computer program capable of running on the first processor 903;

The first processor 903 is configured to execute, when executing the computer program:

determining adjacent reference pixels of a current block in an image;

It will be appreciated that the first memory 902 in embodiments of the application can be either volatile memory or nonvolatile memory, or can include both volatile and nonvolatile memory. The nonvolatile Memory may be a Read-Only Memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically Erasable EPROM (EEPROM), or a flash Memory. The volatile memory may be random access memory (Random Access Memory, RAM) which acts as an external cache. By way of example, and not limitation, many forms of RAM are available, such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (Double Data Rate SDRAM), enhanced SDRAM (ESDRAM), synchronous DRAM (SLDRAM), and Direct RAM (DRRAM). The first memory 902 of the systems and methods described herein is intended to comprise, without being limited to, these and any other suitable types of memory.

And the first processor 903 may be an integrated circuit chip with signal processing capabilities. In implementation, the steps of the above method may be performed by integrated logic circuits of hardware or instructions in software form in the first processor 903. The first processor 903 described above may be a general purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), an off-the-shelf programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components. The disclosed methods, steps, and logic blocks in the embodiments of the present application may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in connection with the embodiments of the present application may be embodied directly in the execution of a hardware decoding processor, or in the execution of a combination of hardware and software modules in a decoding processor. The software modules may be located in a random access memory, flash memory, read only memory, programmable read only memory, or electrically erasable programmable memory, registers, etc. as well known in the art. The storage medium is located in the first memory 902, and the first processor 903 reads information in the first memory 902, and in combination with hardware, performs the steps of the above method.

It is to be understood that the embodiments described herein may be implemented in hardware, software, firmware, middleware, microcode, or a combination thereof. For a hardware implementation, the processing units may be implemented within one or more application specific integrated circuits (Application Specific Integrated Circuits, ASIC), digital signal processors (Digital Signal Processing, DSP), digital signal processing devices (DSP devices, DSPD), programmable logic devices (Programmable Logic Device, PLD), field programmable gate arrays (Field-Programmable Gate Array, FPGA), general purpose processors, controllers, microcontrollers, microprocessors, other electronic units designed to perform the functions described herein, or a combination thereof. For a software implementation, the techniques described herein may be implemented with modules (e.g., procedures, functions, and so on) that perform the functions described herein. The software codes may be stored in a memory and executed by a processor. The memory may be implemented within the processor or external to the processor.

Optionally, as another embodiment, the first processor 903 is further configured to perform the method of any of the preceding embodiments when running the computer program.

The present embodiment provides an encoder, which may include a first determination unit configured to determine adjacent reference pixels of a current block in an image, a first construction unit, and a first calculation unit; the first construction unit is configured to construct a subset of adjacent reference pixels from the adjacent reference pixels; wherein the adjacent reference pixel subset comprises part of pixels in the adjacent reference pixels; the first calculating unit is configured to calculate model parameters of a prediction model according to the adjacent reference pixel subsets; the prediction model comprises N prediction sub-models, the N prediction sub-models correspond to N groups of model parameters, the prediction sub-models are used for performing cross-component prediction processing on image components to be predicted through the corresponding model parameters, and N is a positive integer greater than or equal to 2; in this way, the number of pixels in the adjacent reference pixel set is reduced, so that the number of pixels in the adjacent reference pixel subset is smaller, the calculation complexity and the memory bandwidth are reduced, and the accuracy of the prediction model is improved.

Based on the same inventive concept as the previous embodiments, referring to fig. 10, a schematic diagram of the composition structure of a decoder 100 according to an embodiment of the present application is shown. The decoder 100 may include: a second determination unit 1001, a second construction unit 1002, and a second calculation unit 1003, wherein,

The second determining unit 1001 is configured to determine adjacent reference pixels of a current block in an image;

the second construction unit 1002 is configured to construct a subset of neighboring reference pixels from the neighboring reference pixels; wherein the adjacent reference pixel subset includes a part of pixels in the adjacent reference pixels;

the second calculation unit 1003 is configured to calculate model parameters of a prediction model according to the adjacent reference pixel subset; the prediction model comprises N prediction sub-models, the N prediction sub-models correspond to N groups of model parameters, the prediction sub-models are used for performing cross-component prediction processing on the image component to be predicted through the corresponding model parameters, and N is a positive integer greater than or equal to 2.

In the above-described scheme, referring to fig. 10, the decoder 100 may further include a second acquisition unit 1004 configured to acquire reference pixels adjacent to at least one edge of the current block; wherein at least one edge of the current block includes at least one of: uplink, right uplink, left column and left downlink; and obtaining the adjacent reference pixels according to the acquired reference pixels.

In the above-described scheme, referring to fig. 10, the decoder 100 may further include a second selecting unit 1005, in which,

The second determining unit 1001 is further configured to determine a candidate position of the candidate pixel based on at least one edge of the current block;

the second selecting unit 1005 is configured to select a reference pixel corresponding to the candidate position from the adjacent reference pixels, and form the selected reference pixel into the adjacent reference pixel subset.

In the above aspect, the second determining unit 1001 is further configured to determine the candidate position based on a pixel position corresponding to a reference pixel adjacent to at least one edge of the current block.

In the above aspect, the second determining unit 1001 is further configured to determine the candidate position based on an image component intensity value corresponding to a reference pixel adjacent to at least one edge of the current block.

In the above aspect, the second determining unit 1001 is further configured to determine the candidate position based on a pixel position and an image component intensity value corresponding to a reference pixel adjacent to at least one edge of the current block.

In the above aspect, the second determining unit 1001 is further configured to determine a preset candidate pixel number; wherein the preset candidate pixel number represents the number of pixels sampled from reference pixels adjacent to at least one edge of the current block; determining the candidate position according to the preset candidate pixel number and the length of at least one edge of the current block; wherein the length of at least one edge of the current block is equal to the number of pixels contained in at least one edge of the current block.

In the above aspect, the second calculating unit 1003 is further configured to calculate a first sampling interval according to the preset candidate pixel number and a length of at least one edge of the current block;

the second determining unit 1001 is further configured to determine a reference point from at least one edge of the current block, and determine the candidate position according to the first sampling interval.

In the foregoing scenario, referring to fig. 10, the decoder 100 may further include a second adjustment unit 1006 configured to adjust the first sampling interval to obtain a second sampling interval;

the second determining unit 1001 is further configured to determine the candidate position at the second sampling interval based on the reference point.

In the above aspect, the second determining unit 1001 is further configured to determine, based on the reference point, a candidate position corresponding to one side of the reference point at the first sampling interval, and determine a candidate position corresponding to the other side of the reference point at the second sampling interval.

In the above aspect, the second determining unit 1001 is further configured to determine a preset skipped pixel number K corresponding to at least one edge of the current block, where K is a positive integer greater than or equal to 1; starting from the initial position and/or the tail end position of at least one edge of the current block, determining the positions corresponding to K pixel points to be skipped; continuously skipping the K pixel points to be skipped from the starting position and/or the end position of at least one edge of the current block based on the positions corresponding to the K pixel points to be skipped, so as to obtain at least one new edge of the current block; and determining the candidate position based on at least one new edge of the current block and the preset candidate pixel number.

In the above aspect, the second construction unit 1002 is configured to construct N first adjacent reference pixel subsets according to the adjacent reference pixel subsets;

the second calculating unit 1003 is configured to calculate N groups of model parameters corresponding to the N predictor models based on the N first adjacent reference pixel subsets; wherein each predictor model corresponds to a set of model parameters.

In the above-described scheme, referring to fig. 10, the decoder 100 may further include a second dividing unit 1007 in which,

the second determining unit 1001 is further configured to determine at least one threshold value according to the adjacent reference pixel subset;

the second dividing unit 1007 is configured to divide the adjacent reference pixel subset into the N first adjacent reference pixel subsets according to the at least one threshold.

In the above aspect, the second determining unit 1001 is further configured to determine a first image component reconstruction value corresponding to each pixel in the current block;

the second selecting unit 1005 is further configured to select one predictor model from the N predictor models;

the second calculating unit 1003 is further configured to calculate a second image component predicted value corresponding to each pixel in the current block according to the selected predictor model and the first image component reconstructed value corresponding to each pixel in the current block; wherein the image component to be predicted is a second image component.

In the above scenario, referring to fig. 10, the decoder 100 may further include a second comparing unit 1008 configured to compare the first image component reconstruction value corresponding to each pixel in the current block with the at least one threshold value;

the second selecting unit 1005 is further configured to select, according to the result of the comparison, a predictor model corresponding to each pixel in the current block from the N predictor models.

It will be appreciated that in this embodiment, the "unit" may be a part of a circuit, a part of a processor, a part of a program or software, etc., and may of course be a module, or may be non-modular. Furthermore, the components in the present embodiment may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional modules.

The integrated units may be stored in a computer readable storage medium if implemented in the form of software functional modules, and not sold or used as stand-alone products. Based on such understanding, the present embodiment provides a computer storage medium storing an image component prediction program which, when executed by a second processor, implements the method of any one of the preceding embodiments.

Based on the above composition of the decoder 100 and the computer storage medium, referring to fig. 11, a specific hardware structure of the decoder 100 provided by an embodiment of the present application may include: a second communication interface 1101, a second memory 1102, and a second processor 1103; the various components are coupled together by a second bus system 1104. It is appreciated that the second bus system 1104 is used to enable connected communications between these components. The second bus system 1104 includes a power bus, a control bus, and a status signal bus in addition to the data bus. But for clarity of illustration the various buses are labeled in fig. 11 as the second bus system 1104. Wherein, the liquid crystal display device comprises a liquid crystal display device,

a second communication interface 1101, configured to receive and send signals during information transceiving with other external network elements;

a second memory 1102 for storing a computer program capable of running on the second processor 1103;

a second processor 1103, configured to, when executing the computer program, perform:

determining adjacent reference pixels of a current block in an image;

Optionally, as another embodiment, the second processor 1103 is further configured to perform the method of any of the previous embodiments when running the computer program.

It is understood that the second memory 1102 is similar to the hardware functionality of the first memory 902 and the second processor 1103 is similar to the hardware functionality of the first processor 903; and will not be described in detail herein.

The present embodiment provides a decoder that may include a second determination unit configured to determine adjacent reference pixels of a current block in an image, a second construction unit, and a second calculation unit; the second construction unit is configured to construct a subset of adjacent reference pixels from the adjacent reference pixels; wherein the adjacent reference pixel subset comprises part of pixels in the adjacent reference pixels; the second calculation unit is configured to calculate model parameters of the prediction model according to the adjacent reference pixel subsets; the prediction model comprises N prediction sub-models, the N prediction sub-models correspond to N groups of model parameters, the prediction sub-models are used for performing cross-component prediction processing on image components to be predicted through the corresponding model parameters, and N is a positive integer greater than or equal to 2; in this way, the number of pixels in the adjacent reference pixel set is reduced, so that the number of pixels in the adjacent reference pixel subset is smaller, the calculation complexity and the memory bandwidth are reduced, and the accuracy of the prediction model is improved.

It should be noted that, in the present application, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.

The foregoing embodiment numbers of the present application are merely for the purpose of description, and do not represent the advantages or disadvantages of the embodiments.

The methods disclosed in the method embodiments provided by the application can be arbitrarily combined under the condition of no conflict to obtain a new method embodiment.

The features disclosed in the several product embodiments provided by the application can be combined arbitrarily under the condition of no conflict to obtain new product embodiments.

The features disclosed in the embodiments of the method or the apparatus provided by the application can be arbitrarily combined without conflict to obtain new embodiments of the method or the apparatus.

The foregoing is merely illustrative of the present application, and the present application is not limited thereto, and any person skilled in the art will readily recognize that variations or substitutions are within the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Industrial applicability

In the embodiment of the application, firstly, adjacent reference pixels of a current block in an image are determined; then constructing a subset of adjacent reference pixels according to the adjacent reference pixels; wherein the adjacent reference pixel subset comprises part of pixels in the adjacent reference pixels; calculating model parameters of the prediction model according to the adjacent reference pixel subsets; the prediction model comprises N prediction sub-models, the N prediction sub-models correspond to N groups of model parameters, the prediction sub-models are used for performing cross-component prediction processing on image components to be predicted through the corresponding model parameters, and N is a positive integer greater than or equal to 2; in this way, as the screening process is carried out on the adjacent reference pixels of the current block, unimportant reference pixel points or abnormal reference pixel points can be removed, so that the number of pixels in the adjacent reference pixel set is reduced, the number of pixels in the adjacent reference pixel subset is smaller, the calculation complexity and the memory bandwidth are reduced, and the accuracy of the prediction model is improved; in addition, cross-component prediction processing is carried out on the image component to be processed through at least two predictor models, so that the prediction accuracy of the image component to be processed is further improved, and meanwhile, the prediction efficiency of the video image is improved.

Claims

1. An image component prediction method applied to an encoder, the method comprising:

determining adjacent reference pixels of a current block in an image;

calculating model parameters of a prediction model according to the adjacent reference pixel subsets; the prediction model comprises N prediction sub-models, the N prediction sub-models correspond to N groups of model parameters, the prediction sub-models are used for performing cross-component prediction processing on image components to be predicted through the corresponding model parameters, and N is a positive integer greater than or equal to 2;

wherein said constructing a subset of adjacent reference pixels from said adjacent reference pixels comprises:

determining a candidate location of a candidate pixel based on at least one edge of the current block;

selecting reference pixels corresponding to the candidate positions from the adjacent reference pixels, and forming the adjacent reference pixel subsets by the selected reference pixels;

wherein the determining the candidate location of the candidate pixel based on at least one edge of the current block comprises:

2. The method of claim 1, wherein the determining neighboring reference pixels of the current block in the image comprises:

3. The method of claim 1, wherein the determining the candidate location of the candidate pixel based on at least one edge of the current block comprises:

4. The method of claim 1, wherein the determining the candidate location of the candidate pixel based on at least one edge of the current block comprises:

5. The method of claim 1, wherein the determining the candidate location of the candidate pixel based on at least one edge of the current block comprises:

6. The method of claim 1, wherein the candidate locations of the candidate pixels are determined according to the formula:

Δ＝length/(N ₂ /2)

shift＝Δ/2

wherein delta represents a sampling interval, length represents the number of a row of reference pixel points adjacent to the upper side of the current block or the number of a column of reference pixel points adjacent to the left side of the current block, N ₂ And representing the number of the reference pixel points in the adjacent reference pixel subset expected to form the current block, wherein shift represents the starting point position of the selected reference pixel point.

7. The method of claim 1, wherein after the calculating the first sampling interval, the method further comprises:

adjusting the first sampling interval to obtain a second sampling interval;

8. The method of claim 7, wherein after the second sampling interval is obtained, the method further comprises:

9. The method of any of claims 1 to 8, wherein the determining a candidate location for the candidate pixel based on at least one edge of the current block comprises:

10. The method of claim 1, wherein said constructing N first adjacent reference pixel subsets from said adjacent reference pixel subsets comprises:

11. The method of claim 10, wherein after said calculating model parameters of a predictive model from said subset of adjacent reference pixels, the method further comprises:

selecting one predictor model from the N predictor models;

12. The method of claim 11, wherein the selecting one predictor model from the N predictor models comprises:

13. The method of claim 1, wherein said calculating model parameters of a predictive model from said subset of adjacent reference pixels comprises:

calculating N groups of model parameters corresponding to the N predictor models based on the N first adjacent reference pixel subsets; wherein each predictor model corresponds to a set of model parameters; wherein the predictive model comprises a nonlinear model or a complex model, wherein the complex model comprises a nonlinear model in the form of a conic nonlinearity or a multi-model made up of a plurality of linear models.

14. An image component prediction method applied to a decoder, the method comprising:

determining adjacent reference pixels of a current block in an image;

15. The method of claim 14, wherein the determining neighboring reference pixels of the current block in the image comprises:

16. The method of claim 14, wherein the determining the candidate location of the candidate pixel based on at least one edge of the current block comprises:

17. The method of claim 14, wherein the determining the candidate location of the candidate pixel based on at least one edge of the current block comprises:

18. The method of claim 14, wherein the determining the candidate location of the candidate pixel based on at least one edge of the current block comprises:

19. The method of claim 14, wherein the candidate locations of the candidate pixels are determined according to the formula:

Δ＝length/(N ₂ /2)

shift＝Δ/2

20. The method of claim 14, wherein after the calculating the first sampling interval, the method further comprises:

adjusting the first sampling interval to obtain a second sampling interval;

21. The method of claim 20, wherein after the second sampling interval, the method further comprises:

22. The method of any of claims 14 to 21, wherein the determining the candidate location of the candidate pixel based on at least one edge of the current block comprises:

23. The method of claim 14, wherein constructing N first adjacent reference pixel subsets from the adjacent reference pixel subsets comprises:

24. The method of claim 23, wherein after said calculating model parameters of a predictive model from said subset of adjacent reference pixels, the method further comprises:

selecting one predictor model from the N predictor models;

25. The method of claim 24, wherein the selecting one predictor model from the N predictor models comprises:

26. The method of claim 14, wherein said calculating model parameters of a predictive model from said subset of adjacent reference pixels comprises:

27. An encoder comprising a first determination unit, a first construction unit and a first calculation unit, wherein,

the first determining unit is configured to determine adjacent reference pixels of a current block in an image;

the first construction unit is configured to construct a subset of adjacent reference pixels according to the adjacent reference pixels; wherein the adjacent reference pixel subset includes a part of pixels in the adjacent reference pixels;

The first calculating unit is configured to calculate model parameters of a prediction model according to the adjacent reference pixel subsets; the prediction model comprises N prediction sub-models, the N prediction sub-models correspond to N groups of model parameters, the prediction sub-models are used for performing cross-component prediction processing on image components to be predicted through the corresponding model parameters, and N is a positive integer greater than or equal to 2;

wherein the first construction unit is further configured to:

28. An encoder comprising a first memory and a first processor, wherein,

the first memory is used for storing a computer program capable of running on the first processor;

The first processor being adapted to perform the method of any of claims 1 to 13 when the computer program is run.

29. A decoder includes a second determination unit, a second construction unit, and a second calculation unit, wherein,

the second determining unit is configured to determine adjacent reference pixels of the current block in the image;

the second construction unit is configured to construct a subset of adjacent reference pixels according to the adjacent reference pixels; wherein the adjacent reference pixel subset includes a part of pixels in the adjacent reference pixels;

the second calculating unit is configured to calculate model parameters of a prediction model according to the adjacent reference pixel subsets; the prediction model comprises N prediction sub-models, the N prediction sub-models correspond to N groups of model parameters, the prediction sub-models are used for performing cross-component prediction processing on image components to be predicted through the corresponding model parameters, and N is a positive integer greater than or equal to 2;

wherein the second construction unit is further configured to:

Wherein the second construction unit is further configured to:

30. A decoder comprising a second memory and a second processor, wherein,

the second memory is used for storing a computer program capable of running on the second processor;

the second processor being adapted to perform the method of any of claims 14-26 when the computer program is run.

31. A computer storage medium storing an image component prediction program which, when executed by a first processor or a second processor, implements the method of any one of claims 1 to 26.