WO2020137904A1

WO2020137904A1 - Information processing device, information processing method, and information processing program

Info

Publication number: WO2020137904A1
Application number: PCT/JP2019/050163
Authority: WO
Inventors: 純代江嶋; 優池田; 勇司藤本
Original assignee: ソニー株式会社
Priority date: 2018-12-28
Filing date: 2019-12-20
Publication date: 2020-07-02

Abstract

This information processing device is provided with an intra-frame prediction unit which extracts a plurality of points from a reference pixel adjoining a block prescribed within an image, and calculates a linear model which predicts color information of pixels present in said block, on the basis of color information and brightness information of the plurality of extracted points.

Description

Information processing apparatus, information processing method, and information processing program

The present disclosure relates to an information processing device, an information processing method, and an information processing program. Specifically, it relates to a moving image encoding process.

H.265/HEVC (High Efficiency Video Coding) is standardized as a moving picture compression encoding method. In H.265/HEVC, intra prediction that performs spatial prediction within an image to generate a prediction value and inter prediction that performs motion compensation prediction between images to generate a prediction value are used.

As one of the methods for improving the current intra prediction, CCLM prediction (Cross-Component Liner Model Prediction) that generates a prediction value of color information using a decoded pixel value of luminance information is proposed. According to CCLM prediction, redundancy between components of luminance information and color information can be reduced (for example, Non-Patent Document 1). Furthermore, a method of reducing the calculation cost of CCLM prediction has also been proposed (for example, Non-Patent Document 2).

According to the conventional technology, it is possible to reduce the redundancy of the pixels in the screen (frame), so that it is possible to improve the compression efficiency.

However, the above-mentioned conventional technology has room for improvement. For example, CCLM prediction has a high calculation cost, and thus may reduce the throughput of the entire encoding and decoding processes. Even in the proposed method of Non-Patent Document 2, in order to obtain the maximum value and the minimum value of the luminance information and the color information, it is necessary to loop and calculate all the reference pixels, so that the calculation cost is significantly reduced. Is difficult Further, the proposed method of Non-Patent Document 2 is affected by noise because it calculates a regression line (a linear model used for CCLM prediction) by using each one of the maximum value and the minimum value of luminance information and color information. There is a concern that it will be easier.

Therefore, the present disclosure proposes an information processing device, an information processing method, and an information processing program capable of reducing the calculation cost of encoding and decoding while increasing the durability against noise in CCLM prediction.

In order to solve the above problems, an information processing device according to an aspect of the present disclosure is to extract a plurality of points from reference pixels adjacent to a block defined in an image, and to extract brightness information and color information of the plurality of points. And an intra-prediction unit that calculates a linear model that predicts color information of pixels existing in the block.

FIG. 20 is a block diagram showing a configuration example of an image encoding device which is an example of an information processing device according to the present disclosure. FIG. 20 is a block diagram illustrating a configuration example of an intra prediction unit according to the present disclosure. It is a conceptual diagram which showed the processing block which concerns on 1st Embodiment. 6 is a flowchart showing a procedure of information processing according to the first embodiment. It is a conceptual diagram which showed the processing block which concerns on 2nd Embodiment. It is a figure (1) for explaining the intra prediction processing concerning a 2nd embodiment. It is a figure (2) for demonstrating the intra prediction process which concerns on 2nd Embodiment. It is a flow chart which shows a procedure of information processing concerning a 2nd embodiment. FIG. 16 is a hardware configuration diagram illustrating an example of a computer that realizes a function of the image encoding device according to the present disclosure.

Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. In addition, in each of the following embodiments, the same reference numerals are given to the same portions, and the overlapping description will be omitted.

The present disclosure will be described in the following item order.
1. First Embodiment 1-1. Configuration example of information processing apparatus according to first embodiment 1-2. Details of information processing according to first embodiment 1-3. Information processing procedure according to the first embodiment 2. Second embodiment 2-1. Details of information processing according to second embodiment 2-2. Information processing procedure according to the second embodiment 3. Third embodiment 4. Modification 5. Other Embodiments 6. Effects of information processing apparatus according to the present disclosure 7. Hardware configuration

(1. First embodiment)
[1-1. Configuration example of information processing apparatus according to first embodiment]
First, the configuration of the information processing device according to the present disclosure will be described using FIG. 1. FIG. 1 is a block diagram showing a configuration example of an image encoding device 100 which is an example of an information processing device according to the present disclosure. Although not shown in FIG. 1, the image encoding device 100 includes an external device that is a source of original image data to be encoded and a transmission destination of an encoded bitstream that encodes an image. Other external devices or the like may be communicated with each other using a wired or wireless network.

The image encoding device 100 is a device that encodes a prediction error between an image and its predicted image based on the configuration shown in FIG. Although the configuration of the image encoding device 100 is shown in FIG. 1, the configuration of FIG. 1 is merely an example, and the image encoding device 100 is not necessarily limited to the configuration in FIG. That is, the image encoding device 100 has a configuration for performing the compression encoding standardized by the above-mentioned H.265/HEVC and a configuration for realizing the VVC (Versatile Video Coding) proposed as the next-generation compression encoding technology. If so, the configuration does not necessarily have to be that shown in FIG.

The image encoding device 100 performs intra prediction processing using CCLM prediction in the image compression encoding technique. Here, a problem relating to CCLM prediction will be described.

The CCLM prediction uses information about pixels (hereinafter, referred to as “reference pixels”) adjacent to a block (a group of pixels composed of a plurality of pixels in the horizontal and vertical directions) to be processed in one frame, in the block. Predict the information of the pixel of. The block indicates a processing unit to be encoded, and is generally described as CU (Coding Unit). The CU is a block having a variable size, which is formed by recursively dividing an LCU (Largest Coding Unit) which is the maximum coding unit. In intra prediction, different processing may be performed depending on the size and shape of the CU or PU (Predict Unit), but in the processing proposed in the present disclosure, the size of the CU or PU is not limited. In the following description, CU and the like are collectively referred to as “block”.

In CCLM prediction, luminance information of a reference pixel (may be read as a luminance signal or a luminance value. In the following description, the luminance information may be expressed as “Luma”) and color information (color difference signal or color difference value). Also, in the following description, the pixel information in the block is predicted by using the relationship with (color information may be expressed as “Chroma”). As disclosed in Non-Patent Document 1 described above, in CCLM prediction, a Luma of a block is predicted, and a regression (reconstruct) of the information in the same block is used to reduce a regression error (specifically, a linear model). The regression line) is calculated and Chroma in the block is predicted. According to the CCLM prediction, it is possible to reduce the redundancy of pixels in the frame, and thus it is possible to improve the compression efficiency.

However, the CCLM prediction has a large calculation cost, and the calculation may affect the throughput of the entire coding. For example, the linear model is obtained by the following equation (1).

In the above formula (1), pred _c (i,j) indicates the predicted value of Chroma in the block ((i,j) indicates the pixel position). Also, rec _L '(i,j) indicates information obtained by down-sampling the regressed Luma. Then, α and β are coefficients for specifying the linear model. For example, α and β are represented by the following formula (2).

In the above equation (2), N represents a unit of block (horizontal or vertical number of pixels). As shown in Expression (2), α and β must be calculated by looping all the values of the reference pixels of the block, and since the calculation formula includes division, the calculation cost is very large. Become. Therefore, as disclosed in Non-Patent Document 2 described above, the following formula (3) is proposed which simplifies the calculation of α and β.

In the above formula (3), (x _A ,y _A ) and (x _B ,y _B ) respectively indicate the maximum value or the minimum value of Luma or Chroma in the reference pixel. According to the equation (3), α and β can be easily calculated.

However, in order to use the above formula (3), it is necessary to loop and calculate all the reference pixels in the process of obtaining the maximum value and the minimum value of the brightness information of the reference pixel, etc. It is difficult to reduce to. In addition, the proposed method of Non-Patent Document 2 that uses Expression (3) is likely to be affected by noise because it calculates a linear model by using each one of the maximum value and the minimum value of the luminance information and the color information. I have a concern. Specifically, the pixel that can take the minimum value or the maximum value is likely to be noise, and if a linear model is calculated using these values, the prediction accuracy may decrease.

Therefore, in the present disclosure, as described below, information processing that can reduce the calculation cost of encoding and decoding while improving the durability against noise in CCLM prediction is proposed.

First, the overall flow of the encoding process will be described using FIG. 1, and then the details of the information processing (intra prediction process) according to the present disclosure will be described using FIG. 2 and subsequent figures. It should be noted that FIGS. 1 and 2 show main components such as a processing unit and a data flow, and the components shown in FIGS. 1 and 2 are not necessarily all. That is, in the image encoding device 100, a processing unit not shown as a block in FIG. 1 may exist, or a process or data flow not shown as an arrow or the like in FIG. 1 may exist.

As illustrated in FIG. 1, the image encoding device 100 includes an A/D conversion unit 101, a rearrangement buffer 102, a first calculation unit 103, an orthogonal transformation unit 104, a quantization unit 105, an encoding unit 106, and a storage buffer 107. , Inverse quantizer 108, inverse orthogonal transformer 109, second calculator 110, filter 111, frame memory 112, switch 113, intra predictor 114, inter predictor 115, predicted image selector 116, and rate controller. 117 are included. The image encoding apparatus 100 encodes original image data (image) that is an input moving image in frame units for each block (for example, CU).

For example, the A/D conversion unit 101 of the image encoding device 100 acquires original image data, and performs A/D conversion on an image included in the original image data in units of frames to be encoded. Then, the A/D conversion unit 101 outputs the A/D converted data to the rearrangement buffer 102.

The rearrangement buffer 102 arranges image data output from the A/D conversion unit 101 in units of frames in display order in an order for encoding, for example, according to a GOP (Group of Picture) structure. Change. Then, the rearrangement buffer 102 outputs the rearranged data to the first calculation unit 103 and the inter prediction unit 115.

The first calculation unit 103 sequentially sets an input image as an image to be encoded, and sets a block to be encoded for the image to be encoded. The first calculation unit 103 subtracts the prediction image (prediction block) of the block output from the prediction image selection unit 116 from the image of the block to be encoded (current block) to obtain the prediction error, and orthogonally transforms it. It is output to the unit 104.

The orthogonal transformation unit 104 performs orthogonal transformation or the like on the prediction error output from the first calculation unit 103, and derives a transformation coefficient. The orthogonal transformation unit 104 outputs the transformation coefficient to the quantization unit 105.

The quantization unit 105 scales (quantizes) the transform coefficient output from the orthogonal transform unit 104 and derives a quantized transform coefficient. The quantizer 105 outputs the quantized transform coefficient to the encoder 106 and the inverse quantizer 108.

The encoding unit 106 encodes the quantized transform coefficient output from the quantization unit 105 by a predetermined method. For example, the encoding unit 106 converts the encoding parameter and the quantized transform coefficient output from the quantization unit 105 into the syntax value of each syntax element according to the definition of the syntax table. Then, the encoding unit 106 encodes each syntax value (for example, arithmetic encoding such as CABAC (Context-based Adaptive Binary Arithmetic Coding)).

The encoding unit 106 multiplexes encoded data, which is a bit string of each syntax element obtained as a result of encoding, and outputs the multiplexed data to the accumulation buffer 107 as an encoded bit stream. For example, the encoding unit 106 encodes the block that is processed by the intra prediction unit 114, based on the color information predicted by the intra prediction unit 114, which will be described later. More specifically, the encoding unit 106 predicts an image corresponding to the original image data, an image corresponding to the original image data, and a prediction image including a block including the color information predicted by the intra prediction unit 114. The error and are encoded.

The accumulation buffer 107 temporarily stores the encoded bitstream output from the encoding unit 106. After that, the accumulation buffer 127 transmits the accumulated encoded bitstream to the decoding device or the like via the transmission path.

The inverse quantization unit 108 scales (inverse-quantizes) the value of the quantized transform coefficient output from the quantization unit 105, and derives the transformed coefficient after the inverse quantization. The inverse quantization unit 108 outputs the transform coefficient to the inverse orthogonal transform unit 109. That is, the inverse quantization performed by the inverse quantization unit 108 is an inverse process of the quantization performed by the quantization unit 105, and is the same process as the inverse quantization performed by the image decoding device (decoder).

The inverse orthogonal transform unit 109 performs inverse orthogonal transform or the like on the transform coefficient output from the inverse quantization unit 108 and derives a prediction error. The inverse orthogonal transform unit 109 outputs the prediction error to the second calculation unit 110. That is, the inverse orthogonal transform performed by the inverse orthogonal transform unit 109 is an inverse process of the orthogonal transform performed by the orthogonal transform unit 104, and is the same process as the inverse orthogonal transform performed in the image decoding device.

The second calculation unit 110 adds the prediction error output from the inverse orthogonal transform unit 109 and the prediction image corresponding to the prediction error output from the prediction image selection unit 116 to obtain a local decoded image (decoded image). Image). The second calculation unit 110 outputs the locally decoded image to the filter 111.

The filter 111 removes block distortion by filtering the decoded image output from the second calculation unit 110. The filter 111 outputs the filtered image to the frame memory 112.

The frame memory 112 reconstructs a decoded image in image units using the locally decoded image output from the filter 111 and stores it in a buffer in the frame memory 112. The frame memory 112 reads the decoded image designated by the intra prediction unit 114 or the inter prediction unit 115 as a reference image from the buffer, and outputs the decoded image via the switch 113.

The intra prediction unit 114 acquires, as a reference image, a decoded image stored in the frame memory 112 at the same time as the block to be encoded. Then, the intra prediction unit 114 uses the reference image to perform intra prediction processing in a predetermined intra prediction mode on the block to be encoded. Although a plurality of intra prediction modes are specified in H.265/HEVC and VVC, only the process using CCLM prediction will be described in the present disclosure as described later.

The inter prediction unit 115 includes a motion prediction/compensation unit, a motion vector detection unit, and the like, and executes prediction processing regarding images (inter).

The prediction unit 119 outputs the predicted image generated as a result of the intra prediction process or the inter prediction process to the first calculation unit 103 and the second calculation unit 110.

The overview of the overall configuration of the image encoding device 100 according to the present disclosure has been described above. The details of the intra prediction process that is the information processing according to the present disclosure will be described below with reference to FIG.

[1-2. Details of information processing according to the first embodiment]
Hereinafter, details of information processing according to the present disclosure will be described with reference to FIGS. 2 and 3. First, the configuration of the intra prediction unit 114 according to the present disclosure will be described using FIG. 2. FIG. 2 is a block diagram showing a configuration example of the intra prediction unit 114 according to the present disclosure.

As illustrated in FIG. 2, the intra prediction unit 114 of the image encoding device 100 according to the present disclosure includes a luminance/color difference separation unit 1141, a luminance signal intra prediction unit 1142, a luminance signal downsampling unit 1143, and a color difference signal intra prediction. And a portion 1144.

The luminance/color difference separation unit 1141 acquires the encoded block output from the frame memory 112 and separates it into a luminance signal and a color difference signal. The luminance/color difference separation unit 1141 outputs a luminance signal of the separated information to the luminance signal intra prediction unit 1142. Such a luminance signal corresponds to the regressed luminance signal, and will be referred to as “Rec _L (L means Luma)” in the following description. The luminance/color difference separating unit 1141 also outputs the separated luminance signal to the luminance signal down-sampling unit 1143.

The luminance/color difference separation unit 1141 also outputs the separated color difference signals to the color difference signal intra prediction unit 1144. Such a color difference signal corresponds to the regressed color difference signal, and will be referred to as "Rec _C (C means Chroma)" in the following description.

The luminance signal intra-prediction unit 1142, based on Rec _L output from the luminance-color difference separation unit 1141 and parameters for intra-prediction processing (for example, the size of the block to be processed, etc.), Predict the luminance signal of. In the following description, the predicted luminance signal is referred to as “Pred _L (L means Luma)”.

The luminance signal downsampling unit 1143 downsamples Rec _L output from the luminance/color difference separating unit 1141. In such processing, since the amount of information is usually different between the luminance signal and the color difference signal (for example, the luminance signal Y and the color difference signals Cb and Cr are displayed in the format of 4:2:0), the difference is caused. It is done for adjustment. In the following description, the down-sampled luminance signal will be referred to as “Rec′ _L (L means Luma)”.

The color difference signal intra prediction unit 1144 predicts the color difference signal in the block to be processed by using the CCLM prediction method. Specifically, the color difference signal intra prediction unit 1144 sets the parameters for intra prediction processing, the down-sampled luminance signal “Rec′ _L ”, and the regressed color difference signal “Rec _C ”. The predicted value of the color difference signal is calculated by using this. Specifically, the color difference signal intra prediction unit 1144 calculates the predicted value “Pred _C ”of the color difference signal using the above equation (1).

Then, the intra prediction unit 114 outputs the predicted image based on the calculated “Pred _L ” and “Pred _C ” to the predicted image selection unit 116.

Next, the method for calculating α and β in the above formula (1) will be described. In the present disclosure, the intra prediction unit 114 extracts a plurality of points from reference pixels adjacent to the block, and based on the extracted luminance information and color information of the plurality of points, a linear model that predicts color information of pixels existing in the block. To calculate. Note that the calculation of the linear model means calculation of the above α and β. In addition, a plurality of points means not all but a plurality (two or more) points of the reference pixels adjacent to the block.

For example, in the first embodiment, the intra prediction unit 114 extracts three points from vertically adjacent reference pixels in the block, and extracts three points from horizontally adjacent reference pixels in the block. The midpoints in the luminance information and the color information of each of the three points are specified, and a linear model is calculated based on the specified midpoints.

More specifically, the intra prediction unit 114 extracts three consecutive points from vertically adjacent reference pixels in the block, and extracts three consecutive points from horizontally adjacent reference pixels in the block. , The midpoints in the luminance information and the color information of each of the three points are specified, and a linear model is calculated based on the specified midpoints.

-This point will be explained using FIG. FIG. 3 is a conceptual diagram showing processing blocks according to the first embodiment. FIG. 3 shows an example in which the relationship between the luminance information and the color information in the image is the 4:2:0 format, and the block size (“N” in the above equation (2)) is “8”. .. In this case, as shown in FIG. 3, the block 40 corresponding to the color information has half the number of vertical and horizontal pixels as compared with the block 50 corresponding to the luminance information. Pixels adjacent to the

blocks

40 and 50 indicate reference pixels.

As described above, the intra prediction unit 114 according to the first embodiment does not perform processing using information of all reference pixels in the block 40, but extracts three points from each of the vertical and horizontal reference pixels. To do. Then, the intra prediction unit 114 specifies the intermediate points in the luminance information and the color information of each of the three points, and calculates the linear model based on the specified intermediate points.

With this, the intra prediction unit 114 can reduce the number of coordinates for calculating the brightness information and the like, as compared with the conventional method of searching all the horizontal and vertical points and calculating the maximum value and the minimum value. High-speed calculation can be performed, and the overall processing speed can be improved. Further, the intra prediction unit 114 can improve the durability against noise by taking an intermediate value of a plurality of points instead of the maximum value and the minimum value estimated to be easily influenced by noise.

More preferably, the intra-prediction unit 114 according to the first embodiment provides three consecutive points from the reference pixels that are adjacent in the vertical direction in the block 40 to one end that is different from the base point of the block 40. Extract the three points that are located. Further, the intra prediction unit 114 extracts three consecutive three points located at one end different from the base point of the block 40 from the reference pixels adjacent to each other in the horizontal direction in the block 40. Then, the intra prediction unit 114 specifies the intermediate points in the luminance information and the color information of each of the three points, and calculates the linear model based on the specified intermediate points.

In this way, the intra prediction unit 114 processes each of the three points farthest from the base point (the top end and the left end of the block) in the space of the block 40. In this way, the intra prediction unit 114 can calculate the linear model using the points that are estimated to most reflect the characteristics of the space, and thus the prediction accuracy can be improved.

Specifically, the intra prediction unit 114, as shown in FIG. 3, is a point A0, a point A1, and a point A2 that are three points far from the base point in the horizontal direction and a point L0 that is three points far from the base point in the vertical direction. , Point L1 and point L2 are extracted as processing targets. Note that, in FIG. 3, the same reference numerals are given to corresponding points between the block 40 and the block 50.

Subsequently, the intra prediction unit 114 calculates the intermediate points of the extracted three points. For example, when each point information is represented by coordinates (luminance information, color information), the point A0 is represented as (XA0, YA0). Here, XA0 indicates the luminance information at the point A0, and YA0 indicates the color information at the point A0. Similarly, the point A1 is (XA1, YA1), the point A2 is (XA2, YA2), the point L0 is (XL0, YL0), the point L1 is (XL1, YL1), and the point L2 is , (XL2, YL2).

Then, the intra prediction unit 114 obtains the intermediate points YA _m , YL _m , XA _m, and XL _m of the color information of each of the horizontal and vertical three points.

For example, when YA0≧YA1 and YA2≧YA0 are satisfied at three horizontal points, YA _m is YA0 and YA _m is XA0. Similarly, when YA1≧YA2 and YA0≧YA1 are satisfied, YA _m is YA1 and YA _m is XA1. Similarly, when YA2≧YA0 and YA1≧YA2 are satisfied, YA _m is YA2 and YA _m is XA2.

When YL0≧YL1 and YL2≧YL0 are satisfied at three vertical points, YL _m is YL0 and YL _m is XL0. Similarly, when YL1≧YL2 and YL0≧YL1 are satisfied, YL _m is YL1 and YL _m is XL1. Similarly, YL2 ≧ YL0 and, when the YL1 ≧ YL2 is satisfied, YL _m is YL2, the YL _m is XL2.

Then, the intra prediction unit 114 calculates α and β of the linear equation of (XA _m , YA _m ) and (XL _m , YL _m ), which are two intermediate points of the horizontal and vertical color information. Specifically, the intra prediction unit 114 calculates α and β based on the two equations YA _m =αXA _m +β and YL _m =αXL _m +β. This means that the intermediate point is substituted for (x _A ,x _B ) and (y _A ,y _B ) in the above equation (3). In this way, the intra prediction unit 114 can significantly reduce the calculation cost as compared with the conventional method.

Note that, in the first embodiment, an example in which the extracted three intermediate points are taken has been shown, but the intra prediction unit 114 may calculate the linear model using information other than the intermediate points. For example, the intra prediction unit 114 may calculate the linear model based on the average values of the extracted luminance information and color information of a plurality of points.

[1-3. Information Processing Procedure According to First Embodiment]
Next, the procedure of information processing according to the first embodiment will be described with reference to FIG. FIG. 4 is a flowchart showing the flow of information processing according to the first embodiment.

As illustrated in FIG. 4, the image encoding device 100 calculates a prediction value (Pred _L ) of the luminance signal in the processing target block (step S101).

Subsequently, the image encoding device 100 determines whether or not to perform CCLM prediction (step S102). When CCLM prediction is not performed (step S102; No), the image coding apparatus 100 skips the subsequent process regarding CCLM prediction. On the other hand, when CCLM prediction is performed (step S102; Yes), the image coding apparatus 100 specifies three horizontal and vertical points in each reference pixel of the block to be processed (step S103).

Then, the image coding apparatus 100 calculates the intermediate value of the upper three points (that is, the horizontal side) (step S104). Similarly, the image coding apparatus 100 calculates the intermediate value of the left three points (that is, the vertical side) (step S105).

The image coding apparatus 100 calculates the slope α in the linear equation based on the calculated intermediate value (step S106). Further, the image coding apparatus 100 calculates the intercept β in the linear equation based on the calculated α (step S107).

After that, the image coding apparatus 100 calculates the predicted value of the color difference signal in the processing target block based on the calculated values of α and β (step S108).

(2. Second embodiment)
[2-1. Details of information processing according to the second embodiment]
Next, a second embodiment will be described. In the first embodiment, an example has been shown in which the intra prediction unit 114 calculates the median value of the extracted plural points and calculates the slope and intercept of the linear equation, that is, α and β from the calculated median value. In the second embodiment, the intra prediction unit 114 obtains a linear equation for each of the extracted combinations of a plurality of points, and calculates a linear model in CCLM prediction based on the obtained solution of the linear equation. Note that, in the second embodiment, the functions and configuration of the image encoding device 100 are the same as those in the first embodiment, so description will be omitted.

An outline of information processing according to the second embodiment will be described with reference to FIG. FIG. 5 is a conceptual diagram showing processing blocks according to the second embodiment. Similar to FIG. 3, FIG. 5 shows a block 40 corresponding to color information and a block 50 corresponding to luminance information.

Like the first embodiment, the intra prediction unit 114 does not perform processing using all the reference pixels, but extracts a plurality of points from the reference pixels. Then, the intra prediction unit 114 selects either one of the plurality of points extracted from the vertically adjacent reference pixels of the block 40 or any one of the plurality of points extracted from the horizontally adjacent reference pixels of the block 40. A linear equation calculated from one point is calculated. For example, when the intra prediction unit 114 extracts three points from the vertical direction and three points from the horizontal direction, the intra prediction unit 114 calculates three linear equations that are the number of combinations of each point. Then, the intra prediction unit 114 calculates a linear model based on the comparison of the calculated slopes of the linear equations.

Specifically, the intra-prediction unit 114, as in the first embodiment, has three consecutive points from the reference pixel adjacent in the vertical direction in the block 40 and one end portion different from the base point of the block. Three located points (point L0, point L1, point L2 shown in FIG. 5) are extracted. In addition, the intra prediction unit 114 extracts, from the reference pixels adjacent to each other in the horizontal direction in the block 40, three consecutive points located at one end different from the base point of the block 40 (FIG. 5 ). Point A0, point A1, point A2). Then, the intra prediction unit 114 calculates three linear equations each combining three points, and calculates a linear model based on the comparison of the slopes of the calculated linear equations.

FIG. 5 conceptually shows straight lines obtained from the points A0 and L0, the points A1 and L1, and the points A2 and L2. That is, in the first embodiment, an intermediate point is obtained from the extracted three points and a linear equation is obtained based on the intermediate point, whereas in the second embodiment, a plurality of linear equations are obtained from the extracted points. The difference is that a linear model is calculated from a plurality of linear equations obtained.

Specifically, the intra prediction unit 114 calculates a linear model based on the intermediate value of the slope of the linear equation calculated for the number of combinations of a plurality of points.

Explain specific calculation method. As in the first embodiment, the intra prediction unit 114 uses the point A0 (XA0, YA0), the point A1 (XA1, YA1), the point A2 (XA2, YA2), the point L0 (XL0, YL0), and the point L1 (XL1). , YL1) and a point L2 (XL2, YL2) are extracted.

Subsequently, the intra prediction unit 114 generates a combination of two points. For example, the intra prediction unit 114 combines the points A0 and L0, the points A1 and L1, and the points A2 and L2, respectively. Then, the information of each point is substituted into the above equation (3) to obtain the slope of each linear equation.

Fig. 6 shows the straight line obtained from the three combinations. FIG. 6 is a diagram (1) for explaining the intra prediction process according to the second embodiment. In FIG. 6, a straight line 52 obtained from the combination of points A0 and L0, a straight line 54 obtained from the combination of points A1 and L1, and a straight line obtained from the combination of points A2 and L2. And 56.

Here, assuming that the slope of the straight line 52 is α ₀₀ , the slope of the straight line 54 is α ₁₁ , and the slope of the straight line 56 is α ₂₂ , an intermediate value α _{m of the} slope is obtained. The method of obtaining the intermediate value is common to that of the first embodiment, and will be omitted.

In the example of FIG. 6, it is assumed that the intra prediction unit 114 determines the slope α ₀₀ as the intermediate value α _m . In this case, the intra prediction unit 114 uses the value of the gradient α ₀₀ determined as the intermediate value α _m , and the color information Y _m (Chroma) and the luminance information L _m (Luma) at that time to obtain an intercept β. _{Find m} . Specifically, since β _m is calculated by Y _m ×α _m −X _m , when α _m =α ₀₀ , β _m =YA0×α ₀₀ −XA0.

As described above, the intra prediction unit 114 according to the second embodiment extracts the three furthest points in the space (block) and calculates each linear equation in parallel to obtain the intermediate point of the solution. Then, the intra prediction unit 114 calculates (determines) a linear model in CCLM prediction based on the midpoint of the solution.

As described above, the intra prediction unit 114 according to the second embodiment can improve the durability against noise by obtaining the intermediate points of the solutions of the plurality of linear equations. This point will be described with reference to FIG. 7.

FIG. 7 is a diagram (2) for explaining the intra prediction process according to the second embodiment. FIG. 7 shows a straight line 58 based on the combination of the points L2 and A2′, instead of the straight line 56 based on the combination of the points L2 and A2 shown in FIG.

In FIG. 7, a point A2′ is a point that can become so-called noise that deteriorates the prediction accuracy. However, according to the processing according to the second embodiment, since the intermediate value is selected from the solutions of the three straight lines, the solution corresponding to the straight line 58 is excluded. As described above, according to the processing according to the second embodiment, noise can be excluded in the process of the calculation processing, and thus durability against noise can be improved.

In addition, in the second embodiment, an example in which the midpoint of the solution of the linear equation is taken has been shown, but the intra prediction unit 114 may calculate the linear model using information other than the midpoint. For example, the intra prediction unit 114 may calculate the linear model based on the average value of the slopes of the linear equation calculated for the number of combinations of a plurality of points.

Also, the intra prediction unit 114 may obtain more or less linear equations than the numbers shown in FIGS. 6 and 7.

For example, the intra prediction unit 114 includes points A0 and L0, points A0 and L1, points A0 and L2, points A1 and L0, points A1 and L1, points A1 and L2, points A2 and L0, The solutions of the nine linear equations may be obtained by combining the points A2 and L1 and the points A2 and L2, respectively. Accordingly, the intra prediction unit 114 can calculate a linear model that further improves the prediction accuracy in CCLM prediction.

Alternatively, the intra prediction unit 114 can perform faster calculation by reducing the solution of the linear equation to be calculated.

[2-2. Information Processing Procedure According to Second Embodiment]
Next, a procedure of information processing according to the second embodiment will be described with reference to FIG. FIG. 8 is a flowchart showing the flow of information processing according to the second embodiment.

As illustrated in FIG. 8, the image encoding device 100 calculates a prediction value (Pred _L ) of the luminance signal in the processing target block (step S201).

Subsequently, the image encoding device 100 determines whether or not to perform CCLM prediction (step S202). When the CCLM prediction is not performed (step S202; No), the image coding apparatus 100 skips the subsequent process regarding the CCLM prediction. On the other hand, when CCLM prediction is performed (step S202; Yes), the image coding apparatus 100 specifies three combinations that connect each horizontal and vertical point in the reference pixel of the processing target block (step S203).

Then, the image coding apparatus 100 calculates the slope α of the linear equation in each combination (step S204). Further, the image coding apparatus 100 calculates the intermediate value of the α3 points obtained in step S204 (step S205).

The image coding apparatus 100 calculates the intercept β in the linear equation based on the calculated intermediate value (step S206). Then, the image coding apparatus 100 calculates the predicted value of the color difference signal in the processing target block based on the calculated values of α and β (step S207).

(3. Third embodiment)
Next, a third embodiment will be described. In the first and second embodiments, examples have been shown in which the intra prediction unit 114 obtains the intermediate value of the extracted plural points or the intermediate value of the slope of the linear equation. Here, in parallel calculation such as calculation of an intermediate value, a high-speed calculation can be performed by using a Lookup Table. Therefore, the intra prediction unit 114 according to the third embodiment uses the look-up table to perform the calculation process described in the first and second embodiments. Note that, in the third embodiment, the functions and configuration of the image encoding device 100 are the same as those in the first and second embodiments, so description will be omitted.

For example, the intra prediction unit 114 calculates a variable by using a process of referring to a lookup table in the process of calculating a variable (specifically, α and β) for specifying a linear model. For example, in the second embodiment, the intra prediction unit 114 performs processing such as “selecting the three furthest points in the space (block) and computing the linear equations in parallel to obtain the midpoint of the solution”. .. In this case, the intra prediction unit 114 repeats the processing of substituting the coordinates of three points into the above equation (3) and calculating α and β for each. Here, in the third embodiment, the intra prediction unit 114 performs the calculation by referring to the look-up table as shown in the following expression (4).

In the above formula (4), “Table” indicates a lookup table. In this way, by replacing the calculation of α and β that are variables of the linear equation with the lookup table reference, the intra prediction unit 114 can realize high-speed calculation of α and β with a relatively small circuit scale. ..

Here, in the above formula (4), when it is limited to “y _B >y _A ”, the table requires an area of “2 ⁽⁸⁺¹⁾ ×2 ⁸ =512×256” in the case of 8 bits. Therefore, in order to further reduce the scale, the intra prediction unit 114 can adopt a method of performing quantization. For example, when the above equation (4) is quantized, it can be expressed as follows.

In the above formula (5), “n” indicates an integer corresponding to the amount of quantization. For example, when n=16, the table has a size of “32×16”, and the size is reduced.

As described above, the intra prediction unit 114 can perform the calculation process shown in the first and second embodiments at a higher speed by adopting the method of referring to the lookup table.

(4. Modified example)
The information processing according to the present disclosure described above may be implemented in various different forms other than the above-described embodiments. Therefore, below, a modification of the information processing according to the present disclosure will be described.

In each of the above embodiments, the information processing device according to the present disclosure has been described as the image encoding device 100, but the information processing according to the present disclosure may be performed by the decoding device. That is, the information processing device according to the present disclosure is not limited to a device that performs image encoding, and may be a device that performs image decoding.

In each of the above-described embodiments, an example of calculating an intermediate value or an average value of a plurality of extracted points is shown, but the numerical value to be calculated is not limited to these examples, and any value can be used as long as it is a reference value May be adopted.

In the embodiment, the image encoding device 100 is described as an integrated device, but the image encoding device 100 may be realized by a plurality of devices.

(5. Other embodiments)
The processing according to each of the above-described embodiments may be performed in various different forms other than each of the above-described embodiments.

Further, of the processes described in the above embodiments, all or part of the processes described as being automatically performed can be manually performed, or the processes described as being manually performed. All or part of the above can be automatically performed by a known method. In addition, the processing procedures, specific names, information including various data and parameters shown in the above-mentioned documents and drawings can be arbitrarily changed unless otherwise specified. For example, the various information shown in each drawing is not limited to the illustrated information.

Also, each component of each device shown in the drawings is functionally conceptual, and does not necessarily have to be physically configured as shown. That is, the specific form of distribution/integration of each device is not limited to the one shown in the figure, and all or part of the device may be functionally or physically distributed/arranged in arbitrary units according to various loads and usage conditions. It can be integrated and configured.

Also, the above-described respective embodiments and modified examples can be appropriately combined within a range in which the processing content is not inconsistent.

Also, the effects described in this specification are merely examples and are not limited, and there may be other effects.

(6. Effect of information processing apparatus according to the present disclosure)
As described above, the information processing device according to the present disclosure (the image encoding device 100 in the embodiment) includes the intra prediction unit (intra prediction unit 114 in the embodiment). The intra-prediction unit extracts a plurality of points from reference pixels adjacent to the block defined in the image, and linearly predicts color information of pixels existing in the block based on the brightness information and the color information of the extracted points. Calculate the model.

As described above, the information processing apparatus according to the present disclosure reduces the number of coordinates for calculating brightness information and the like as compared with the conventional method of searching all horizontal and vertical points and calculating the maximum value and the minimum value. Therefore, high-speed calculation can be performed and the overall processing speed can be improved.

Also, the intra prediction unit extracts three points from the reference pixels that are vertically adjacent to each other in the block, and also extracts three points from the reference pixels that are horizontally adjacent to each other from the block, and the luminance information and the color of each of the three points are extracted. A midpoint in the information is specified, and a linear model is calculated based on the specified midpoint. As a result, the information processing apparatus can improve the processing speed while increasing the durability against noise.

In addition, the intra prediction unit extracts three consecutive points from the reference pixels that are vertically adjacent to each other in the block, and also extracts three consecutive points from the reference pixels that are horizontally adjacent to each other from the block. An intermediate point in the luminance information and color information is specified, and a linear model is calculated based on the specified intermediate point. As a result, the information processing apparatus can improve the processing speed while increasing the durability against noise.

The intra prediction unit extracts three consecutive three points, which are located at one end different from the base point of the block, from the reference pixels that are vertically adjacent to each other in the block, and the horizontal direction of the block is extracted. From the adjacent reference pixels, three consecutive points, which are located at one end different from the base point of the block, are extracted, and intermediate points in the luminance information and the color information of the three points are identified and identified. A linear model is calculated based on the midpoint. In this way, the information processing apparatus generates a linear model from the three points that are estimated to most represent the feature of the space (block), so the prediction accuracy can be further improved.

The intra prediction unit also calculates a linear model based on the average values of the extracted luminance information and color information of multiple points. As a result, the information processing device can improve the processing speed while increasing the durability against noise.

In addition, the intra prediction unit includes one of a plurality of points extracted from vertically adjacent reference pixels of the block, and one of a plurality of points extracted from horizontally adjacent reference pixels of the block. The number of combinations of the extracted plural linear equations is calculated, and the linear model is calculated based on the comparison of the slopes of the calculated linear equations. As a result, the information processing device can perform the prediction process with improved durability against noise.

In addition, the intra prediction unit calculates a linear model based on the intermediate value of the slope of the linear equation calculated for the number of combinations of multiple points. As a result, the information processing device can perform the prediction process with improved durability against noise.

Also, the intra prediction unit calculates a linear model based on the average value of the slopes of the linear equation calculated for the number of combinations of multiple points. As a result, the information processing device can perform the prediction process with improved durability against noise.

In addition, the intra prediction unit extracts three consecutive three points located at one end different from the base point of the block from the reference pixels adjacent to each other in the vertical direction of the block, and the horizontal prediction block of the block. From the reference pixels adjacent to each other in the direction, three consecutive points, which are located at one end different from the base point of the block, are extracted, and three linear equations combining each of the three points are calculated and calculated. A linear model is calculated based on the comparison of the slopes of the linear equations. As described above, the information processing apparatus generates a linear model from a combination of three points that are estimated to most represent the feature of the space (block), and thus the prediction accuracy can be further improved.

Also, the intra-prediction unit calculates variables using a process that refers to a lookup table (Lookup Table) in the variable calculation process for specifying the linear model. As a result, the information processing apparatus can significantly improve arithmetic processing such as parallel arithmetic.

(7. Hardware configuration)
The image coding apparatus 100 according to each of the embodiments described above is realized by, for example, a computer 1000 having a configuration illustrated in FIG. 9. FIG. 9 is a hardware configuration diagram illustrating an example of a computer 1000 that realizes the functions of the image encoding device 100 according to the present disclosure. The computer 1000 has a CPU 1100, a RAM 1200, a ROM (Read Only Memory) 1300, an HDD (Hard Disk Drive) 1400, a communication interface 1500, and an input/output interface 1600. The respective units of the computer 1000 are connected by a bus 1050.

The CPU 1100 operates based on a program stored in the ROM 1300 or the HDD 1400, and controls each part. For example, the CPU 1100 expands a program stored in the ROM 1300 or the HDD 1400 into the RAM 1200 and executes processing corresponding to various programs.

The ROM 1300 stores a boot program such as a BIOS (Basic Input Output System) executed by the CPU 1100 when the computer 1000 starts up, a program dependent on the hardware of the computer 1000, and the like.

The HDD 1400 is a computer-readable recording medium that non-temporarily records a program executed by the CPU 1100, data used by the program, and the like. Specifically, the HDD 1400 is a recording medium that records an information processing program according to the present disclosure, which is an example of the program data 1450.

The communication interface 1500 is an interface for connecting the computer 1000 to an external network 1550 (for example, the Internet). For example, the CPU 1100 receives data from another device or transmits the data generated by the CPU 1100 to another device via the communication interface 1500.

The input/output interface 1600 is an interface for connecting the input/output device 1650 and the computer 1000. For example, the CPU 1100 receives data from an input device such as a keyboard or a mouse via the input/output interface 1600. The CPU 1100 also transmits data to an output device such as a display, a speaker, a printer, etc. via the input/output interface 1600. Also, the input/output interface 1600 may function as a media interface for reading a program or the like recorded in a predetermined recording medium (medium). Examples of media include optical recording media such as DVD (Digital Versatile Disc) and PD (Phase change rewritable Disk), magneto-optical recording media such as MO (Magneto-Optical disk), tape media, magnetic recording media, and semiconductor memory. Is.

For example, when the computer 1000 functions as the image encoding device 100 according to the embodiment, the CPU 1100 of the computer 1000 executes the information processing program loaded on the RAM 1200 to realize the functions of the intra prediction unit 114 and the like. .. Further, the HDD 1400 stores the information processing program according to the present disclosure and various data used for information processing. Note that the CPU 1100 reads the program data 1450 from the HDD 1400 and executes the program data, but as another example, these programs may be acquired from another device via the external network 1550.

Note that the present technology may also be configured as below.
(1)
A plurality of points are extracted from reference pixels adjacent to the block defined in the image, and a linear model for predicting color information of pixels existing in the block is calculated based on the luminance information and the color information of the extracted points. An information processing device equipped with an intra prediction unit.
(2)
The intra prediction unit is
Three points are extracted from vertically adjacent reference pixels in the block, three points are extracted from horizontally adjacent reference pixels in the block, and three intermediate points in the luminance information and the color information are extracted respectively. The information processing apparatus according to (1), wherein the linear model is specified and the linear model is calculated based on the specified midpoint.
(3)
The intra prediction unit is
Three consecutive points are extracted from vertically adjacent reference pixels in the block, and three consecutive points are extracted from horizontally adjacent reference pixels in the block. The luminance information and the color information of each of the three points are extracted. The information processing device according to (1) or (2), wherein the linear model is calculated based on the specified intermediate point.
(4)
The intra prediction unit is
From the reference pixels adjacent to each other in the vertical direction in the block, three continuous points located at one end different from the base point of the block are extracted, and the adjacent three pixels are adjacent to each other in the horizontal direction. From the reference pixel, three consecutive three points, which are located at one end different from the base point of the block, are extracted, the intermediate points in the luminance information and the color information of the three points are identified, and the identified intermediate points The information processing apparatus according to any one of (1) to (3), wherein the linear model is calculated based on points.
(5)
The intra prediction unit is
The information processing apparatus according to any one of (1) to (4), wherein the linear model is calculated based on average values of the extracted luminance information and color information of a plurality of points.
(6)
The intra prediction unit is
A linear equation obtained from any one of a plurality of points extracted from vertically adjacent reference pixels of the block and any one of a plurality of points extracted from horizontally adjacent reference pixels of the block The information processing device according to (1), wherein the linear model is calculated based on a comparison of the slopes of the calculated linear equations.
(7)
The intra prediction unit is
The information processing apparatus according to (6), wherein the linear model is calculated based on an intermediate value of the slopes of the linear equation calculated for the number of combinations of the plurality of points.
(8)
The intra prediction unit is
The information processing apparatus according to (6) or (7), wherein the linear model is calculated based on an average value of the slopes of the linear equations calculated for the number of combinations of the plurality of points.
(9)
The intra prediction unit is
From the reference pixels adjacent to each other in the vertical direction in the block, three continuous points located at one end different from the base point of the block are extracted, and the adjacent three pixels are adjacent to each other in the horizontal direction. From the reference pixel, three consecutive three points located at one end different from the base point of the block are extracted, three linear equations combining the three points are calculated, and the calculated linear equations are calculated. The information processing apparatus according to any one of (6) to (8), wherein the linear model is calculated based on a comparison of the slopes of.
(10)
The intra prediction unit is
The information processing apparatus according to any one of (1) to (9), wherein in the process of calculating a variable for specifying the linear model, the variable is calculated using a process that refers to a Lookup Table. ..
(11)
Computer
A plurality of points are extracted from reference pixels adjacent to the block defined in the image, and a linear model for predicting color information of pixels existing in the block is calculated based on the luminance information and the color information of the extracted points. Information processing method.
(12)
Computer,
A plurality of points are extracted from reference pixels adjacent to the block defined in the image, and a linear model for predicting color information of pixels existing in the block is calculated based on the luminance information and the color information of the extracted points. An information processing program that functions as an intra prediction unit.

100 Image Coding Device 101 A/D Conversion Unit 102 Sorting Buffer 103 First Operation Unit 104 Orthogonal Transformation Unit 105 Quantization Unit 106 Encoding Unit 107 Storage Buffer 108 Inverse Quantization Unit 109 Inverse Orthogonal Transformation Unit 110 Second Operation Unit 111 filter 112 frame memory 113 switch 114 intra prediction unit 1141 luminance color difference separation unit 1142 luminance signal intra prediction unit 1143 luminance signal downsampling unit 1144 color difference signal intra prediction unit 115 inter prediction unit 116 predicted image selection unit 117 rate control unit

Claims

A plurality of points are extracted from reference pixels adjacent to the block defined in the image, and a linear model for predicting color information of pixels existing in the block is calculated based on the luminance information and the color information of the extracted points. An information processing device equipped with an intra prediction unit.
The intra prediction unit is
Three points are extracted from vertically adjacent reference pixels in the block, three points are extracted from horizontally adjacent reference pixels in the block, and three intermediate points in the luminance information and the color information are extracted respectively. The information processing apparatus according to claim 1, wherein the linear model is specified and calculated based on the specified midpoint.
The intra prediction unit is
Three consecutive points are extracted from vertically adjacent reference pixels in the block, and three consecutive points are extracted from horizontally adjacent reference pixels in the block. The luminance information and the color information of each of the three points are extracted. The information processing apparatus according to claim 1, wherein the linear model is calculated on the basis of the specified intermediate point in the.
The intra prediction unit is
From the reference pixels adjacent to each other in the vertical direction in the block, three continuous points located at one end different from the base point of the block are extracted, and the adjacent three pixels are adjacent to each other in the horizontal direction. From the reference pixel, three consecutive three points, which are located at one end different from the base point of the block, are extracted, the intermediate points in the luminance information and the color information of the three points are identified, and the identified intermediate points The information processing apparatus according to claim 1, wherein the linear model is calculated based on points.
The intra prediction unit is
The information processing apparatus according to claim 1, wherein the linear model is calculated based on average values of the extracted luminance information and color information of a plurality of points.
The intra prediction unit is
A linear equation obtained from any one of a plurality of points extracted from vertically adjacent reference pixels of the block and any one of a plurality of points extracted from horizontally adjacent reference pixels of the block The information processing apparatus according to claim 1, wherein the linear model is calculated based on a comparison of the slopes of the calculated linear equations.
The intra prediction unit is
The information processing apparatus according to claim 6, wherein the linear model is calculated based on an intermediate value of the slopes of the linear equation calculated for the number of combinations of the plurality of points.
The intra prediction unit is
The information processing apparatus according to claim 6, wherein the linear model is calculated based on an average value of the slopes of the linear equation calculated for the number of combinations of the plurality of points.
The intra prediction unit is
From the reference pixels adjacent to each other in the vertical direction in the block, three continuous points located at one end different from the base point of the block are extracted, and the adjacent three pixels are adjacent to each other in the horizontal direction. From the reference pixel, three consecutive three points located at one end different from the base point of the block are extracted, three linear equations combining the three points are calculated, and the calculated linear equations are calculated. The information processing apparatus according to claim 6, wherein the linear model is calculated based on a comparison of the slopes of.
The intra prediction unit is
The information processing apparatus according to claim 1, wherein in the process of calculating the variable for specifying the linear model, the variable is calculated by using a process that refers to a lookup table.
Computer
A plurality of points are extracted from reference pixels adjacent to a block defined in the image, and a linear model for predicting color information of pixels existing in the block is calculated based on the brightness information and the color information of the extracted points. Information processing method.
Computer,
A plurality of points are extracted from reference pixels adjacent to the block defined in the image, and a linear model for predicting color information of pixels existing in the block is calculated based on the luminance information and the color information of the extracted points. An information processing program that functions as an intra prediction unit.