JP2018078545A - Method for selecting prediction mode of intraprediction, video coding device and image processing apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a method for selecting prediction mode of intraprediction capable of improving video encoding efficiency and processing speed, while reducing the video encoding hardware mounting cost, and to provide a video coding device and an image processing apparatus.SOLUTION: A method for selecting prediction mode of intraprediction includes a step of calculating multiple prediction costs corresponding to multiple prediction modes in the intraprediction based on a block, when a conversion unit performs operation based on a preset conversion index, a step of selecting multiple candidate prediction modes from the prediction mode based on the prediction cost, a step of calculating multiple distortion costs corresponding to the candidate prediction modes in multiple conversion indices, based on the prediction cost corresponding to the block and candidate prediction mode, and a step of selecting one of the candidate prediction modes based on the distortion cost, as a prediction mode used for intraprediction corresponding to the block.SELECTED DRAWING: Figure 4

Description

  The present invention relates to a method for selecting a prediction mode for intra prediction, a video encoding device, and an image processing apparatus.

  With the recent development of new technologies for applications such as networks, communication systems, displays and computers, many applications are all like high video compression rate, virtual reality (VR) and 360 degree video content, for example. There is a need for an efficient video coding solution. It is common practice to improve the video resolution so that more details can be seen in the video in order to provide an on-the-spot visual effect. VR technology is usually realized by a head mounted device (HMD), and the distance between the head mounted device and the eyes is sufficiently close, so that the necessary video content resolution is 4K-8K, Is preferably improved to 32K or more. Further, since the refresh rate of the screen also affects the feeling of using the VR, it is preferable to increase the refresh rate to 30 images per second, 90 images per second, and further 120 images per second. Based on the above requirements, conventional High Efficiency Video Coding (HEVC) (also referred to as H.265) cannot provide a better visual effect and experience to the user.

  In order to further improve the coding efficiency and image quality for digital video, the Joint Video Exploration Team (JVET) has developed several enhanced video coding techniques that solve potential demands in a joint search test model ( This is applied to the Joint Exploration Test Model (JEM), and the advancement of video coding technology is being promoted on a trial basis. The intra prediction technology adopted by JEM is expanded from 35 kinds of prediction modes provided in the conventional HEVC to 67 kinds of prediction modes, and is used for more accurate angle prediction.

  Further, JEM introduces a mode-dependent non-separable secondary transform (NSST) technique in a transform unit (Transform Unit; TU). NSST can be implemented between primary transform (also called core transform or first transform) and quantization of a video encoder, It can also be realized in de-quantization and inverse linear transformation of the encoder. NSST can reach better compression ratios in directional texture patterns, but requires relatively complex operations.

  The present invention relates to a method for selecting a prediction mode for intra prediction, a video encoding device, and an image, which can improve the efficiency and processing speed of video encoding and reduce the cost of hardware implementation of video encoding. An object is to provide a processing apparatus.

  The method for selecting a prediction mode for intra prediction according to the present invention includes the following steps. When the conversion unit performs an operation based on a preset conversion index, calculating a plurality of prediction costs corresponding to a plurality of prediction modes in intra prediction based on a block of the input image, the plurality of prediction costs Selecting a plurality of candidate prediction modes from the plurality of prediction modes based on the prediction cost corresponding to the block and the plurality of candidate prediction modes, the plurality of candidate prediction modes corresponding to a plurality of transform indexes Calculating a plurality of distortion costs, and selecting one of the plurality of candidate prediction modes based on the distortion costs to make a prediction mode used for in-screen prediction corresponding to the block.

  The video encoding device of the present invention includes at least a transform unit and an intra prediction unit. The transform unit is used to transform a residual corresponding to a block of the input image based on a plurality of transform indexes. An in-screen prediction unit is coupled to the conversion unit. When the conversion unit performs an operation based on a preset conversion index, the intra prediction unit obtains a block of the input image, and a plurality of prediction modes corresponding to a plurality of prediction modes in the intra prediction based on the block are obtained. Calculate the estimated cost. The preset conversion index is one of the plurality of conversion indexes. The in-screen prediction unit selects a plurality of candidate prediction modes from the prediction mode based on the prediction cost, and the plurality of conversion units based on the prediction cost to which the block and the plurality of candidate prediction modes correspond. Calculating a plurality of distortion costs corresponding to the plurality of candidate prediction modes in the transformed index, selecting one of the plurality of candidate prediction modes based on the distortion cost, and performing an intra-screen prediction corresponding to the block The prediction mode to be used is used.

  The image processing apparatus of the present invention includes a processor and a memory. When the processor converts a residual based on a preset conversion index, it calculates a plurality of prediction costs corresponding to a plurality of prediction modes in intra prediction based on a block of the input image. The residual corresponds to the block. The processor selects a plurality of candidate prediction modes from the plurality of prediction modes based on the prediction cost, and the candidate prediction mode is selected in a plurality of conversion indexes based on the prediction cost corresponding to the block and the candidate prediction mode. Calculate the corresponding multiple distortion costs. The preset conversion index is one of the plurality of conversion indexes. The processor selects one of the plurality of candidate prediction modes based on the distortion cost, and sets it as the prediction mode used by the intra prediction corresponding to the block.

  Based on the above, when the selection method of the intra prediction mode, the video encoding device, and the image processing apparatus described in the embodiments of the present invention select the prediction mode of the intra prediction, first, the conversion unit is set in advance. Based on the block of the input image (for example, the conversion unit is set to an operation mode in which the second conversion unit is disabled and the residual is converted only by the first conversion unit). Then, the prediction cost corresponding to each prediction mode in the intra prediction is calculated, and a plurality of candidate prediction modes are selected from these prediction modes. Next, a candidate prediction mode having an optimal (for example, the lowest) distortion cost from these candidate prediction modes according to the prediction cost corresponding to the candidate prediction mode and the block is set as a prediction mode to be used. In other words, the embodiment of the present invention does not calculate the prediction cost corresponding to each prediction mode for different operation modes in the conversion unit (i.e., when converting the residual based on different conversion indexes), The prediction cost corresponding to each prediction mode in the intra prediction is calculated once for the prediction operation mode in the conversion unit (that is, when the residual is converted based on a preset conversion index). Then, by combining the cases where the prediction cost and the conversion unit having different conversion units are used to convert the residual, the distortion cost is calculated, and the subsequent candidate prediction mode is selected. Thus, the embodiment of the present invention can greatly reduce the amount of calculation of the prediction cost, improve the efficiency and processing speed of video encoding, and reduce the cost of hardware implementation of video encoding.

  The method for selecting the mode of intra prediction of the present invention, the video encoding device, and the image processing apparatus greatly reduce the calculation amount of the prediction cost, improve the efficiency and processing speed of the video encoding, and improve the video encoding. Hardware implementation costs can be reduced.

FIG. 2 is a structural block diagram of a video encoding device according to an embodiment of the present invention. 1 is a block diagram of an image processing apparatus consistent with an embodiment of the present invention. It is a figure which shows two steps of the prediction in a screen in a joint search test model (JEM). It is a flowchart of the method of selecting the prediction mode of the prediction in a screen corresponding to the Example of this invention.

  In order to make the above features and advantages of the present invention easier to understand, a detailed description will be given below in conjunction with the embodiments and the drawings.

  FIG. 1 is a structural block diagram of a video encoding device 100 according to an embodiment of the present invention. The video encoding device 100 performs video encoding based on a plurality of input images IM in the obtained input video, thereby reducing the data amount of the input video and facilitating transmission and storage of the input video. . The video encoding used by the video encoding device 100 may be a joint search test model (JEM), and in the video conversion, the first and second conversions (e.g., NSST) consistent with embodiments of the present invention. ) May be used.

  The video encoding device 100 according to the present embodiment mainly includes a transform / quantization unit 110, an inverse quantization / inverse transform unit 120, a prediction unit 130, and an adder located at the input terminal N1 of the video encoding device 100. 140, an adder 150 located at the output terminal N2 of the inverse quantization / inverse transform unit 120, a screen buffer 160, and an entropy coding unit 170. The transform / quantization unit 110 includes a transform unit 112 and a quantization unit 115. The prediction unit 130 includes an intra-screen prediction unit 132 and an inter-screen prediction unit 134. The adder 140 subtracts the information provided by the prediction unit 130 from the input image IM to obtain a residual MR of the input image IM.

  In the EM, the conversion unit 112 includes a first conversion unit 113 and a second conversion unit 114. The first conversion unit 113 performs a first conversion (also referred to as core conversion or primary conversion) on the residual MR of the input image IM. The second conversion unit 114 performs a second conversion on the residual subjected to the first conversion. The second transformation here is a mode-dependent non-separable secondary transformation (NSST). The NSST residual processing may be related to the intra prediction mode selected and used by the prediction unit 130 (for example, the intra prediction unit 132). NSST in JEM can comprise three types of transform cores, and the intra prediction unit can selectively use these transform cores to enhance the efficiency of residual coding. In other words, the JEM selectively performs residual encoding using one of the three transform cores in the first transform and NSST, or disables NSST and only the first transform. Can be used to perform residual encoding. In this embodiment, the operation mode of NSST is represented by a plurality of “conversion indexes”. One of the transform indexes represents that the transform unit 112 transforms the residual of the current block without using the second transform unit 114, and this operation mode is represented by “preset transform index”. Can. The conversion index other than the preset conversion index is one of the conversion units 112 out of at least one conversion core in the second conversion unit 114 (NSST used in the present invention includes three types of conversion cores). Is used to represent the mode of operation for transforming the residual of the current block. In other words, the present invention includes four types of conversion indexes, each of which disables NSST (the conversion index is “0”), and performs NSST using the first conversion core (the conversion index is NSST using the second conversion core (conversion index is “2”) and NSST using the third conversion core (conversion index is “1”). 3 ”).

  The data TD subjected to the residual transform by the transform unit 112 becomes the data DA after being processed by the quantization unit 115, and becomes the video data VD compressed by the processing of the entropy encoding unit 170. The video data VD may include various intra prediction modes and inter prediction modes generated by the prediction unit 130 in addition to the data DA.

  In order to simulate the video-encoded data, the video encoding device 100 reduces the data DA to video-encoded image data by the inverse quantization unit 122 and the inverse transform unit 124 in the inverse quantization and inverse transform unit 120. . This image data is temporarily stored in the screen buffer 160 through the processing of the adder 150 and the input image IM. The video-encoded image data can be used by the intra prediction unit 132 and the inter prediction unit 134 as the mode prediction of the current block.

  The in-screen prediction unit 132 performs pixel value prediction and residual conversion on the block being processed by the analyzed blocks on the same screen. The inter-screen prediction unit 134 performs pixel prediction and residual conversion on a block between a plurality of consecutive input images.

  Each functional block in FIG. 1 may be realized by a hardware method or a software program or a firmware module method. FIG. 2 is a block diagram of an image processing apparatus 200 consistent with the embodiment of the present invention. If the video encoding device 100 in FIG. 1 is implemented by a software program or firmware module, it may be executed by the processor 210 and the memory 220 in the image processing apparatus 200 to implement an embodiment of the present invention. The memory 220 can store each software program or firmware module in the video encoding device 100 represented by a command. The processor 210 can access the memory 220 to execute these software programs or firmware modules. The processor 210 may be a central processing unit, a drawing processing unit, a microprocessor, a field programmable logic gate array, etc.

  In JEM intra-screen prediction technology, the prediction mode of intra-screen encoding to be used for the current block to be encoded is determined in two stages. FIG. 3 is a diagram showing two stages of intra-screen prediction in JEM. The first stage ST1 is a rough mode detection (RMD) stage. In detail, the RMD stage includes two sub-stages ST11 and ST12. These two sub-stages ST11 and ST12 can be realized by the intra-screen prediction mode 132 in FIG. The sub-stage ST11 includes a plurality of intra-screen prediction modes (35 to 67 types of intra-screen prediction modes in JEM) to which the current block corresponds using the sum of absolute transform difference (SATD) method. ) Prediction cost (also referred to as a SATD cost) is calculated, and here it is referred to as “calculate the SATD cost for intra-screen prediction”. The sub-stage ST12 selects a plurality of candidate prediction modes from the plurality of intra-screen prediction modes based on these prediction costs, and is referred to as “select candidate prediction mode” here. The application of the present embodiment may adjust the number of candidate prediction modes selected according to the necessity. For example, three to five intra-screen prediction modes having a low SATD cost as candidate prediction modes. You may choose. This embodiment implements the embodiment of the present invention by “selecting three prediction modes as candidate prediction modes”.

  The second stage ST2 is a rate-distortion optimization (RDO) stage. Specifically, the stage ST2 includes four sub-stages ST21 to ST24. The sub-stage 21 can be realized by the first conversion unit 113 of FIG. The sub-stage ST22 can be realized by the second conversion unit 114 of FIG. The sub-stage ST23 can be realized by the quantization unit 115 of FIG. The sub-stage ST24 can be realized by one of the intra-screen prediction unit 132 or the quantization unit 115 of FIG. The application of the embodiment of the present invention may adjust the functional blocks for realizing each of the sub-stages as needed, and the present invention is not limited to this.

The sub-stage ST21 performs first transformation / core transformation / primary transformation on the current block and these candidate prediction modes. In order to enhance the coding efficiency, in the present embodiment, in the sub-stage ST22, the second transformation (for example, NSST) is performed on the residual data of the current block on which the first transformation has been performed. The sub-stage ST23 calculates a rate-distortion cost (RDCost) corresponding to each candidate prediction mode by performing quantization coding on the residual data of the current block of the sub-stage ST22, thereby calculating the distortion cost. And In the present invention, the rate distortion cost is the distortion cost. The sub-stage ST24 selects a candidate prediction mode having an optimal rate distortion cost between the actual number of encoded bits and the quantization distortion, and uses a prediction mode to be used for intra prediction corresponding to the current block. Here, it is referred to as “selecting the prediction mode used by the current block”.

  In the design of JEM, NSST has four types of operation modes since it has three types of conversion units. These operating modes are represented by different conversion indexes. Therefore, each candidate prediction mode needs to be calculated in a different NSST operation mode. Of particular note, JEM has 67 types of intra prediction modes and 4 types of NSST operation modes (represented by NSST conversion indexes (“0” to “3”)). JEM is an NSST operation with different in-screen prediction modes, because the optimal in-screen prediction mode can be calculated accurately, and the results of the RDO stage (selection for candidate prediction mode) can vary due to different NSST operation modes. In each mode, the RMD stage ST1 and the RDO stage ST2 are executed, and it is recognized that the intra prediction mode finally selected in this way is relatively accurate.

  Viewed from another perspective, NSST is applied to a second preset transformation in the screen to further reduce the number of residual bits. The process of selecting the intra prediction mode of the above four types of NSST conversion indexes can be generally described as the following operations 1 to 8.

  Calculation 1: RMD stage when NSST conversion index is “0” (selecting three candidate prediction modes from 67 types of intra prediction modes based on SATD cost).

  Arithmetic 2: RDO stage when NSST conversion index is “0” (optimal in-screen prediction mode is selected from three candidate prediction modes).

  Calculation 3: RMD stage when NSST conversion index is “1” (selecting three candidate prediction modes from 67 types of intra-screen prediction modes based on SATD cost).

  Arithmetic 4: RDO stage when NSST conversion index is “1” (optimal in-screen prediction mode is selected from three candidate prediction modes).

  Calculation 5: RMD stage when the NSST conversion index is “2” (selecting three candidate prediction modes from 67 types of intra-screen prediction modes based on the SATD cost).

  Arithmetic 6: RDO stage when NSST conversion index is “2” (selecting an optimal in-screen prediction mode from three candidate prediction modes).

  Calculation 7: RMD stage when the NSST conversion index is “3” (selecting three candidate prediction modes from 67 types of intra-screen prediction modes based on the SATD cost).

  Arithmetic 8: RDO stage when NSST conversion index is “3” (optimal in-screen prediction mode is selected from three candidate prediction modes).

  As can be seen from the above operations 1 to 8, even in the SATD method that can quickly calculate which intra-frame encoding has the smallest cost for encoding a block, the RMD stage In order to calculate the smallest SATD cost in the three candidate prediction modes, it is necessary to calculate a plurality of times (for example, calculation 1, calculation 3, calculation 5, and calculation 7).

  However, according to the embodiment of the present invention, the calculation for the SATD cost in the sub-stage ST11 of FIG. 3 is not directly related to the operation mode of the NSST, in other words, the calculation of the SATD cost and the operation mode of the NSST are not related. The impact on the final video encoding result is not significant. Therefore, in different NSST conversion indexes, the SATD cost of each intra-screen prediction mode can be used to calculate the operation mode in different NSST in the subsequent RDO stage. As a result, the embodiment of the present invention enables the SATD cost for these intra prediction modes only when NSST is set to a preset conversion index (for example, the conversion index of NSST is set to “0”). Is calculated once, these SATD costs are temporarily stored, and “when NSST is set to another conversion index (for example, the conversion index of NSST is set to“ 1 ”to“ 3 ”)” By eliminating the “calculate SATD cost” step, the calculation process can be saved significantly. In other words, the embodiment of the present invention temporarily stores the calculation result of the SATD cost of the calculation 1 and omits the calculations 3, 5, and 7 to calculate the SATD cost obtained by the calculation 1. By performing the calculations 4, 6, and 8, the amount of calculation can be saved.

  FIG. 4 is a flowchart of a method for selecting a prediction mode for intra-screen prediction consistent with an embodiment of the present invention. The method described in FIG. 4 can be applied to the video encoding device 100 illustrated in FIG. 1 and the image processing apparatus 200 illustrated in FIG. 1 and 4, in step S410, the operation mode of the second conversion unit 114 in the conversion unit 112 is invalidated, that is, the conversion index of the second conversion unit 114 is set to “0”. In step S420, when the second conversion unit 114 performs an operation based on the preset conversion index, the in-screen prediction unit 132 uses the sum of absolute conversion differences (SATD) method to calculate the current input image IM. Based on the block, a plurality of prediction costs corresponding to a plurality of prediction modes in the intra prediction are calculated. The predicted cost is a SATD cost.

  In step S430, the intra prediction unit 132 selects a plurality of candidate prediction modes from a plurality of intra prediction modes (for example, 67 types of intra prediction modes) based on the prediction cost in step S420. In this embodiment, the optimum prediction cost can be found from the prediction costs corresponding to the 67 types of intra-screen prediction modes. The number of intra prediction modes is greater than the number of selected candidate prediction modes. For example, an in-screen prediction mode corresponding to the lowest three prediction costs is searched from these prediction costs, and the prediction mode is selected in advance.

  After selecting these candidate prediction modes in step S440, the in-screen prediction unit 132 temporarily stores the prediction costs to which these candidate prediction modes correspond, as used in subsequent steps. In some embodiments, the intra-screen prediction unit 132 may temporarily store the prediction cost corresponding to each intra-screen prediction mode.

  In step S450, in order to calculate a plurality of distortion costs corresponding to these candidate prediction modes in a plurality of transformation indexes (this embodiment includes four transformation indexes “0” to “3”), transformation / quantization is performed. Rate distortion based on the prediction cost corresponding to the current block and the plurality of candidate prediction modes selected in step S430 by the first transform unit 113, the second transform unit 114, and the quantization unit 115 in the quantization unit 110 Optimization (Rate-Distortion Optimization; RDO) detection may be performed. The distortion cost of the embodiment of the present invention is realized by the rate distortion cost described in the sub-stage ST23 in the RDO stage ST2 in FIG. In other words, the distortion cost calculation method in step S450 can refer to the RDO stage ST2 in FIG.

  In step S460, it is determined whether or not the conversion index set in the second conversion unit 114 is the last conversion index (that is, whether or not it is the conversion index “3”). When the conversion index set by the second conversion unit 114 is not the conversion index “3”, the process enters from step S460 to step S470, and 1 is added to the conversion index set by the second conversion unit 114. Further, after adding 1 to the conversion index, the process returns to step S450, and the distortion cost corresponding to each candidate prediction cost in the case of the conversion index of NSST is calculated. Based on steps S450 to S470, the present invention can calculate the distortion cost to which these candidate prediction modes correspond in the case of different transform indexes.

  In step S480, the intra-screen prediction unit 134 (or other component that executes step S480) selects one of these candidate prediction modes based on the distortion cost obtained by the calculation in step S450. Thus, the prediction mode used by the intra prediction corresponding to the current block can be set.

  Table 1 is a comparison between the video compression rate and the image quality employing the embodiment of the present invention. “Y”, “U”, and “V” in Table 1 are color coding methods. “Y” represents luminance (Luminance), and “U” and “V” represent chrominance (Chroma) and chroma (Chroma), respectively.

Table 1 shows a comparison result between an image to be decoded after video coding using the embodiment of the present invention and an original pattern. The difference between the Y, U, and V values of the video-encoded image and the original pattern is very small, but the encoding time is shortened by 9% and the video encoding processing speed is greatly improved. I understand.

  Based on the above, when the selection method of the intra prediction mode, the video encoding device, and the image processing apparatus described in the embodiments of the present invention select the prediction mode of the intra prediction, first, the conversion unit is set in advance. Based on the block of the input image (for example, the conversion unit is set to an operation mode in which the second conversion unit is disabled and the residual is converted only by the first conversion unit). Then, the prediction cost corresponding to each prediction mode in the intra prediction is calculated, and a plurality of candidate prediction modes are selected from these prediction modes. Next, a candidate prediction mode having an optimal (for example, the lowest) distortion cost from these candidate prediction modes according to the prediction cost corresponding to the candidate prediction mode and the block is set as a prediction mode to be used. In other words, the embodiment of the present invention does not calculate the prediction cost corresponding to each prediction mode for different operation modes in the conversion unit (i.e., when converting the residual based on different conversion indexes), The prediction cost corresponding to each prediction mode in the in-screen prediction is calculated once for the prediction operation mode in the conversion unit (that is, when the residual is converted based on a preset conversion index). Then, by combining the cases where the prediction cost and the conversion unit having different conversion units are used to convert the residual, the distortion cost is calculated, and the subsequent candidate prediction mode is selected. Thus, the embodiment of the present invention can greatly reduce the amount of calculation of the prediction cost, improve the efficiency and processing speed of video encoding, and reduce the cost of hardware implementation of video encoding.

  While the present invention has been disclosed in the examples above, it is not intended to limit the invention and those skilled in the art will make minor variations and modifications without departing from the spirit and scope of the invention. Therefore, the protection scope of the present invention is based on the content limited to the scope of the patent claims later.

100 video encoding device 110 transform / quantization unit 112 transform unit 113 first transform unit 114 second transform unit 115 quantization unit 120 inverse quantization / inverse transform unit 122 inverse quantization unit 124 inverse transform unit 130 prediction unit 132 Intra-screen prediction unit 134 Inter-screen prediction unit 140, 150 Adder 160 Screen buffer 170 Entropy encoding unit 200 Image processing device 210 Processor 220 Memory S410 to S480 Step ST1 of method for selecting prediction mode of intra-screen prediction Rough mode detection ( RMD) stage ST11 STAT cost of intra prediction is calculated ST12 candidate prediction mode is selected ST2 rate distortion optimization (RDO) stage ST21 first conversion is performed ST22 second conversion is performed S 23 ST24 that performs quantization coding IM selecting prediction mode used by current block Input image MR Input image residual TD, DA data VD Video data N1 Video encoding device input terminal N2 Inverse quantization / inverse transformation unit Output terminal

Claims (20)

When the conversion unit performs an operation based on a preset conversion index, calculating a plurality of prediction costs corresponding to a plurality of prediction modes in intra prediction based on a block of the input image;
Selecting a plurality of candidate prediction modes from the plurality of prediction modes based on the plurality of prediction costs;
Based on the prediction cost to which the block and the plurality of candidate prediction modes correspond, calculate a plurality of distortion costs to which the plurality of candidate prediction modes correspond in a plurality of conversion indexes, and the preset conversion index is the A step that is one of a plurality of transform indexes;
Selecting one of the plurality of candidate prediction modes based on the distortion cost and setting the prediction mode to be used for the intra-screen prediction corresponding to the block. Method.   The prediction of intra prediction according to claim 1, wherein the conversion unit includes a first conversion unit and a second conversion unit, and the second conversion unit uses non-separable secondary conversion (NSST). How to select a mode. The second conversion unit includes at least one conversion core;
The preset transformation index is used to represent an operation mode in which the transformation unit transforms the residual of the block without using the second transformation unit, and other than the preset transformation index. The transform index is used to represent an operation mode in which the transform unit transforms the residual of the block using one of at least one transform core in the second transform unit. The method of selecting the prediction mode of the in-screen prediction described in 2.   The screen according to claim 1, wherein the plurality of prediction costs corresponding to the plurality of prediction modes in the intra-screen prediction are calculated based on the block of the input image by a method of sum of absolute conversion differences (SATD). A method of selecting a prediction mode for intra prediction.   The plurality of candidate prediction modes that correspond to the plurality of transform indexes based on the prediction cost that is tested using rate distortion optimization (RDO) and that corresponds to the block and the plurality of candidate prediction modes. The method for selecting a prediction mode for intra prediction according to claim 1, wherein a distortion cost of the image is calculated.   The method of selecting a prediction mode for intra prediction according to claim 1, further comprising temporarily storing the prediction cost corresponding to the candidate prediction mode after selecting the plurality of candidate prediction modes.   The in-screen prediction according to claim 1, wherein the video encoding used for the in-screen prediction is a joint search test model (JEM), and the number of the plurality of prediction modes is larger than the number of the plurality of candidate prediction modes. A method of selecting a prediction mode of prediction. A transform unit used to transform a residual corresponding to a block of input images based on a plurality of transform indexes;
When the conversion unit is coupled to the conversion unit and performs an operation based on a preset conversion index, a block of the input image is acquired, and a plurality of prediction modes in the intra prediction are supported based on the block. Calculating a plurality of prediction costs, wherein the preset conversion index is one of the plurality of conversion indexes, and an in-screen prediction unit,
The intra-screen prediction unit selects a plurality of candidate prediction modes from the plurality of prediction modes based on the plurality of prediction costs, and the conversion is performed based on the prediction costs corresponding to the block and the plurality of candidate prediction modes. Calculating a plurality of distortion costs corresponding to the plurality of candidate prediction modes in the plurality of transform indexes of a unit, and selecting one of the plurality of candidate prediction modes based on the distortion cost to correspond to the block A video encoding device having a prediction mode used for intra prediction. The conversion unit is
A first conversion unit for performing a first conversion on the residual;
A second transformation unit that generates a residual transformed selectively using a non-separable quadratic transformation with respect to the residual subjected to the first transformation as a second transformation; and The video encoding device of claim 8, comprising: The conversion unit includes at least one conversion core;
The preset conversion index is used to represent an operation mode in which the conversion unit does not use the second conversion unit but converts the residual using the first conversion unit; The transformation index other than a preset transformation index is used to transform the residual using the transformation unit using one of at least one transformation core in the first transformation unit and the second transformation unit. The video encoding device of claim 9, wherein the video encoding device is used to represent a mode of operation to be performed.   The video code according to claim 8, wherein the intra prediction unit calculates the plurality of prediction costs corresponding to the plurality of prediction modes in the intra prediction based on the block by a method of sum of absolute conversion differences. Device.   The plurality of candidate prediction modes in the plurality of transform indexes correspond based on the prediction cost that the intra prediction unit examines using rate distortion optimization and the block and the plurality of candidate prediction modes correspond. The video encoding device of claim 8, wherein the plurality of distortion costs are calculated.   The video encoding device according to claim 8, wherein the prediction cost corresponding to the candidate prediction mode is temporarily stored after the intra prediction unit selects the plurality of candidate prediction modes. The video of claim 8, wherein the video encoding used for the video encoding device is a joint search test model (JEM) and the number of the plurality of prediction modes is greater than the number of the plurality of candidate prediction modes. Encoding device. A processor;
A memory coupled to the processor;
When the processor converts a residual based on a preset conversion index, the processor calculates a plurality of prediction costs corresponding to a plurality of prediction modes in intra prediction based on a block of an input image, Corresponding to the block, the processor selects a plurality of candidate prediction modes from the plurality of prediction modes based on the plurality of prediction costs, and based on the prediction cost corresponding to the block and the plurality of candidate prediction modes, Calculating a plurality of distortion costs corresponding to the plurality of candidate prediction modes in a plurality of conversion indexes, and the preset conversion index is one of the plurality of conversion indexes;
The image processing apparatus, wherein the processor selects one of the plurality of candidate prediction modes based on the distortion cost and sets the prediction mode to be used for intra prediction corresponding to the block.   The processor performs a first transformation on the residual, and a second transformation is performed using a non-separable quadratic transformation on the residual on which the first transformation has been performed. The image processing apparatus according to claim 15, wherein a residual is generated. The second transformation includes at least one transformation core;
The preset transformation index is used to represent an operation mode in which the processor does not use the second transformation and transforms the residual using the first transformation. The transformation index other than the transformation index represents an operation mode in which the processor transforms the residual using one of at least one transformation core in the first transformation and the second transformation. The image processing apparatus according to claim 16, wherein the image processing apparatus is used.   The image processing device according to claim 15, wherein the processor calculates the plurality of prediction costs corresponding to the plurality of prediction modes in the intra prediction based on the block by a method of sum of absolute conversion differences.   The processor checks using rate distortion optimization, and the plurality of candidate prediction modes correspond to the plurality of transform indexes based on the prediction cost to which the block and the plurality of candidate prediction modes correspond. The image processing apparatus according to claim 15, wherein a distortion cost is calculated. After the processor selects the plurality of candidate prediction modes, temporarily stores the prediction cost corresponding to the candidate prediction mode;
The image processing apparatus according to claim 15, wherein the video encoding to be used is a joint search test model (JEM), and the number of the plurality of prediction modes is larger than the number of the plurality of candidate prediction modes.