RU2768971C1 - Moving image encoding device, moving image encoding method, moving image encoding program, moving image decoding device, moving image decoding method and moving image decoding program - Google Patents

Abstract

FIELD: data processing.

SUBSTANCE: group of inventions relates to video data processing technologies. Disclosed is a moving frame coding device. Device comprises a module for extracting potentially suitable versions of spatial information of movement. Device also includes a module for extracting potentially suitable versions of motion information based on history. Module for extracting potentially suitable versions of spatial information of movement is made with the possibility of extracting a potentially suitable version of the spatial motion information from the motion information of a block adjacent to the target decoding block in the spatial domain. Module for extracting potentially suitable versions of motion information based on the history preferably extracts old motion information without comparing the motion information with a potentially suitable version of spatial information of motion.

EFFECT: reduction of data processing load on video data encoding/decoding devices.

4 cl, 55 dwg

Description

The field of technology to which the invention belongs

[0001] The present invention relates to a frame encoding and decoding technology for dividing a frame into blocks and performing prediction.

State of the art

[0002] When encoding and decoding frames, the target frame is divided into blocks, each of which is a set of a prescribed number of samples, and the process is performed in units of blocks. Encoding efficiency is improved by dividing the frame into appropriate blocks and properly specifying intra-prediction (intra-prediction) and inter-prediction (inter-prediction).

[0003] When encoding/decoding moving frames, encoding efficiency is improved by inter-prediction to perform prediction from an encoded/decoded frame. Patent Document 1 describes a technology for applying an affine transformation during inter-prediction. Quite often, an object causes deformation such as enlargement/reduction and rotation in moving frames, and efficient encoding is achieved by applying the technology of Patent Document 1.

List of bibliographic references

Patent Documents

[0004] Patent Document 1

Unexamined Patent Application (Japan), First Publication No. H9-172644.

The essence of the invention

Technical task

[0005] However, since the technology of Patent Document 1 includes frame conversion, there is a problem that the processing load is large. In view of the above problem, the present invention provides an efficient low load coding technique.

The solution of the problem

[0006] According to an aspect of the present invention, in order to solve the above-described problems, a moving frame coding apparatus is provided, including: a motion spatial information candidate extractor, configured to extract a motion spatial information candidate from the motion information of a block adjacent to the target block. coding, in the spatial domain; and a history-based motion information candidate extraction module, configured to extract the history-based motion information candidates from a memory storing motion information of the encoded block, wherein the history-based motion information candidate extraction module preferably extracts the old motion information without comparing the motion information with a potentially suitable version of the spatial motion information.

[0007] In addition, according to another aspect of the present invention, there is provided a method for encoding moving frames, including: a first process of extracting a potentially suitable case of spatial motion information from motion information of a block adjacent to an encoding target block in a spatial domain; and a second process of retrieving a potentially relevant candidate motion information based on the history from a memory storing motion information of the encoded block, the second process including preferentially retrieving old motion information without comparing the motion information with a potential candidate spatial motion information.

[0008] In addition, according to yet another aspect of the present invention, a program for encoding moving frames is provided for causing a computer to function as: a motion spatial information potential candidate extractor, configured to extract motion spatial information potential candidates from block motion information. , adjacent to the target coding block, in the spatial domain; and a history-based motion information candidate extraction module, configured to extract the history-based motion information candidates from a memory storing motion information of the encoded block, wherein the history-based motion information candidate extraction module preferably extracts the old motion information without comparing the motion information with a potentially suitable version of the spatial motion information.

[0009] In addition, according to still another aspect of the present invention, a frame decoding apparatus is provided, including: a potential candidate spatial motion information extraction module, configured to extract a candidate spatial motion information candidate from the motion information of a block adjacent to the target a decoding unit, in the spatial domain; and a history-based motion information candidate extraction module, configured to extract the history-based motion information candidates from a memory storing the motion information of the decoded block, the history-based motion information candidate extraction module preferably extracting the old motion information without comparing the motion information with a potentially suitable version of the spatial motion information.

[0010] In addition, according to yet another aspect of the present invention, there is provided a method for decoding frames, including: a first process of extracting a potentially suitable case of spatial motion information from motion information of a block adjacent to a decoding target block in a spatial domain; and a second process of retrieving a potentially candidate motion information based on the history from a memory storing the motion information of the decoded block, the second process including preferentially retrieving the old motion information without comparing the motion information with the potential candidate spatial motion information.

[0011] In addition, according to yet another aspect of the present invention, a frame decoding program is provided for causing a computer to function as: a motion spatial information candidate extractor, configured to extract motion spatial information candidates from the block motion information, adjacent to the decoding target block, in the spatial domain; and a history-based motion information candidate extraction module, configured to extract the history-based motion information candidates from a memory storing motion information of the decoded block, the history-based motion information candidate extraction module preferably extracting the old motion information without comparing the motion information with a potentially suitable version of the spatial motion information.

Benefits of the Invention

[0012] According to the present invention, it is possible to realize a highly efficient frame encoding/decoding process with low load.

Brief description of the drawings

[0013] FIG. 1 is a block diagram of a frame encoding apparatus according to an embodiment of the present invention.

Fig. 2 is a block diagram of a frame decoding apparatus according to an embodiment of the present invention.

Fig. 3 is an explanatory flowchart showing a tree block splitting operation.

Fig. 4 is a diagram showing a state in which an input frame is divided into tree blocks.

Fig. 5 is an explanatory diagram showing a Z-scan.

Fig. 6A is a diagram showing a divided block shape.

Fig. 6B is a diagram showing a divided block shape.

Fig. 6C is a diagram showing a divided block shape.

Fig. 6D is a diagram showing a divided block shape.

Fig. 6E is a diagram showing a divided block shape.

Fig. 7 is an explanatory flowchart showing the operation of dividing a block into four parts.

Fig. 8 is an explanatory flowchart showing the operation of dividing a block into two or three parts.

Fig. 9 is the syntax for expressing the block split shape.

Fig. 10A is an explanatory diagram showing intra prediction.

Fig. 10B is an explanatory diagram showing intra prediction.

Fig. 11 is an explanatory diagram showing an inter prediction reference block.

Fig. 12 is a syntax for expressing a coding block prediction mode.

Fig. 13 is a diagram showing a correspondence between a syntax element related to inter prediction and a mode.

Fig. 14 is an explanatory diagram showing affine motion compensation of two control points.

Fig. 15 is an explanatory diagram showing motion affine compensation of three control points.

Fig. 16 is a block diagram of a detailed configuration of the inter prediction module 102 of FIG. one.

Fig. 17 is a block diagram of a detailed configuration of the normal motion vector predictor identification mode extraction unit 301 of FIG. sixteen.

Fig. 18 is a block diagram of a detailed configuration of the normal combining mode extractor 302 of FIG. sixteen.

Fig. 19 is an explanatory flowchart showing a normal motion vector predictor identification mode extraction process of the normal motion vector predictor identification mode extraction unit 301 of FIG. sixteen.

Fig. 20 is a flowchart showing a normal motion vector predictor identification mode extraction process processing procedure.

Fig. 21 is an explanatory flowchart showing a normal combining mode extraction process processing procedure.

Fig. 22 is a block diagram of a detailed configuration of the inter prediction module 203 of FIG. 2.

Fig. 23 is a block diagram of a detailed configuration of the normal motion vector predictor identification mode extraction unit 401 of FIG. 22.

Fig. 24 is a block diagram of a detailed configuration of the normal combining mode extractor 402 of FIG. 22.

Fig. 25 is an explanatory flowchart showing a normal motion vector predictor identification mode extraction process of the normal motion vector predictor identification mode extraction unit 401 of FIG. 22.

Fig. 26 is an explanatory diagram showing a processing procedure for initializing/updating a list of potentially candidate candidate motion vector predictors based on history.

Fig. 27 is a flowchart for the identity check processing procedure in the processing procedure for initializing/updating a list of potentially matching motion vector predictor cases based on history.

Fig. 28 is a flowchart for a pixel shift processing procedure in a processing procedure for initializing/updating a list of potentially suitable motion vector predictor cases based on history.

Fig. 29 is an explanatory flowchart showing a history-based motion vector predictor potential candidate extraction processing procedure.

Fig. 30 is an explanatory flowchart showing a processing procedure for extracting potentially suitable combinations based on history.

Fig. 31A is an explanatory diagram showing an example of a process for updating lists of potentially candidate motion vector predictors based on history.

Fig. 31B is an explanatory diagram showing an example of a process for updating lists of potentially candidate motion vector predictors based on history.

Fig. 31C is an explanatory diagram showing an example of a process for updating lists of potentially candidate motion vector predictors based on history.

Fig. 32 is an explanatory diagram showing motion compensation prediction when the cycle time of the reference frame (RefL0Pic) L0 is earlier than the cycle time of the target frame (CurPic) as the L0 prediction.

Fig. 33 is an explanatory diagram showing motion compensation prediction when the cycle time of the L0 prediction reference frame is later than the cycle time of the target frame as L0 prediction.

Fig. 34 is an explanatory diagram showing a prediction direction for motion-compensated prediction when the clock time of the L0 prediction reference frame is earlier than the clock time of the target frame, and the clock time of the L1 prediction reference frame is later than the clock time of the target frame, as bi-prediction.

Fig. 35 is an explanatory diagram showing a prediction direction for motion-compensated prediction when the cycle time of the L0 prediction reference frame and the cycle time of the L1 prediction reference frame are earlier than the cycle time of the target frame, as bi-prediction.

Fig. 36 is an explanatory diagram showing a prediction direction for motion-compensated prediction when the cycle time of the L0 prediction reference frame and the cycle time of the L1 prediction reference frame are later than the cycle time of the target frame, as bi-prediction.

Fig. 37 is an explanatory diagram showing a hardware configuration example of an encoding/decoding device according to an embodiment of the present invention.

Fig. 38 is a flowchart for describing a history-based motion vector predictor candidate extraction processing procedure in the case of a configuration in which the identical candidate candidate deletion process is not performed, and a history-based motion vector predictor potential candidate list is used with by referring to the motion information in order from the oldest motion information to the newly added motion information.

Fig. 39 is a flowchart for a procedure for processing a shift and adding a list entry of potentially suitable history motion vector predictor options.

Fig. 40 is a flowchart for describing a history-based merge candidate extraction processing procedure in the case of a configuration in which the process of deleting identical candidates is not performed, and a history-based motion vector predictor candidate list is used with reference to motion information in order from the oldest motion information to the newly added motion information.

Fig. 41 is an explanatory diagram showing the configuration of a history-based motion vector predictor candidate list configuration.

Fig. 42 is an explanatory diagram showing a state in which an element at the head of the history-based motion vector predictor candidate candidate list is removed at the time of adding to the history-based motion vector predictor candidate candidate list.

Fig. 43 is an explanatory diagram showing a state in which each element is shifted in the history-based motion vector predictor candidate list at the time of adding to the history-based motion vector predictor candidate list.

Fig. 44 is an explanatory diagram showing a state in which a new entry is added at the time of adding to the history-based motion vector predictor candidate list.

Fig. 45 is a diagram for describing a reference order of a list of potentially candidate candidate motion vector predictors based on history.

Fig. 46 is a diagram for describing a reference order of a list of potentially candidate motion vector predictors based on history.

Fig. 47 is a flowchart for describing a history-based motion vector predictor potential candidate extraction processing procedure in the case of a configuration in which the identical candidate candidate deletion process is not performed, and a history-based motion vector predictor potential candidate list is used with reference in reverse order, which is the reverse of the storage order.

Fig. 48 is a flowchart for describing a history-based motion vector predictor candidate extraction processing procedure in the case of a configuration in which the process of deleting identical candidates is performed using a history-based motion vector predictor potential candidate list with reference to reverse order, which is the reverse of the storage order.

Detailed description of embodiments

[0014] Hereinafter, the technological and technical terms used in the embodiment are specified.

[0015] Tree block

In an embodiment, the encoding/decoding target frame is equally divided into units of a predetermined size. This unit is specified as a tree block. Although the treeblock size is 128×128 samples in FIG. 4, the size of the tree block is not limited to this, and any size can be set. The tree block of the target (corresponding to the encoding target in the encoding process or the decoding target in the decoding process) is switched in the raster scan order, i.e. left to right and top to bottom. The interior of each treeblock may further be recursively split. The block that is the encoding/decoding target after the tree block is recursively divided is set as the encoding block. In addition, the tree block and the coding block are jointly defined as blocks. Efficient coding is ensured by performing appropriate block splitting. The treeblock size may be a fixed value predetermined by the encoder and the decoder, or the treeblock size determined by the encoder may be configured to be transmitted to the decoder. Here, the maximum treeblock size is 128×128 samples, and the minimum treeblock size is 16×16 samples. In addition, the maximum coding block size is 64×64 samples, and the minimum coding block size is 4×4 samples.

[0016] Prediction mode

Switching is performed between intra-prediction (MODE_INTRA), in which prediction is performed from the processed frame signal for the target frame, and inter-prediction (MODE_INTER), in which prediction is performed from the frame signal for the processed frame in units of target coding units.

The processed frame is used for a frame, a frame signal, a treeblock, a block, an encoding block, or the like obtained by decoding a signal completely encoded in an encoding process, and is used for a frame, a frame signal, a treeblock, a block, an encoding block, etc. .p. obtained by completing decoding in the decoding process.

The mode in which intra-prediction (MODE_INTRA) and inter-prediction (MODE_INTER) are identified is set as a prediction mode (PredMode). The prediction mode (PredMode) has intra-prediction (MODE_INTRA) or inter-prediction (MODE_INTER) as a value.

[0017] Inter prediction

In inter-prediction, in which prediction is performed from a frame signal for a processed frame, a plurality of processed frames may be used as reference frames. In order to manage a plurality of reference frames, two types of reference lists in the form of L0 (reference list 0) and L1 (reference list 1) are defined, and the reference frame is identified using each reference index. In P-slice, L0 prediction (Pred_L0) can be used. In a B-slice, L0 prediction (Pred_L0), L1 prediction (Pred_L1), and biprediction (Pred_BI) can be used. L0 prediction (Pred_L0) is an inter prediction that refers to a reference frame driven in L0, and L1 prediction (Pred_L1) is an inter prediction that refers to a reference frame driven in L1. Bi-prediction (Pred_BI) is inter-prediction in which both L0 prediction and L1 prediction are performed, and refer to one reference frame driven in each of L0 and L1. Information for identifying L0 prediction, L1 prediction, and bi-prediction is set as an inter-prediction mode. In post-processing, constants and variables with subscript LX in the output are supposed to be processed for each of L0 and L1.

[0018] Motion Vector Predictor Identification Mode

The motion vector predictor identification mode is a mode for transmitting an index for identifying the motion vector predictor, the motion vector difference, the inter prediction mode, and the reference index, and determining target block inter prediction information. The motion vector predictor is derived from a potentially suitable motion vector predictor derived from a processed block adjacent to the target block, or a block located at or near (near) the target block from among the blocks belonging to the processed frame, and an index to identify the predictor motion vector.

[0019] Combine Mode

The merge mode is a mode in which the inter-prediction information of the target block is extracted from the inter-prediction information of the processed block adjacent to the target block, or a block located at the identical position or near (near) the target block from among the blocks belonging to the processed frame. , without transferring the difference of the motion vectors and the reference index.

[0020] The processed block adjacent to the target block and the inter-prediction information of the processed block are defined as spatial potential join candidates. A block located at or near the same position as a target block among blocks belonging to the processed frame and inter-prediction information extracted from the block's inter-prediction information are set as temporal potentially suitable merges. Each join candidate is registered in a join candidate list, and the join candidate used to predict the target block is identified by the join index.

[0021] Neighbor block

Fig. 11 is an explanatory diagram showing a reference block referred to in extracting inter prediction information in the motion vector predictor identification mode and the combining mode. A0, A1, A2, B0, B1, B2 and B3 are processed blocks adjacent to the target block. T0 is a block located at or near the same position as the target block in the target frame from among the blocks belonging to the processed frame.

[0022] A1 and A2 are blocks located on the left side of the coding target block and adjacent to the coding target block. B1 and B3 are blocks located on the upper side of the coding target block and adjacent to the coding target block. A0, B0, and B2 are bottom left, top right, and top left blocks of the target coding block, respectively.

[0023] The following describes the details of how to process adjacent blocks in the motion vector predictor identification mode and the merging mode.

[0024] Affine Motion Compensation

Affine motion compensation is a process of performing motion compensation by dividing a coding block into subblocks of a predetermined unit and separately determining a motion vector for each of the subblocks into which the coding block is divided. The motion vector of each sub-block is extracted based on one or more control points extracted from the inter-prediction information of the processed block adjacent to the target block, or a block located in an identical position or near (next to) the target block from among the blocks belonging to the processed frame. Although the sub-block size is 4×4 samples in the present embodiment, the sub-block size is not limited to this, and the motion vector may be extracted in units of samples.

[0025] An example of affine motion compensation in the case of two control points is shown in FIG. 14. In this case, the two control points have two parameters as a horizontal direction component and a vertical direction component. Thus, an affine transformation in the case of two control points is called a "four-parameter affine transformation". CP1 and CP2 in FIG. 14 are control points.

An example of affine motion compensation in the case of three control points is shown in FIG. 15. In this case, the three control points have two parameters as a horizontal direction component and a vertical direction component. Thus, the affine transformation in the case of three control points is called a "six-parameter affine transformation". CP1, CP2 and CP3 of FIG. 15 are control points.

[0026] Affine motion compensation can be used in both the motion vector predictor identification mode and the pooling mode. The mode in which affine motion compensation is applied in the motion vector predictor identification mode is set as the sub-block-based motion vector predictor identification mode, and the mode in which affine motion compensation is applied in the combining mode is set as the sub-block-based combining mode.

[0027] Inter prediction syntax

The following describes the syntax associated with inter-prediction using FIG. 12 and 13.

The merge_flag in FIG. 12 indicates whether the target coding block is set to the combining mode or the motion vector predictor identification mode. The merge_affine_flag flag indicates whether or not the subblock-based merging mode is applied to the target coding block of the merging mode. The inter_affine_flag indicates whether or not to apply the sub-block motion vector predictor identification mode to the target coding block of the motion vector predictor identification mode. The cu_affine_type_flag flag is used to determine the number of control points in the subblock-based motion vector predictor identification mode.

Fig. 13 shows the meaning of each syntax element and the prediction method corresponding to it. The normal merging mode corresponds to merge_flag=1 and merge_affine_flag=0 and is not a sub-block based merging mode. The subblock-based merging mode corresponds to merge_flag=1 and merge_affine_flag=1. The normal motion vector predictor identification mode corresponds to merge_flag=0 and inter_affine_flag=0. The normal motion vector predictor identification mode is a motion vector predictor combining mode, which is not a subblock-based motion vector predictor identification mode. The subblock-based motion vector predictor identification mode corresponds to merge_flag=0 and inter_affine_flag=1. When merge_flag=0 and inter_affine_flag=1, cu_affine_type_flag is additionally passed to determine the number of control points.

[0028] POC

A frame sequence number (POC) is a variable associated with a frame to be encoded and is set to a value that increments by 1 according to the frame output order. According to the POC value, it is possible to distinguish whether or not the frames are identical, to distinguish the anteroposterior relationship between frames in the output order, or to extract the distance between frames. For example, if the POCs of two frames have the same value, they may be determined to be the same frame. When the POCs of two frames have different values, it may be determined that the frame with the smaller POC value is the frame to be output first. The difference between the POC of two frames indicates the inter-frame distance in the direction of the time axis.

[0029] First Embodiment

In the following, the frame encoding apparatus 100 and the frame decoding apparatus 200 according to the first embodiment of the present invention will be described.

[0030] FIG. 1 is a block diagram of a frame encoding apparatus 100 according to the first embodiment. The frame encoding apparatus 100 according to the embodiment includes a block splitting unit 101, an inter-prediction unit 102, an intra-prediction unit 103, a decoded frame memory 104, a prediction method determining unit 105, a residual unit 106, an orthogonal transformation/quantization unit 107 , a bit string encoding unit 108, an inverse quantization/inverse orthogonal transform unit 109, a frame signal overlay unit 110 for decoding, and a storage device 111 for storing encoding information.

[0031] The block splitter 101 recursively splits the input frame to form a coding block. The block splitter 101 includes a quadratic splitter that splits a split target block in a horizontal direction and a vertical direction, and a binary/ternary splitter that splits a split target block in either the horizontal direction or the vertical direction. The block splitter 101 sets the generated coding block as the target coding block, and provides the frame signal of the target coding block to the inter-prediction module 102, the intra-prediction module 103, and the residual generation module 106 . In addition, the block splitter 101 provides information indicating the determined recursive split structure to the bit string encoding unit 108 . The detailed operation of the block splitter 101 will be described below.

[0032] The inter-prediction unit 102 performs inter-prediction of the target coding block. The inter prediction unit 102 extracts a plurality of potentially suitable inter prediction information from the inter prediction information stored in the encoding information storage device 111 and the decoded frame signal stored in the decoded frame memory 104, selects an appropriate inter prediction mode from the plurality of retrieved potential candidates, and provides the selected inter-prediction mode and the predicted frame signal according to the selected inter-prediction mode to the prediction method determining unit 105 . The detailed configuration and operation of the inter-prediction module 102 is described below.

[0033] The intra prediction unit 103 performs intra prediction of the target coding block. The intra prediction unit 103 refers to the decoded frame signal stored in the decoded frame memory 104 as a reference sample, and generates a predicted frame signal according to the intra prediction based on encoding information such as an intra prediction mode stored in the memory 111 for storing coding information. In intra prediction, the intra prediction unit 103 selects a suitable intra prediction mode from a plurality of intra prediction modes, and provides the selected intra prediction mode and the predicted frame signal according to the selected intra prediction mode to the prediction method determination unit 105 .

Examples of intra-prediction are shown in FIG. 10A and 10B. Fig. 10A shows the correspondence between the intra-prediction prediction direction and the intra-prediction mode number. For example, in intra-prediction mode 50, an intra-prediction frame is generated by copying reference samples in the vertical direction. Intra-prediction mode 1 is a DC mode, and is a mode in which all sample values of the target block average the value of the reference samples. Intra-prediction mode 0 is a planar mode, and is a mode for creating a two-dimensional intra-prediction frame from reference samples in the vertical and horizontal directions. Fig. 10B is an example in which an intra prediction frame is generated in the case of the intra prediction mode 40 . The intra prediction unit 103 copies the value of the reference sample in the direction indicated by the intra prediction mode with respect to each target block sample. When the intra prediction mode reference sample is not at an integer position, the intra prediction module 103 determines the reference sample value according to interpolation from the reference sample values of neighboring integer positions.

[0034] The decoded frame memory 104 stores the decoded frame generated by the frame signal superposition unit 110 for decoding. The decoded frame memory 104 provides the stored decoded frame to the inter prediction module 102 and the intra prediction module 103 .

[0035] The prediction method determining unit 105 determines the optimal prediction mode by evaluating each intra-prediction and inter-prediction using coding information, residual code amount, amount of distortion between the predicted frame signal and the target frame signal, and the like. In the case of intra prediction, the prediction method determination unit 105 provides intra prediction information such as the intra prediction mode as encoding information to the bit string encoding unit 108 . In the case of the inter-predicted combining mode, the prediction method determination unit 105 provides inter-prediction information such as the combining index and information indicating whether or not the mode is a sub-block combining mode (sub-block combining flag) as information encoding module 108 encoding bit strings. In the case of the inter prediction motion vector predictor identification mode, the prediction method determination unit 105 provides inter prediction information such as the inter prediction mode, motion vector predictor index, reference indices L0 and L1, motion vector difference, and information indicating whether a sub-frame motion vector predictor identification mode (sub-frame motion vector predictor flag) as itself or not, as encoding information to the bit string encoding unit 108 . Further, the prediction method determination unit 105 provides certain encoding information to the storage device 111 for storing encoding information. The prediction method determination unit 105 provides the predicted frame signal to the residual generation unit 106 and the frame signal overlay unit 110 for decoding.

[0036] The residual generator 106 generates a residual by subtracting the predicted frame signal from the target frame signal, and provides the residual to the orthogonal transform/quantization module 107.

[0037] The orthogonal transform/quantization unit 107 performs orthogonal transformation and quantization on the residual according to the quantization parameter to generate an orthogonally transformed/quantized residual, and provides the generated residual to the bit string encoding unit 108 and the inverse quantization/inverse orthogonal transform unit 109.

[0038] The bit string encoding unit 108 encodes encoding information according to the prediction method determined by the prediction method determination unit 105 for each coding block, in addition to information of units of sequences, frames, slices, and coding blocks. In particular, the bitstring encoding unit 108 encodes the prediction mode PredMode for each coding block. When the prediction mode is inter prediction (MODE_INTER), the bit string encoding unit 108 encodes encoding information (inter prediction information) such as a flag for distinguishing whether or not the mode is a combining mode, a combining flag based on subblocks, a combining index when the mode is a pooling mode, the inter-prediction mode when the mode is not a pooling mode, the motion vector predictor index, the motion vector difference information, and the motion vector predictor flag based on sub-blocks according to the specified syntax (syntax rule for bit strings ) and forms the first bit strings. When the prediction mode is intra prediction (MODE_INTRA), encoding information (intra prediction information) such as intra prediction mode is encoded according to a specified syntax (syntax rule for bit strings), and first bit strings are generated. In addition, the bit string encoding unit 108 entropy encodes the orthogonally transformed and quantized residual according to the specified syntax to generate second bit strings. The bit string encoding unit 108 multiplexes the first bit strings and the second bit strings according to the specified syntax, and outputs a bitstream.

[0039] The inverse quantization/inverse orthogonal transform unit 109 calculates the residual by performing inverse quantization and inverse orthogonal transform on the orthogonally transformed/quantized residual provided from the orthogonal transform/quantization unit 107, and provides the calculated residual to the frame signal overlay unit 110 for decoding.

[0040] The frame signal superposition unit 110 for decoding superimposes the predicted frame signal according to the definition of the prediction method determination unit 105 and the residual inversely quantized and inversely orthogonally transformed by the inverse quantization/inverse orthogonal transform unit 109 to generate a decoded frame, and stores the decoded frame in storage device 104 decoded frames. In addition, the frame signal superposition module 110 for decoding may store the decoded frame in the decoded frame memory 104 after performing a filtering process to reduce distortion such as blocking due to encoding for the decoded frame.

[0041] The encoding information storage 111 stores encoding information such as a prediction mode (inter-prediction or intra-prediction) determined by the prediction method determination unit 105 . In the case of inter prediction, the encoding information stored in the encoding information storage device 111 includes inter prediction information such as a determined motion vector, reference indexes of reference lists L0 and L1, and a list of potentially suitable motion vector predictor candidates based on background. In addition, in the case of the inter-predicted combining mode, the encoding information stored in the encoding information storage device 111 includes inter-prediction information such as the combining index and information indicating whether or not the mode is a combining mode on sub-unit basis (sub-unit-based merge flag), in addition to the above information. In addition, in the case of the inter prediction motion vector predictor identification mode, encoding information stored in the encoding information storage device 111 includes inter prediction information such as inter prediction mode, motion vector predictor index, motion vector difference and information indicating whether or not the mode is a sub-frame motion vector predictor identification mode (sub-frame motion vector predictor flag), in addition to the above-described information. In the case of intra prediction, the encoding information stored in the encoding information storage device 111 includes intra prediction information such as a specific intra prediction mode.

[0042] FIG. 2 is a block diagram showing the configuration of a frame decoding apparatus according to an embodiment of the present invention corresponding to the frame encoding apparatus of FIG. 1. The frame decoding apparatus according to the embodiment includes a bit string decoding unit 201, a block splitting unit 202, an inter prediction unit 203, an intra prediction unit 204, an encoding information storage unit 205, an inverse quantization/inverse orthogonal transformation unit 206 , a frame overlay module 207 for decoding, and a decoded frame memory 208 .

[0043] Since the decoding process of the frame decoder of FIG. 2 corresponds to the decoding process provided in the frame encoder of FIG. 1, components of the memory 205 for storing encoding information, the inverse quantization/inverse orthogonal transform unit 206, the frame signal superposition unit 207 for decoding, and the decoded frame memory 208 of FIG. 2 have functions corresponding to the components of the memory 111 for storing encoding information, the inverse quantization/inverse orthogonal transform unit 109, the frame signal overlay unit 110 for decoding, and the decoded frame memory 104 of the frame encoding apparatus of FIG. one.

[0044] The bit stream provided to the bit string decoding module 201 is divided according to the specified syntax rule. The bit string decoding unit 201 decodes the separated first bit string and obtains information of sequence units, frames, slices, coding units, and coding information of coding units. Specifically, the bit string decoding unit 201 decodes the prediction mode PredMode to distinguish between inter-prediction (MODE_INTER) or intra-prediction (MODE_INTRA) in units of coding blocks. When the prediction mode is inter prediction (MODE_INTER), the bit string decoding unit 201 decodes coding information (inter prediction information) regarding a flag to distinguish whether or not the mode is a combining mode, a combining index when the mode is a combining mode, a sub-block combining flag, an inter-prediction mode when the mode is a motion vector predictor identification mode, a motion vector predictor index, a motion vector difference, a sub-block motion vector predictor flag, and the like. according to the specified syntax, and provides encoding information (inter prediction information) to the encoding information storage device 205 via the inter prediction unit 203 and the block splitter 202 . When the prediction mode is intra prediction (MODE_INTRA), encoding information (intra prediction information) such as the intra prediction mode is decoded according to the specified syntax, and encoding information (intra prediction information) is provided to the memory 205 for storage. encoding information through the inter prediction module 203 or the intra prediction module 204 and the block splitter 202 . The bit string decoding module 201 decodes the separated second bit strings to calculate an orthogonally transformed/quantized residual, and provides the orthogonally transformed/quantized residual to the inverse quantization/inverse orthogonal transform module 206 .

[0045] When the prediction mode PredMode of the target coding block is the inter-prediction motion vector predictor identification mode (MODE_INTER), the inter-prediction module 203 extracts a plurality of potentially suitable motion vector predictor cases using the encoding information of the previously decoded frame signal stored in the memory the coding information storage device 205, and registers the plurality of extracted motion vector predictor candidates in a motion vector predictor candidate list, which is described below. The inter-prediction unit 203 selects a motion vector predictor according to the motion vector predictor index decoded and provided by the bit string decoding unit 201 from a plurality of motion vector predictor candidate candidates registered in the motion vector predictor candidate candidate list, calculates a motion vector from the vector difference motion decoded by the bit string decoding unit 201 and the selected motion vector predictor, and stores the calculated motion vector in the memory 205 for storing encoding information along with other encoding information. The encoding information of the coding unit provided/stored here is the prediction mode PredMode, flags predFlagL0[xP][yP] and predFlagL1[xP][yP] indicating whether L0 prediction and L1 prediction should be used, reference indices refIdxL0[ xP][yP] and refIdxL1[xP][yP] L0 and L1, vectors mvL0[xP][yP] and mvL1[xP][yP] of movement L0 and L1, etc. Here, xP and yP are indices indicating the position of the top left coding block sample in the frame. When the prediction mode PredMode is inter prediction (MODE_INTER) and the inter prediction mode is L0 prediction (Pred_L0), the predFlagL0 flag indicating whether or not to use L0 prediction is 1, and the predFlagL1 flag indicating whether whether or not to use L1 prediction is 0. When the inter-prediction mode is L1 prediction (Pred_L1), the predFlagL0 flag indicating whether or not to use L0 prediction is 0, and the predFlagL1 flag indicating whether whether or not to use L1 prediction is 1. When the inter-prediction mode is bi-prediction (Pred_BI), and predFlagL0 indicating whether or not to use L0 prediction, and predFlagL1 indicating whether or not to use L1 - prediction are equal to 1. Additionally, potentially suitable combining options are extracted in a combining mode in which the PredMode of the target coding block prediction is It is an inter-prediction (MODE_INTER). A plurality of candidate joins are retrieved using the encoding information of previously decoded coding blocks stored in the encoding information storage device 205, and registered in a list of candidate joins, which is described below, the join candidate corresponding to the join index to be decoded. and provided by the bit string decoding unit 201 is selected from a plurality of candidate joins registered in the list of candidate joins, and inter-prediction information such as flags predFlagL0[xP][yP] and predFlagL1[xP][yP], indicating whether or not to use L0-prediction and L1-prediction of the selected potentially suitable join option, reference indices refIdxL0[xP][yP] and refIdxL1[xP][yP] L0 and L1, and vectors mvL0[xP][yP] and mvL1[xP][yP] movements L0 and L1, with stored in the storage device 205 for storing encoding information. Here, xP and yP are indices indicating the position of the top left coding block sample in the frame. The detailed configuration and operation of the inter-prediction module 203 is described below.

[0046] The intra prediction module 204 performs intra prediction when the prediction mode PredMode of the target coding block is intra prediction (MODE_INTRA). The encoding information decoded by the bit string decoding unit 201 includes an intra prediction mode. The intra prediction unit 204 generates a predicted frame signal according to the intra prediction from the decoded frame signal stored in the decoded frame memory 208 according to the intra prediction mode included in the encoding information decoded by the bit string decoding unit 201, and provides the generated the predicted frame signal to the frame signal overlay unit 207 for decoding. Since the intra prediction unit 204 corresponds to the intra prediction unit 103 of the frame encoding apparatus 100, a process similar to that of the intra prediction unit 103 is performed.

[0047] The inverse quantization/inverse orthogonal transform unit 206 performs inverse orthogonal transform and inverse quantization on the orthogonally transformed/quantized residual decoded by the bit string decoding unit 201, and obtains the orthogonally transformed/inversely quantized residual.

[0048] The decoding frame signal overlay unit 207 decodes the frame signal for decoding by overlaying the predicted frame signal inter-predicted by the inter-prediction unit 203 or the predicted frame signal intra-predicted by the intra prediction unit 204 and the remainder, back orthogonally transformed/inversely quantized by the inverse quantization/inverse orthogonal transform unit 206, and stores the decoded frame signal for decoding in the decoded frame memory 208 . While storing the decoded frames in the memory 208, the decoding frame signal superposition unit 207 may store the decoded frame in the decoded frame memory 208 after the filtering process to reduce blocking distortion or the like is applied. due to encoding is performed for the decoded frame.

[0049] Next, the operation of the block splitter 101 in the frame encoding device 100 will be described. Fig. 3 is a flowchart showing the operation of dividing a frame into treeblocks and further dividing each treeblock. First, the input frame is divided into tree blocks having a predetermined size (step S1001). Each treeblock is scanned in a predetermined order, i.e. in raster scan order (step S1002), and the inside of the target tree block is divided (step S1003).

[0050] FIG. 7 is a flowchart showing the detailed operation of the partitioning process of step S1003. First, it is determined whether or not the target block is to be divided into four parts (step S1101).

[0051] When it is determined that the target block is to be divided into four parts, the target block is divided into four parts (step S1102). Each block obtained by splitting the target block is scanned in Z-scan order, i.e. in the order of top left, top right, bottom left, and bottom right (step S1103). Fig. 5 shows an example of a Z-scan order, and reference number 601 in FIG. 6A shows an example in which the target block is divided into four parts. Numbers 0-3 of reference number 601 in FIG. 6A indicate the processing order. Then, the partitioning process of FIG. 7 is recursively executed for each block from the division in step S1101 (step S1104).

[0052] When it is determined that the target block is not to be quadrupled, binary/ternary splitting is performed (step S1105).

[0053] FIG. 8 is a flowchart showing the detailed operation of the binary/ternary partitioning process of step S1105. First, it is determined whether or not the target block should be divided into two or three parts, i.e. then whether or not binary or ternary partitioning is to be performed (step S1201).

[0054] When it is not determined that the target block should be divided into two or three parts, i.e. when it is determined that the target block is not to be split, splitting is completed (step S1211). Thus, the recursive partitioning process is further not performed for the blocks divided according to the recursive partitioning process.

[0055] When it is determined that the target block is to be divided into two or three parts, it is further determined whether or not the target block is to be divided into two parts (step S1202).

[0056] When it is determined that the target block is to be divided into two parts, it is determined whether or not the target block is to be divided into upper and lower parts (in the vertical direction) (step S1203). Based on the determination result, the target block is divided into two parts that are upper and lower parts (in the vertical direction) (step S1204), or the target block is divided into two parts that are left and right parts (in the horizontal direction) ( step S1205). As a result of step S1204, the target block is divided into two parts, which are upper and lower parts (in the vertical direction) as indicated by reference 602 in FIG. 6b. As a result of step S1205, the target block is divided into two parts, which are left and right parts (in the horizontal direction), as indicated by reference number 604 in FIG. 6d.

[0057] When it is not determined that the target block should be divided into two parts, i.e. when it is determined that the target block is to be divided into three parts, in step S1202, it is determined whether or not the target block is to be divided into upper, middle and lower parts (in the vertical direction) (step S1206). Based on the determination result, the target block is divided into three parts that are top, middle and bottom (in the vertical direction) (step S1207), or the target block is divided into three parts that are left, middle and right (in horizontal direction) (step S1208). As a result of step S1207, the target block is divided into three parts, which are upper, middle, and lower parts (in the vertical direction) as indicated by reference 603 in FIG. 6C. As a result of step S1208, the target block is divided into three parts, which are left, middle, and right parts (in the horizontal direction) as indicated by reference 605 in FIG. 6E.

[0058] After any of steps S1204, S1205, S1207, and S1208 are executed, each of the blocks into which the target block is divided is scanned in order from left to right and top to bottom (step S1209). Reference numbers 0-2 602-605 of FIG. 6B to 6E indicate the processing order. For each of the blocks into which the target block is divided, the binary/ternary partitioning process of FIG. 8 is recursively executed (step S1210).

[0059] The recursive block splitting described here may limit the need for splitting according to the number of splits or target block size, or the like. The information that limits the need for splitting may be realized by a configuration in which information is not delivered by performing negotiation between the encoding device and the decoding device in advance, or implemented by a configuration in which the encoding device determines information that limits the need for splitting, writes information to a bit string. and delivers the information to the decoding device.

[0060] When a certain block is split, the block before splitting is called a "parent block", and each block after splitting is called a "child block".

[0061] The following describes the operation of the block splitter 202 in the frame decoder 200 . The block splitter 202 divides the tree block according to a processing procedure similar to that of the block splitter 101 of the frame encoding apparatus 100 . However, there is a difference in that the block splitter 101 of the frame encoding apparatus 100 applies an optimization technique such as best shape estimation based on frame recognition or distortion index optimization to determine the optimal shape of the block split, while the block splitter 202 of the frame The frame decoding 200 determines the block split shape by the block decoding split information recorded in the bit string.

[0062] The syntax (syntax rule for bit strings) associated with block splitting according to the first embodiment is shown in FIG. nine; coding_quadtree() represents the syntax associated with the quad splitting process for a block; multi_type_tree() represents the syntax associated with the binary or ternary splitting process for a block; qt_split is a flag indicating whether or not the block is split into four parts; qt_split=1 when the block is quadrupled, and qt_split=0 when the block is not quadrupled. When a block is split into four parts (qt_split=1), the quadratic splitting process is recursively performed on blocks each divided into four parts (coding_quadtree(0), coding_quadtree(1), coding_quadtree(2), coding_quadtree(3), and arguments 0-3 correspond to the numbers indicated by reference number 601 in Fig. 6A). When a block is not split into four parts (qt_split=0), the subsequent split is determined according to multi_type_tree(); mtt_split is a flag indicating whether additional splitting is performed or not. When splitting is further performed (mtt_split=1), mtt_split_vertical is transmitted, which is a flag indicating whether a block is split vertically or horizontally, and mtt_split_binary, which is a flag for determining whether binary or ternary split is performed; mtt_split_vertical=1 indicates a split in the vertical direction, and mtt_split_vertical=0 indicates a split in the horizontal direction; mtt_split_binary=1 indicates binary split, and mtt_split_binary=0 indicates ternary split. With binary splitting (mtt_split_binary=1), the splitting process is performed recursively for blocks, each of which is split into two parts (multi_type_tree (0), multi_type_tree (1), and arguments 0-1 correspond to the numbers indicated by reference number 602 or 604 in Fig. 6B to 6D). In the case of ternary splitting (mtt_split_binary=0), the splitting process is performed recursively for blocks, each of which is split into three parts (multi_type_tree (0), multi_type_tree (1), multi_type_tree (2), and arguments 0-2 correspond to the numbers specified by reference number 603 in Fig. 6B, or numbers indicated by reference number 605 in Fig. 6E). Until mtt_split=0 is reached, hierarchical block splitting is done by recursively calling multi_type_tree.

[0063] Inter prediction

The inter-prediction method according to the embodiment is performed in the inter-prediction unit 102 of the frame encoder of FIG. 1 and in the inter prediction module 203 of the frame decoder of FIG. 2.

[0064] Hereinafter, the inter-prediction method according to the embodiment will be described with reference to the drawings. The inter-prediction method is carried out in the coding and decoding processes in units of coding blocks.

[0065] Description of the inter-prediction module 102 on the coding side

Fig. 16 is a diagram showing a detailed configuration of the inter prediction unit 102 of the frame encoding apparatus in FIG. 1. The normal motion vector predictor identification mode extraction unit 301 extracts a plurality of potential candidate normal motion vector predictors to select a motion vector predictor, and calculates a motion vector difference between the selected motion vector predictor and the detected motion vector. The detected inter-prediction mode, the reference index and the motion vector, and the calculated motion vector difference become the normal motion vector predictor identification mode inter-prediction information. This inter-prediction information is provided to the inter-prediction mode determination module 305 . The detailed configuration and process of the normal motion vector predictor identification mode extraction unit 301 will be described below.

[0066] The normal combining mode extractor 302 extracts a plurality of potentially suitable normal combining options to select a potentially suitable normal combining option, and obtains normal combining mode inter-prediction information. This inter-prediction information is provided to the inter-prediction mode determination module 305 . The detailed configuration and process of the normal combining mode extractor 302 is described below.

[0067] The sub-block-based motion vector predictor identification mode extraction unit 303 extracts a plurality of potentially suitable sub-block-based motion vector predictors to select a sub-block-based motion vector predictor, and calculates the motion vector difference between the selected sub-block-based motion vector predictor and the detected motion vector. The detected inter-prediction mode, the reference index and the motion vector, and the calculated motion vector difference become inter-prediction information of the sub-block based motion vector predictor identification mode. This inter-prediction information is provided to the inter-prediction mode determination module 305 .

[0068] The subblock-based combining mode extractor 304 extracts a plurality of subblock-based combining candidates to select a subblock-based combining candidate, and obtains inter-prediction information of the subblock combining mode. This inter-prediction information is provided to the inter-prediction mode determination module 305 .

[0069] The inter prediction mode determination unit 305 determines the inter prediction information based on the inter prediction information provided from the normal motion vector predictor identification mode extraction unit 301, the normal combining mode extraction unit 302, the motion vector predictor identification mode extraction unit 303 on on a sub-block basis; and a sub-block-based combining modes extraction module 304 . The inter prediction information according to the determination result is provided from the inter prediction mode determination unit 305 to the motion compensation prediction unit 306 .

[0070] The motion compensation prediction unit 306 performs inter-prediction on the frame reference signal stored in the decoded frame memory 104 based on the determined inter-prediction information. The detailed configuration and process of motion compensation prediction module 306 is described below.

[0071] Description of the decoding inter-prediction module 203 on the decoding side

Fig. 22 is a diagram showing a detailed configuration of the inter prediction unit 203 of the frame decoding apparatus of FIG. 2.

[0072] The normal motion vector predictor identification mode extractor 401 extracts a plurality of potentially suitable normal motion vector predictors to select a motion vector predictor, calculates the sum of the selected motion vector predictor and the decoded motion vector difference, and sets the calculated sum as the motion vector. The decoded inter-prediction mode, the reference index, and the motion vector become the inter-prediction information of the normal motion vector predictor identification mode. This inter-prediction information is provided to the motion compensation prediction module 406 via the switch 408. The detailed configuration and process of the normal motion vector predictor identification mode extraction module 401 will be described below.

[0073] The normal combining mode extractor 402 extracts a plurality of potentially suitable normal combining options to select a potentially suitable normal combining option, and obtains normal combining mode inter-prediction information. This inter-prediction information is provided to the motion-compensated prediction module 406 via the switch 408. The detailed configuration and process of the normal combining mode extractor 402 is described below.

[0074] The sub-block-based motion vector predictor identification mode extractor 403 extracts a plurality of potentially suitable sub-block-based motion vector predictor candidates to select a sub-block-based motion vector predictor, calculates the sum of the selected sub-block-based motion vector predictor and the decoded motion vector difference, and specifies the computed sum as the motion vector. The decoded inter-prediction mode, the reference index, and the motion vector become inter-prediction information of the sub-block based motion vector predictor identification mode. This inter-prediction information is provided to motion compensation prediction module 406 via switch 408.

[0075] The subblock combining mode extractor 404 extracts a plurality of subblock based combining candidates to select a subblock based combining candidate, and obtains subblock combining mode inter-prediction information. This inter-prediction information is provided to motion compensation prediction module 406 via switch 408.

[0076] The motion compensation prediction unit 406 performs inter-prediction on the frame reference signal stored in the decoded frame memory 208 based on the determined inter-prediction information. The detailed configuration and process of motion compensation prediction module 406 are similar to the detailed configuration and process of motion compensation prediction module 306 on the encoding side.

[0077] Normal Motion Vector Predictor Identification Mode Extractor (Normal AMVP)

The Normal Motion Vector Predictor Identification Mode Extractor 301 of FIG. 17 includes a potential candidate spatial motion vector predictor candidate extractor 321, a potential candidate temporal motion vector predictor candidate extractor 322, a history-based potential candidate motion vector predictor candidate extractor 323, a motion vector predictor potential candidate replenishment module 325, a module 326 detecting normal motion vectors, a module 327 for selecting potentially suitable variants of motion vector predictors, and a module 328 for subtracting motion vectors.

[0078] The Normal Motion Vector Predictor Identification Mode Extractor 401 of FIG. 23 includes a potential candidate spatial motion vector predictor candidate extractor 421, a potential candidate temporal motion vector predictor candidate extractor 422, a history-based potential candidate motion vector predictor candidate extractor 423, a motion vector predictor potential candidate replenishment module 425, a module 426 selecting potentially suitable motion vector predictor options and a motion vector summation module 427 .

[0079] In the following, processing procedures of the normal motion vector predictor identification mode extraction unit 301 on the encoding side and the normal motion vector predictor identification mode extraction unit 401 on the decoding side will be described using the flowcharts of FIG. 19 and 25, respectively. Fig. 19 is a flowchart showing a normal motion vector predictor identification mode extraction processing procedure of the normal motion vector predictor identification mode extraction unit 301 on the encoding side, and FIG. 25 is a flowchart showing a normal motion vector predictor identification mode extraction processing procedure of the normal motion vector predictor identification mode extraction unit 401 on the decoding side.

[0080] Normal Motion Vector Predictor Identification Mode Extractor (Normal AMVP): Coding Side Description

In the following, a normal motion vector predictor identification mode extraction processing procedure on the encoding side will be described with reference to FIG. 19. In the description of the processing procedure of FIG. 19, the term "normal" shown in FIG. 19 may be omitted.

[0081] First, the normal motion vector detection module 326 detects the normal motion vector for each inter prediction mode and each reference index (step S100 of FIG. 19).

[0082] Then, in the spatial motion vector predictor candidate candidate extraction module 321, the temporal motion vector predictor candidate candidate extraction module 322, the history-based potential candidate motion vector predictor candidate extraction module 323, the motion vector predictor potential candidate replenishment module 325 , a motion vector predictor candidate selection module 327, and a motion vector subtraction module 328, a motion vector difference for the motion vector used for inter-prediction of the normal motion vector predictor identification mode is calculated for each of L0 and L1 (steps S101 to S106 of FIG. . nineteen). Specifically, when the prediction mode PredMode of the target block is inter prediction (MODE_INTER) and the inter prediction mode is L0 prediction (Pred_L0), a list mvpListL0 of potentially candidate L0 motion vector predictor candidates is computed to select the motion vector predictor mvpL0, and the motion vector difference mvdL0 of the motion vector mvL0 L0 is calculated. When the inter-prediction mode of the target block is L1 prediction (Pred_L1), the L1 motion vector predictor candidate list mvpListL1 is calculated to select the motion vector predictor mvpL1, and the motion vector difference mvdL1 of the L1 motion vector mvL1 is calculated. When the inter-prediction mode of the target block is bi-prediction (Pred_BI), L0 prediction and L1 prediction are performed, the list mvpListL0 of potentially candidate L0 motion vector predictors is computed to select the motion vector predictor mvpL0 of L0, the motion vector difference mvdL0 of motion vector mvL0 of L0 is computed, a list mvpListL1 of potentially candidate L1 motion vector predictors is computed to select the motion vector predictor L1 mvpL1, and the motion vector difference mvdL1 of the motion vector mvL1 L1 is computed.

[0083] Although the motion vector difference calculation process is performed for each of L0 and L1, the motion vector difference calculation process becomes a process common to L0 and L1. Therefore, in the description below, L0 and L1 are represented as common LX. X LX is equal to 0 in the process of calculating the motion vector difference L0, and X LX is equal to 1 in the process of calculating the motion vector difference L1. In addition, when other list information instead of LX is referenced during the LX motion vector difference calculation process, the other list is represented as LY.

[0084] When an LX motion vector mvLX is used (step S102 of FIG. 19: Yes), potentially candidate LX motion vector predictor candidates are computed to construct a list mvpListLX of potentially candidate LX motion vector predictor candidates (step S103 of FIG. 19) . In the spatial motion vector predictor candidate candidate extraction module 321, the temporal motion vector predictor candidate candidate extraction module 322, the history-based motion vector predictor candidate candidate extraction module 323, and the motion vector predictor potential candidate replenishment module 325 of the identification mode extraction module 301 predictors of normal motion vectors, a set of potentially suitable motion vector predictors are extracted to construct a list mvpListLX of potentially suitable motion vector predictors. The detailed processing procedure of step S103 in FIG. 19 using the flowchart of FIG. 20.

[0085] Next, the potentially candidate motion vector predictor candidate selection module 327 selects the LX motion vector predictor mvpLX from the mvpListLX list of potentially candidate LX motion vector predictor candidates (step S104 of FIG. 19). Here, one element (i-th element when counting from element 0) in the mvpListLX list of potentially candidate motion vector predictors is represented as mvpListLX[i]. Each motion vector difference, which is the difference between the motion vector mvLX and each motion vector predictor candidate mvpListLX[i] stored in the motion vector predictor candidate list mvpListLX, is calculated. The amount of code when motion vector differences are encoded is computed for each element (potential motion vector predictor candidate) of the list mvpListLX of motion vector predictor candidates. Then, a motion vector predictor candidate mvpListLX[i] that minimizes the amount of code for each motion vector predictor candidate from among the elements registered in the motion vector predictor candidate list mvpListLX is selected as the motion vector predictor mvpLX, and its index i turns out. When a plurality of motion vector predictor candidates having the smallest amount of generated code in the mvpListLX list of motion vector predictor candidates is provided, the motion vector predictor candidate mvpListLX[i] represented by the lower number at index i in the mvpListLX candidate list of predictor candidates motion vectors is chosen as the optimal motion vector predictor mvpLX, and its index i is obtained.

[0086] Next, the motion vector subtractor 328 subtracts the selected LX motion vector predictor mvpLX from the LX motion vector mvLX, and calculates the LX motion vector difference mvdLX as mvdLX=mvLX-mvpLX (step S105 of FIG. 19).

[0087] Normal Motion Vector Predictor Identification Mode Extractor (Normal AMVP): Decoding Side Description

Next, the normal motion vector predictor identification mode processing procedure on the decoding side will be described with reference to FIG. 25. On the decoding side, in the spatial motion vector predictor potential candidate extractor 421, the temporal motion vector predictor candidate candidate extractor 422, the history-based potential candidate motion vector predictor candidate extractor 423, and the vector predictor candidate candidate replenishment module 425 motion vector, a motion vector for use in inter-prediction of the normal motion vector predictor identification mode is computed for each of L0 and L1 (steps S201 to S206 of FIG. 25). Specifically, when the target block prediction mode PredMode is inter-prediction (MODE_INTER) and the target block inter-prediction mode is L0 prediction (Pred_L0), a list mvpListL0 of potentially candidate L0 motion vector predictor candidates is computed to select the vector predictor mvpL0 motion, and the motion vector mvL0 L0 is computed. When the inter-prediction mode of the target block is L1 prediction (Pred_L1), the L1 motion vector predictor candidate list mvpListL1 is calculated to select the motion vector predictor mvpL1, and the L1 motion vector mvL1 is calculated. When the inter-prediction mode of the target block is bi-prediction (Pred_BI), L0 prediction and L1 prediction are performed, the list mvpListL0 of potentially suitable L0 motion vector predictors is calculated to select the predictor mvpL0 of the L0 motion vector, the motion vector mvL0 L0 is calculated, the list mvpListL1 of potentially candidate L1 motion vector predictors is computed to select the L1 motion vector predictor mvpL1, and each L1 motion vector mvL1 is computed.

[0088] Although the motion vector calculation process is performed for each of L0 and L1 on the decoding side, similarly to the encoding side, the motion vector calculation process becomes a process common to L0 and L1. Therefore, in the description below, L0 and L1 are represented as common LX. LX represents an inter-prediction mode for use in inter-prediction of a target coding block. X is 0 in the process of calculating the motion vector L0, and X is 1 in the process of calculating the motion vector L1. In addition, when other reference list information instead of a reference list identical to the reference list of the reference list identical to the reference list LX of the calculation target is referenced during the calculation process of the motion vector LX, the other reference list is represented as LY.

[0089] When an LX motion vector mvLX is used (step S202 of FIG. 25: Yes), potentially candidate LX motion vector predictor candidates are computed to construct a list mvpListLX of potentially candidate LX motion vector predictor candidates (step S203 of FIG. 25) . In the spatial motion vector predictor candidate candidate extractor 421, the temporal motion vector predictor candidate candidate extractor 422, the history-based motion vector predictor candidate candidate extractor 423, and the motion vector predictor candidate replenishment module 425 of the identification mode extractor 401 predictors of normal motion vectors, a set of potentially suitable motion vector predictors are computed to construct a list mvpListLX of potentially suitable motion vector predictors. The following describes the detailed processing procedure of step S203 in FIG. 25 using the flowchart of FIG. 20.

[0090] Then, the motion vector predictor candidate mvpListLX[mvpIdxLX] corresponding to the motion vector predictor index mvpIdxLX decoded and provided by the bit string decoding module 201 from the mvpListLX list of candidate motion vector predictor candidates mvpListLX is extracted as the selected motion vector predictor mvpLX in module 426 for selecting potentially suitable motion vector predictor options (step S204 of FIG. 25).

[0091] Next, the motion vector addition unit 427 sums the LX motion vector difference mvdLX that is decoded and provided by the bit string decoding unit 201 and the LX motion vector predictor mvpLX, and calculates the LX motion vector mvLX as mvLX=mvpLX+mvdLX (step S205 according to Fig. 25).

[0092] Normal Motion Vector Predictor Identification Mode Extractor (Normal AMVP): Motion Vector Prediction Method

Fig. 20 is a flowchart showing a processing procedure of the normal motion vector predictor identification mode extraction process having a function common to the normal motion vector predictor identification mode extraction unit 301 of the frame encoding apparatus and the normal motion vector predictor identification mode extraction unit 401 of the decoding apparatus. frames according to an embodiment of the present invention.

[0093] The normal motion vector predictor identification mode extractor 301 and the normal motion vector predictor identification mode extractor 401 include a mvpListLX list of potentially suitable motion vector predictor options. The motion vector predictor candidate list mvpListLX has a list structure and contains a storage area in which the motion vector predictor index indicating the location in the motion vector predictor candidate list and the motion vector predictor candidate corresponding to the index are stored as elements. The motion vector predictor index number starts from 0, and the motion vector predictor candidate candidates are stored in the mvpListLX list of candidate motion vector predictor candidates. In the present embodiment, it is assumed that at least two potentially suitable motion vector predictor candidates (inter-prediction information) can be registered in the mvpListLX list of potential motion vector predictor candidates. In addition, the numCurrMvpCand variable indicating the number of motion vector predictor candidates registered in the motion vector predictor candidate list mvpListLX is set to 0.

[0094] The potential candidate spatial motion vector predictor candidate extractors 321 and 421 extract potentially candidate motion vector predictor candidates from adjacent blocks on the left side. In this process, the motion vector predictor mvLXA is extracted with reference to the inter-prediction information of the adjacent block on the left side (A0 or A1 in FIG. 11), i.e. to a flag indicating whether or not a potentially suitable motion vector predictor candidate, motion vector, reference index, or the like can be used, and the extracted mvLXA is added to the list mvpListLX of potentially candidate motion vector predictor candidates (step S301 of FIG. 20). In addition, X is 0 during L0 prediction and X is 1 during L1 prediction (this is true below). Next, the candidate spatial motion vector predictor candidate extractors 321 and 421 extract the candidate motion vector predictor candidate from the upper side adjacent block. In this process, the motion vector predictor mvLXB is extracted with reference to the inter-prediction information of the neighboring block on the upper side (B0, B1 or B2 in FIG. 11), i.e. to a flag indicating whether or not a potential candidate motion vector predictor can be used, a motion vector, a reference index, etc., and mvLXB is added to the mvpListLX list of potentially candidate motion vector predictors if the extracted mvLXA is not equal to the extracted mvLXB (step S302 in Fig. 20). The processing of steps S301 and S302 in FIG. 20 is common except that the referenced neighboring block positions and the number of referenced neighboring blocks are different, and an availableFlagLXN flag is retrieved indicating whether or not a potentially suitable coding block motion vector predictor option can be used, vector mvLXN movement and reference index refIdxN (N represents A or B, and this is true below).

[0095] Next, modules 322 and 422 extract potentially good temporal motion vector predictor candidates extract potentially good motion vector predictor candidates from blocks in a frame whose time is different from that of the current target frame. In this process, the availableFlagLXCol flag indicating whether or not the motion vector predictor candidate of the frame encoding block of another time point can be used, the motion vector mvLXCol, the reference index refIdxCol and the reference list listCol are extracted, and mvLXCol is added to the mvpListLX list of potentially candidate predictors. motion vectors (step S303 of FIG. 20).

[0096] In addition, it is contemplated that the processes of modules 322 and 422 for extracting potentially suitable motion vector temporal predictors may be omitted in units of sequences (SPS), frames (PPS), or slices.

[0097] Next, the history-based motion vector predictor potential candidates 323 and 423 add the history-based motion vector predictor candidates registered in the history-based motion vector predictor candidate list HmvpCandList to the mvpListLX of potential candidates. motion vector predictors (step S304 of FIG. 20). The details of the registration processing procedure of step S304 will be described below using the flowchart of FIG. 29.

[0098] Next, the potential candidate motion vector predictor candidate replenishment modules 325 and 425 add candidate motion vector predictor candidates having a predetermined value, e.g., (0, 0), until the mvpListLX list of potential candidate motion vector predictor candidates not satisfied (S305 of FIG. 20).

[0099] Normal Combination Mode Extractor (Normal Combination)

The normal combining mode extractor 302 of FIG. 18 includes a spatial candidate pool extractor 341, a temporal pool candidate extractor 342, an average pool candidate extractor 344, a history-based pool candidate extractor 345, a pool candidate augmentation module 346, and a module 347 for selecting potentially suitable combination options.

[0100] The normal combining mode extractor 402 of FIG. 24 includes a spatial join candidate extractor 441, a temporal join candidate extractor 442, an average join candidate extractor 444, a history based join candidate extractor 445, a join replenishment module 446, and a module 447 for selecting potentially suitable combination options.

[0101] FIG. 21 is an explanatory flowchart showing a procedure of a normal combining mode extraction process having a function common to a normal combining mode extraction unit 302 of a frame encoding apparatus and a normal combining mode extraction unit 402 of a frame decoding apparatus according to an embodiment of the present invention.

[0102] The following describes the various processes step by step. Although the following describes the case in which the slice_type of the slice is a B-slice, unless otherwise indicated in the description below, the present invention can also be applied to the case of a P-slice. However, when the slice_type of a slice is a P-slice, since only L0 prediction (Pred_L0) is provided as an inter-prediction mode, and L1 prediction (Pred_L1) and bi-prediction (Pred_BI) are not present, the process associated with L1 may go down.

[0103] The normal merge mode extractor 302 and the normal merge mode extractor 402 have a mergeCandList of potentially suitable merge options. The list mergeCandList of candidate merges has a list structure and contains a merge index indicating a location in the list of candidate merges and a storage area in which candidate merges corresponding to the index are stored as elements. The merge index number starts at 0, and potentially suitable merges are stored in the mergeCandList storage area of potentially suitable merges. In the subsequent process, a merge candidate i of the merge index registered in the merge candidate merge list mergeCandList is represented by mergeCandList[i]. In the present embodiment, it is assumed that at least six potentially suitable merges (inter-prediction information) can be registered in the mergeCandList of potentially suitable merges. Additionally, the numCurrMergeCand variable indicating the number of merge candidates registered in the mergeCandList of merge candidates is set to 0.

[0104] The spatial candidate merge extractor 341 and the spatial candidate merge extractor 441 extract spatial candidate merges from the blocks (B1, A1, B0, A0, and B2 of FIG. 11) adjacent to the left side and top side of the target block in the order of B1, A1, B0, A0 and B2, from the encoding information stored in the memory 111 for storing encoding information of the frame encoder or in the memory 205 for storing encoding information of the frame decoder, and register the extracted spatial potential suitable merge options in the list mergeCandList of potentially suitable merge options (step S401 of FIG. 21). Here, N is set to indicate any one of the spatial pool candidate B1, A1, B0, A0, and B2 and the temporal pool candidate Col. The availableFlagN flag indicating whether inter-prediction information of block N is available as a spatial join candidate is available or not, the reference index refIdxL0N L0 and the reference index refIdxL1N L1 of the spatial join candidate N, the L0 prediction flag predFlagL0N indicating whether or not L0 prediction, L1 prediction flag predFlagL1N indicating whether or not L1 prediction is executed, L0 motion vector mvL0N, and L1 motion vector mvL1N. However, in the present embodiment, since the matching candidate is extracted without referring to the block inter-prediction information included in the coding block serving as the target, the spatial matching candidate using the block inter-prediction information included in the target coding block, not retrieved.

[0105] Next, the temporal candidate merge extractor 342 and the temporal candidate merge extractor 442 extract the temporal candidate merges from the frames of different points in time, and register the extracted temporal candidate merges in the list of candidate merges mergeCandList (step S402 in Fig. 21). Extracted flag availableFlagCol indicating whether or not a temporary join candidate can be used, predFlagL0Col L0 prediction flag indicating whether or not L0 prediction of the temporary join candidate is performed, predFlagL1Col L1 prediction flag indicating whether or not L1 prediction, L0 motion vector mvL0Col and L1 motion vector mvL1Col.

[0106] In addition, it is contemplated that the processes of temporal candidate merge extractor 342 and temporal candidate merge extractor 442 may be omitted in units of sequences (SPS), frames (PPS), or slices.

[0107] Next, the history-based candidate pooling candidate extractor 345 and the history-based candidate pooling module 445 register the history-based motion vector predictor candidates registered in the history-based motion vector predictor candidate list HmvpCandList , in the mergeCandList of potentially suitable merge options (step S403 of FIG. 21).

In addition, when the number numCurrMergeCand of candidate merges registered in the list of merge candidates mergeCandList is less than the maximum number of candidate merges MaxNumMergeCand, the maximum number of candidate merges MaxNumMergeCand is set as the upper limit of the number of candidate merges numCurrMergeCand registered in mergeCandList of potentially suitable merges, and potentially suitable merges based on history are retrieved and registered in the mergeCandList of potentially suitable merges.

[0108] Next, the average candidate merge extractor 344 and the average candidate merge extractor 444 extract the average candidate merge from the mergeCandList of candidate merges, and add the extracted average candidate merge to the mergeCandList of candidate merges. (step S404 of FIG. 21).

In addition, when the number numCurrMergeCand of candidate merges registered in the list of merge candidates mergeCandList is less than the maximum number of candidate merges MaxNumMergeCand, the maximum number of candidate merges MaxNumMergeCand is set as the upper limit of the number of candidate merges numCurrMergeCand registered in mergeCandList of merge candidates, and the average merge candidates are retrieved and registered in the mergeCandList of merge candidates.

Here, the average candidate merge is a new candidate merge having a motion vector obtained by averaging the motion vectors of the first candidate merge and the second candidate merge registered in the list of candidate merges mergeCandList for each of L0- predictions and L1 predictions.

[0109] Then, in the candidate merge replenishment module 346 and the candidate merge replenishment module 446, when the number numCurrMergeCand of candidate merges registered in the mergeCandList of candidate merges is less than the maximum number of candidate merges MaxNumMergeCand , the maximum number MaxNumMergeCand of merge candidates is set as an upper limit of the number of merge candidates numCurrMergeCand registered in the merge candidate mergeCandList, and an additional merge candidate is retrieved and registered in the merge candidate list mergeCandList (step S405 of FIG. 21) . In P-slice, a candidate merge for which the motion vector has a value of (0, 0) and the prediction mode is L0 prediction (Pred_L0) is added using the maximum number MaxNumMergeCand of potentially eligible merges as an upper limit. In the B-slice, a potentially suitable join case for which the motion vector has a value of (0, 0) and the prediction mode is bi-prediction (Pred_BI) is added. The pivot index when a potentially suitable join option is added is different from the pivot index previously added.

[0110] Next, the merge candidate selection module 347 and the merge candidate selection module 447 select merge candidates from the merge candidates registered in the merge candidate list mergeCandList. The coding-side candidate join selection unit 347 selects the join candidate by calculating the amount of code and the amount of distortion, and provides a join index indicating the selected join candidate and inter-prediction information of the join candidate to the motion compensation prediction unit 306 through the inter-prediction mode determination module 305 . On the other hand, the decoding side candidate join selection unit 447 selects the join candidate based on the decoded join index, and provides the selected join candidate to the motion compensation prediction unit 406 .

[0111] Updating the list of potentially suitable motion vector predictor options based on history

The following describes in detail the initialization method and the method for updating the list HmvpCandList of potentially suitable motion vector predictor candidates based on the history provided in the encoding information storage 111 on the encoding side and in the encoding information storage 205 on the decoding side. Fig. 26 is an explanatory flowchart showing a processing procedure for initializing/updating a list of potentially candidate candidate motion vector predictors based on history.

[0112] In the present embodiment, it is assumed that the list HmvpCandList of potentially suitable motion vector predictors based on history is updated in the encoding information storage 111 and the encoding information storage 205 . A history-based motion vector predictor candidate candidate list updater may be set in the inter-prediction module 102 and the inter-prediction module 203 to update the HmvpCandList list of potentially candidate historical motion vector predictor candidates.

[0113] The history-based motion vector predictor candidate list HmvpCandList is initially set at the beginning of the slice, the history-based motion vector predictor candidate list HmvpCandList is updated when the normal motion vector predictor identification mode or the normal combining mode is selected by the method determination module 105 prediction on the encoding side, and the list HmvpCandList of potentially suitable motion vector predictors based on history is updated when the prediction information decoded by the bit string decoding unit 201 is set to the normal motion vector predictor identification mode or normal combining mode on the decoding side.

[0114] The inter-prediction information used when inter-prediction is performed in the normal motion vector predictor identification mode or the normal combining mode is registered as the inter prediction information potential candidate hMvpCand in the history-based motion vector predictor potential candidate list HmvpCandList . Potentially suitable inter prediction information hMvpCand includes L0 reference index refIdxL0, L1 reference index refIdxL1, L0 prediction flag predFlagL0 indicating whether L0 prediction is performed or not, L1 prediction flag predFlagL1 indicating whether L1 is performed or not. -prediction, motion vector mvL0 L0 and motion vector mvL1 L1.

[0115] When there is inter prediction information having a value identical to that of the inter prediction information candidate candidate hMvpCand, from among the elements (i.e., inter prediction information) registered in the list HmvpCandList of motion vector predictor candidate candidates based on provided in the encoding information storage 111 on the encoding side and in the encoding information storage 205 on the decoding side, the item is removed from the list HmvpCandList of potentially suitable motion vector predictors based on the history. On the other hand, when there is no inter prediction information having a value identical to that of the potential candidate hMvpCand of the inter prediction information, the element at the head of the list HmvpCandList of potential candidates of motion vector predictors based on the history is removed, and the potential candidate hMvpCand of the inter prediction information is added to the end of the HmvpCandList list of potentially suitable motion vector predictors based on history.

[0116] The number of elements of the HmvpCandList list of potentially suitable motion vector predictors based on history provided in the encoding information storage 111 on the encoding side and in the encoding information storage 205 on the decoding side according to the present invention is supposed to be six.

[0117] First, the list HmvpCandList of potentially suitable motion vector predictor options based on history is initialized in units of slices (step S2101 of FIG. 26). All elements of the list HmvpCandList of potential candidates of motion vector predictors based on history are empty at the beginning of the slice, and the value of the number NumHmvpCand of potential candidates of motion vector predictors based on history (the current number of candidates of potential candidates) registered in the list HmvpCandList of potential candidates of motion vector predictors based on history, can be set to 0.

[0118] In addition, the initialization of the list HmvpCandList of potentially suitable history-based motion vector predictor candidates is performed in units of slices (of the first slice coding unit), but may be performed in units of frames, tiles, or rows of the treeblock.

[0119] Then, the following process of updating the list HmvpCandList of potentially suitable motion vector predictor options based on the history is iteratively performed for each coding block in the slice (steps S2102-S2107 of FIG. 26).

[0120] First, the initial setting is performed for each coding block. The identicalCandExist flag indicating whether or not there is an identical candidate is set to false, and the removal target index removeIdx indicating the potential candidate to be removed is set to 0 (step S2103 of FIG. 26).

[0121] It is determined whether or not there is a potential candidate hMvpCand of registration target inter prediction information (step S2104 of FIG. 26). When the encoding side prediction method determination unit 105 determines that the mode is the normal motion vector predictor identification mode or the normal combining mode, or when the decoding side bit string decoding unit 201 decodes the mode as the normal motion vector predictor identification mode or the normal merging, its inter-prediction information is set as a potential candidate hMvpCand of the registration target inter-prediction information. When the encoding side prediction method determination unit 105 determines that the mode is an intra prediction mode, a sub-block motion vector predictor identification mode, or a sub-block combining mode, or when the decoding side bit string decoding unit 201 decodes the mode as of the intra-prediction mode, the sub-block based motion vector predictor identification mode, or the sub-block based combining mode, the process of updating the HmvpCandList list of potentially suitable history-based motion vector predictors is not performed, and the potentially suitable candidate hMvpCand of registration target inter-prediction information does not exist. When there is no potential candidate hMvpCand of registration target inter prediction information, steps S2105 to S2106 are skipped (step S2104 of FIG. 26: "No"). When there is a potential candidate hMvpCand of registration target inter prediction information, the processing from step S2105 is performed (step S2104 of FIG. 26: "Yes").

[0122] Next, it is determined whether or not there is an element (inter prediction information) having a value identical to the value of a potentially suitable variant hMvpCand of the registration target inter prediction information, i. an identical element from among the elements of the list HmvpCandList of potentially suitable motion vector predictors based on history (step S2105 of FIG. 26). Fig. 27 is a flowchart for a process of identical check processing. When the value of the number NumHmvpCand of the history-based motion vector predictor potential candidates NumHmvpCand is 0 (S2121 of FIG. 27: No), the history-based motion vector predictor candidate list HmvpCandList is empty, and there is no identical candidate, so that steps S2122-S2125 of FIG. 27 are skipped, and the current identity check processing routine is terminated. When the value of the number NumHmvpCand of potentially suitable history-based motion vector predictor cases exceeds 0 (Yes in step S2121 of FIG. 27), the processing of step S2123 is iteratively performed until the history-based motion vector predictor index hMvpIdx changes from 0 to NumHmvpCand-1 (steps S2122-S2125 of FIG. 27). First, a comparison is made as to whether or not the hMvpIdx th element of HmvpCandList[hMvpIdx] is, when counted from the 0 th element of the history-based motion vector predictor candidate candidate list, identical to the inter prediction information candidate candidate hMvpCand (step S2123 to Fig. 27). When they are identical (step S2123 of FIG. 27: Yes), the identicalCandExist flag indicating whether or not there is an identical candidate is set to true, and the removeIdx index of the removal target indicating the element position of the removal target, is set to the current value of the history-based motion vector predictor index hMvpIdx, and the current identical checking process is terminated. When they are not identical (step S2123 of FIG. 27: No), hMvpIdx is incrementally increased by 1. If the history-based motion vector predictor index hMvpIdx is less than or equal to NumHmvpCand-1, the processing from step S2123 is executed.

[0123] With reference to the flowchart of the method of FIG. 26 again, the process of shifting and adding a list item HmvpCandList of potentially suitable motion vector predictors based on history is performed (step S2106 of FIG. 26). Fig. 28 is a flowchart for the shift/add item HmvpCandList processing procedure of potentially suitable motion vector predictor options based on the history of step S2106 of FIG. 26. First, it is determined whether to add a new element after deleting an element stored in the HmvpCandList list of potentially suitable motion vector predictors based on history, or to add a new element without deleting the element. Specifically, a comparison is made as to whether or not the identicalCandExist flag indicating whether or not an identical candidate exists or whether or not NumHmvpCand is equal to or not six is "true" (step S2141 of FIG. 28). When either the condition that the identicalCandExist flag indicating whether or not there is an identical candidate is "true" or the condition that the current number of potential candidates NumHmvpCand is six is satisfied (step S2141 of FIG. 28: " Yes"), a new element is added after deleting an element stored in the HmvpCandList list of potentially eligible motion vector predictors based on history. The initial value of index i is set to the value removeIdx+1. The bit-shift process of step S2143 is iteratively executed from this initial value to NumHmvpCand (steps S2142-S2144 of FIG. 28). By copying the element HmvpCandList[i] to HmvpCandList[i-1], the element is shifted forward (step S2143 in FIG. 28) and i is gradually increased by 1 (steps S2142-S2144 in FIG. 28). Then, the inter-prediction information candidate hMvpCand is added to the (NumHmvpCand-1)th element of HmvpCandList[NumHmvpCand-1] counting from the 0th element, which corresponds to the end of the history-based motion vector predictor potential candidate list (step S2145 28), and the current process of shifting/adding an element of the list HmvpCandList of potentially suitable variants of motion vector predictors based on the history is completed. On the other hand, when neither the condition that the identicalCandExist flag indicating whether or not there is an identical candidate is "true" nor the condition that the current number of potential candidates NumHmvpCand is six is satisfied (step S2141 of 28: "No"), the potential candidate hMvpCand of the inter prediction information is added to the end of the history-based motion vector predictor candidates list without deleting the element stored in the history-based motion vector predictor candidates list HmvpCandList (step S2146 according to Fig. 28). Here, the end of the list of potentially candidate history-based motion vector predictors is the NumHmvpCand-th element of HmvpCandList[numHmvpCand] counted from the 0-th element. In addition, NumHmvpCand is incremented by 1, and the current process of shifting/adding an element of the list HmvpCandList of potentially suitable motion vector predictors based on history is completed.

[0124] FIG. 31 is an explanatory diagram showing an example of a process for updating a list of potentially candidate motion vector predictors based on history. When a new element is added to the history-based motion vector predictor potential candidate list HmvpCandList in which six elements (inter-prediction information) are registered, the elements are compared with the new inter-prediction information in order from the leading element of the motion vector predictor candidate list HmvpCandList based on history (FIG. 31A). If the new element has a value identical to the value of the third element of HMVP2 from the beginning of the HMVPCandList list of potentially candidate history-based motion vector predictors, the HMVP2 element is removed from the HMVPCandList of potentially candidate history-based motion vector predictor candidates, and subsequent HMVP3-HMVP5 elements are shifted forward ( are copied) one at a time, and a new element is added to the end of the HmvpCandList list of potentially suitable history-based motion vector predictor candidates (FIG. 31B) to complete the update of the HmvpCandList list of potentially candidate history-based motion vector predictor candidates (FIG. 31C).

[0125] The process of extracting potentially suitable variants of motion vector predictors based on the history

The following describes in detail a method for extracting a potentially suitable history-based motion vector predictor candidate from a history-based motion vector predictor potential candidate list HmvpCandList, which is the processing procedure of step S304 of FIG. 20, which is a process common to the module 323 for extracting potential candidate motion vector predictor cases based on the history of the normal motion vector predictor identification mode extractor 301 on the encoding side and the module 423 for extracting potential candidate motion vector predictor candidates based on the history of the mode extraction module 401. identify predictors of normal motion vectors on the decoding side. Fig. 29 is an explanatory flowchart showing a history-based motion vector predictor potential candidate extraction processing procedure.

[0126] When the current number numCurrMvpCand of potential candidate motion vector predictor candidates is greater than or equal to the maximum number of elements in the list mvpListLX of candidate motion vector predictor candidates (here, 2), or the value of the number NumHmvpCand of potential candidate motion vector predictor candidates based on history is 0 ( No at step S2201 of Fig. 29), the processing of steps S2202 to S2209 of Fig. 29 is omitted, and the history-based motion vector predictor candidate extraction process is terminated. When the current number of candidate motion vector predictor candidates numCurrMvpCand is less than 2, which is the maximum number of elements of the candidate motion vector predictor candidate list mvpListLX, and the value of the number NumHmvpCand of candidate motion vector predictor candidates based on history is greater than 0 ("Yes" in step S2201 in Fig. 29), the processing of steps S2202-S2209 in Fig. 29 is being implemented.

[0127] Next, the processing of steps S2203-S2208 in FIG. 29 is iteratively executed until index i changes from 1 to the lower of 4 and the number numCheckedHMVPCand of potentially suitable history-based motion vector predictor candidates (steps S2202-S2209 of FIG. 29). When the current number of candidate motion vector predictor candidates numCurrMvpCand is greater than or equal to 2, which is the maximum number of elements of the candidate motion vector predictor candidate list mvpListLX (step S2203 of FIG. 29: No), the processing of steps S2204 to S2209 in FIG. 29 is omitted, and the current history-based motion vector predictor potential candidate extraction processing procedure is terminated. When the current number of candidate motion vector predictor candidates numCurrMvpCand is less than 2, which is the maximum number of elements of the candidate motion vector predictor candidate list mvpListLX (step S2203 of FIG. 29: "Yes"), the processing from step S2204 of FIG. 29 is being implemented.

[0128] Next, the processing of steps S2205-S2207 is performed for Y=0 and 1 (L0 and L1) (steps S2204-S2208 in FIG. 29). When the current number of candidate motion vector predictor candidates numCurrMvpCand is greater than or equal to 2, which is the maximum number of elements of the candidate motion vector predictor candidate list mvpListLX (step S2205 of FIG. 29: "No"), the processing of steps S2206 to S2209 of FIG. 29 is omitted, and the current history-based motion vector predictor potential candidate extraction processing procedure is terminated. When the current number of candidate motion vector predictor candidates numCurrMvpCand is less than 2, which is the maximum number of elements of the candidate motion vector predictor candidate list mvpListLX (step S2205 of FIG. 29: "Yes"), the processing from step S2206 of FIG. 29 is being implemented.

[0129] Then, in the case of an element that has a reference index that is identical to the motion vector reference index refIdxLX of the encoding/decoding target, and that is different from any element of the motion vector predictor list mvpListLX in the list HmvpCandList of potentially suitable motion vector predictor candidates based on history (" Yes" in step S2206 of FIG. 29), the motion vector LY of the motion vector predictor based on the history HmvpCandList[NumHmvpCand-i] is added to the numCurrMvpCand th element of mvpListLX[numCurrMvpCand] when counting from the 0th element of the potentially suitable candidate list motion vector predictors (step S2207 of FIG. 29), and the current number numCurrMvpCand of potentially suitable motion vector predictor options is incremented by one. When there is no element that has a reference index identical to the motion vector reference index refIdxLX of the encoding/decoding target and that is different from any element of the motion vector predictor list mvpListLX in the history-based motion vector predictor candidate list HmvpCandList ("No" in step S2206) in Fig. 29), the additional processing of step S2207 is skipped.

[0130] The above processing of steps S2205-S2207 in FIG. 29 is performed for L0 and L1 (steps S2204-S2208 in FIG. 29). When the index i is incremented by 1 and the index i is less than or equal to the smaller of 4 and the number of NumHmvpCand potentially suitable history-based motion vector predictors, the processing from step S2203 is performed again (steps S2202-S2209 of FIG. 29).

[0131] The process of extracting potentially suitable combinations based on history

The following describes in detail a method for extracting potentially matching history-based merges from the list of potentially matching history-based merges HmvpCandList, which is the processing procedure of step S404 of FIG. 21, which is a process common to the potentially good combining mode extractor 345 based on the history of the normal combining mode extractor 302 on the encoding side and the potential candidate combining mode extractor 445 based on the history of the normal combining mode extractor 402 on the decoding side. Fig. 30 is an explanatory flowchart showing a processing procedure for extracting potentially suitable combinations based on history.

[0132] First, an initialization process is performed (step S2301 of FIG. 30). Each (numCurrMergeCand-1)th element from 0 to isPruned[i] is set to false, and numOrigMergeCand is set to the number of numCurrMergeCand elements registered in the current list of potentially suitable merges.

[0133] Next, the hMvpIdx index initial value is set to 1, and the adding process of steps S2303-S2310 in FIG. 30 is iteratively executed until the hMvpIdx index changes from the initial value to NumHmvpCand (steps S2302-S2311 in FIG. 30). If the number of numCurrMergeCand elements registered in the current merge candidate list is not less than or equal to (MaxNumMergeCand maximum merge candidates is 1), candidate merges are added to all elements of the merge candidate list, so that the current retrieval process is potentially matching options based on the history is completed ("No" at step S2303 of Fig. 30). When the number numCurrMergeCand of items registered in the current merge candidate list is less than or equal to (the maximum number of candidate merge candidates MaxNumMergeCand is 1) ("Yes" in step S2303 of FIG. 30), the processing from step S2304 is executed.

[0134] First, the value "false" is set in the sameMotion (step S2304 in Fig. 30). Then, the initial value of the index i is set to 0, and the processing of steps S2306 and S2307 in FIG. 30 is executed from this initial value to 1 (S2305-S2308 in FIG. 30).

[0135] Next, a comparison is made as to whether or not (NumHmvpCand-hMvpIdx)-th element of HmvpCandList[NumHmvpCand-hMvpIdx] when counted from 0 for a history-based list of potentially eligible motion vector predictor candidates is identical to the i-th element of mergeCandList[i ] while counting from 0 for the list of potentially matching merge options (step S2306 in FIG. 30). Here, the fact that the values of the candidate joins are identical indicates that the values of all components (inter prediction mode, reference index, and motion vector) of the candidate joins are identical. However, the processing of step S2306 is limited to the case in which hMvpIdx exceeds NumHmvpCand-2, mergeCandList[i] is a spatial merge candidate, and isPruned[i] is false. If the values are identical ("Yes" in step S2306 of FIG. 30), "true" is set for both sameMotion and isPruned[i] (step S2307 of FIG. 30). If the values are not identical ("No" in step S2306 of FIG. 30), the processing of step S2307 is skipped. When the iterative processing of steps S2305 to S2308 in FIG. 30 ends, a comparison is made as to whether or not sameMotion is "false" (step S2309 in FIG. 30), (NumHmvpCand-hMvpIdx)th element of HmvpCandList[NumHmvpCand-hMvpIdx] when counted from 0 for a list of potentially matching vector predictor options history-based motion is added to the numCurrMergeCand th element of mergeCandList[numCurrMergeCand] of the merge candidate list when sameMotion is false ("Yes" in step S2309 in FIG. 30), and numCurrMergeCand is gradually incremented by 1 (step S2310 to Fig. 30). The hMvpIdx index is gradually increased by 1 (step S2302 in FIG. 30), and the iterative processing of steps S2302-S2311 in FIG. 30 is in progress.

[0136] When the check of all elements of the history-based motion vector predictor candidate candidate list is completed, or when candidate joins are added to all elements of the candidate join list, the current history-based matching candidate extraction process ends.

[0137] Motion compensation prediction process

Motion compensation prediction module 306 obtains the position and size of a block that is the current target of the encoding prediction process. In addition, the motion compensation prediction unit 306 obtains inter prediction information from the inter prediction mode determination unit 305 . The reference index and the motion vector are extracted from the obtained inter-prediction information, and the prediction signal is generated after the frame signal of the position at which the reference frame identified by the reference index in the decoded frame memory 104 moves from a position identical to the position of the frame signal is obtained. prediction block, by the magnitude of the motion vector.

[0138] The motion-compensated prediction signal is provided to the prediction method determination unit 105 using the prediction signal obtained from one reference frame as the motion-compensated prediction signal when the inter-prediction mode in inter-prediction is prediction from one reference frame , such as L0 prediction or L1 prediction, and using a prediction signal obtained by weighted average prediction signals obtained from two reference frames as a motion compensation prediction signal when the prediction mode is prediction from two reference frames , such as bi-prediction inter-prediction mode. Although the weighted average bi-prediction ratio here is 1:1, weighted averaging can be performed using a different ratio. For example, the weighting factor may increase as the inter-frame interval between a frame that is a prediction target and a reference frame decreases. In addition, the weighting factor may be calculated using an appropriate table between combinations of interframe intervals and weighting factors.

[0139] The motion compensation prediction module 406 has a function similar to that of the motion compensation prediction module 306 on the encoding side. The motion-compensated prediction module 406 obtains inter-prediction information from the normal motion vector predictor identification mode extractor 401, the normal combining mode extractor 402, the subblock-based motion vector predictor identification mode extractor 403, and the subblock-based combining mode extractor 404 via switch 408. The motion compensation prediction module 406 provides the received motion compensation prediction signal to the frame signal overlay module 207 for decoding.

[0140] Regarding the inter prediction mode

The process of performing prediction from one reference frame is defined as uni-prediction. In the case of uniprediction, prediction is performed using one of the two reference frames registered in the L0 and L1 reference lists, such as L0 prediction or L1 prediction.

[0141] FIG. 32 shows a case of uni-prediction in which the clock time of the reference frame (RefL0Pic) L0 is earlier than the clock time of the target frame (CurPic). Fig. 33 shows a case of uniprediction in which the cycle time of the L0 prediction reference frame is later than the cycle time of the target frame. Similarly, the L0 prediction reference frame of FIG. 32 and 33 may be replaced by an L1 prediction reference frame (RefL1Pic) to perform uni-prediction.

[0142] The process of performing prediction from two reference frames is defined as bi-prediction, and the bi-prediction is represented as bi-prediction using both L0 prediction and L1 prediction. Fig. 34 shows a case of biprediction in which the cycle time of the L0 prediction reference frame is earlier than the clock time of the target frame, and the clock time of the L1 prediction reference frame is later than the clock time of the target frame. Fig. 35 shows a case of bi-prediction in which the cycle times of the L0 prediction reference frame and the L1 prediction reference frame are earlier than the cycle time of the target frame. Fig. 36 shows a case of bi-prediction in which the cycle time of the L0 prediction reference frame and the cycle time of the L1 prediction reference frame are later than the cycle time of the target frame.

[0143] As described above, the relationship between L0/L1 prediction type and time can be used without limitation of L0, which is in the previous direction, and L1, which is in the future direction. In the case of bi-prediction, each of L0 prediction and L1 prediction may be performed using the same reference frame. In addition, it is determined whether motion compensation prediction according to uni-prediction or bi-prediction should be performed based on, for example, information (eg, a flag) indicating whether or not to use L0 prediction, and whether or not to use L1 prediction. .

[0144] Regarding the reference index

In an embodiment of the present invention, it is possible to select an optimal reference frame from a plurality of reference frames in motion-compensated prediction in order to improve the accuracy of motion-compensated prediction. Thus, a reference frame used in motion compensation prediction is used as a reference index, and the reference index is encoded in the bitstream along with the motion vector difference.

[0145] Motion compensation process based on normal motion vector predictor identification mode

As shown in the inter-prediction module 102 on the encoding side of FIG. 16, when the inter prediction information from the normal motion vector predictor identification mode extraction unit 301 is selected in the inter prediction mode determination unit 305, the motion compensation prediction unit 306 obtains the inter prediction information from the inter prediction mode determination unit 305, extracts the inter prediction mode -prediction, reference index, and motion vector of the current target block, and generates a motion-compensated prediction signal. The generated motion compensation prediction signal is provided to the prediction method determining unit 105 .

[0146] Similarly, as shown in the inter prediction unit 203 on the decoding side of FIG. 22, when the switch 408 is connected to the normal motion vector predictor identification mode extraction unit 401 in the decoding process, the motion compensation prediction unit 406 obtains inter prediction information from the normal motion vector predictor identification mode extraction unit 401, extracts the inter prediction mode, reference index and a motion vector of the current target block, and generates a motion-compensated prediction signal. The generated motion-compensated prediction signal is provided to the frame superposition module 207 for decoding.

[0147] Motion compensation process based on normal combining mode

In addition, as shown in the inter prediction unit 102 on the encoding side of FIG. 16, when the inter prediction information is selected from the normal combining mode derivation unit 302 in the inter prediction mode determination unit 305, the motion compensation prediction unit 306 obtains the inter prediction information from the inter prediction mode determination unit 305, extracts the inter prediction mode, a reference index and a motion vector of the current target block, and generates a motion-compensated prediction signal. The generated motion compensation prediction signal is provided to the prediction method determining unit 105 .

[0148] Similarly, as shown in the inter prediction unit 203 on the decoding side of FIG. 22, when the switch 408 is connected to the normal combining mode extractor 402 in the decoding process, the motion compensation prediction module 406 obtains inter-prediction information from the normal combining mode extractor 402, extracts the inter-prediction mode, the reference index, and the motion vector of the current target block. and generates a motion-compensated prediction signal. The generated motion-compensated prediction signal is provided to the frame superposition module 207 for decoding.

[0149] Motion compensation process based on sub-block based motion vector predictor identification mode

In addition, as shown in the inter prediction unit 102 on the encoding side of FIG. 16, when the inter prediction information from the sub-block motion vector predictor identification mode extraction unit 303 is selected in the inter prediction mode determination unit 305, the motion compensation prediction unit 306 obtains the inter prediction information from the inter prediction mode determination unit 305, extracts an inter-prediction mode, a reference index, and a motion vector of the current target block, and generates a motion-compensated prediction signal. The generated motion compensation prediction signal is provided to the prediction method determining unit 105 .

[0150] Similarly, as shown in the inter prediction unit 203 on the decoding side of FIG. 22, when the switch 408 is connected to the sub-block motion vector predictor identification mode extraction unit 403 in the decoding process, the motion compensation prediction unit 406 obtains inter-prediction information from the sub-block motion vector predictor identification mode extraction unit 403, extracts the inter- prediction, reference index, and motion vector of the current target block, and generates a motion-compensated prediction signal. The generated motion-compensated prediction signal is provided to the frame superposition module 207 for decoding.

[0151] Motion compensation process based on sub-block combining mode

In addition, as shown in the inter prediction unit 102 on the encoding side of FIG. 16, when the inter prediction information from the sub-block combining mode derivation unit 304 is selected in the inter prediction mode determination unit 305, the motion compensation prediction unit 306 obtains the inter prediction information from the inter prediction mode determination unit 305, extracts the inter prediction mode prediction, reference index, and motion vector of the current target block, and generates a motion-compensated prediction signal. The generated motion compensation prediction signal is provided to the prediction method determining unit 105 .

[0152] Similarly, as shown in the inter prediction unit 203 on the decoding side of FIG. 22, when the switch 408 is connected to the subblock combining mode extractor 404 in the decoding process, the motion compensation prediction module 406 obtains inter prediction information from the subblock combining mode extractor 404, extracts the inter prediction mode, the reference index, and the vector movement of the current target block and generates a motion-compensated prediction signal. The generated motion-compensated prediction signal is provided to the frame superposition module 207 for decoding.

[0153] Motion Compensation Process Based on Affine Transform Prediction

In normal motion vector predictor identification mode and normal pooling mode, affine model motion compensation can be used based on the following flags. The following flags are reflected in the following flags based on the inter-prediction conditions determined by the inter-prediction mode determination unit 305 during encoding, and are encoded in the bitstream. In the decoding process, whether or not to perform motion compensation of the affine model is identified based on the following flags in the bitstream.

[0154] sps_affine_enabled_flag represents whether or not affine model motion compensation can be used in inter-prediction. If sps_affine_enabled_flag is 0, suppression is performed in such a way that it does not represent motion compensation of the affine model in units of sequences. In addition, inter_affine_flag and cu_affine_type_flag are not passed in the syntax of the CU (coding units) of the video sequence for encoding. If sps_affine_enabled_flag is equal to 1, affine model motion compensation can be used in the video sequence for encoding.

[0155] The sps_affine_type_flag represents whether or not six-parameter affine model motion compensation can be used in inter-prediction. If sps_affine_type_flag is 0, suppression is performed in such a way that it does not represent motion compensation of the six-parameter affine model. In addition, cu_affine_type_flag is not passed in the video sequence CU syntax for encoding. If sps_affine_type_flag is equal to 1, six-parameter affine model motion compensation may be used in the video sequence for encoding. When sps_affine_type_flag does not exist, it is assumed to be 0.

[0156] When a P or B slice is decoded, if inter_affine_flag is equal to 1 in the current target CU, the affine model motion compensation is used to generate a motion compensated prediction signal of the current target CU. If inter_affine_flag is 0, the affine model is not used in the current target CU. When inter_affine_flag does not exist, it is assumed to be 0.

[0157] When a P- or B-slice is decoded, if cu_affine_type_flag is equal to 1 in the current target CU, the six-parameter affine model motion compensation is used to generate a motion-compensated prediction signal of the current target CU. If cu_affine_type_flag is equal to 0, the motion compensation of the four-parameter affine model is used to generate the motion-compensated prediction signal of the current target CU.

[0158] In the motion compensation of the affine model, since the reference index and the motion vector are extracted in units of sub-blocks, a motion-compensated prediction signal is generated using the reference index or motion vector, which is the target in units of sub-blocks.

[0159] The four-parameter affine model is a mode in which the motion vector of a subblock is extracted from the four parameters of the horizontal components and the vertical components of the motion vectors of two control points, and motion compensation is performed in units of subblocks.

[0160] In the present embodiment, the reference order, when a list of potentially suitable motion vector predictor options based on history is used, can be further configured to be a reverse order, which is the reverse of the storage order. Here, when the motion information is stored at the end side of the history-based motion vector predictor potential candidate list, and the motion information is sequentially stored such that the motion information at the head end of the history-based motion vector predictor potential candidate list is configured to be older than the motion information at its end side, the storage state is the same as the storage state of the present embodiment. In addition, when the motion information is sequentially stored such that the motion information in the leading side of the history-based motion vector predictor potential candidate list is configured to be newer than the motion information in its trailing side, the storage state is the inverse of the storage state, which is the reverse of the storage state shown in the present embodiment. In the case of the reverse storage state described above, the reference order is set as a storage order reversal, so that when the reference content is referred to as shown in the present embodiment, i.e. a list of potentially candidate motion vector predictors based on history can be referred to motion information in order from oldest motion information to newest added motion information. In addition, a configuration can be adopted in which the case in which the reference order is used as the order having the same direction as the storage order and the case in which the reference order is used as the order having the opposite direction to the storage order are different, when a list of potentially candidate candidate motion vector predictors based on history is used. Thus, a configuration can be adopted in which the case in which the motion information stored in the history-based motion vector predictor candidate candidate list can be referred to in order from the oldest motion information to the newest added motion information, and the case in which it is possible to refer to the motion information stored in the history-based motion vector predictor candidate list in order from the newest added motion information to the oldest motion information differ when the history-based motion vector predictor candidate list is used.

[0161] In addition, a configuration can be adopted in which the process of extracting an element having identical information from a list of potential candidate motion vector predictor candidates based on history, and deleting the extracted element is not performed when a new element is added to the list of potential candidate vector predictor candidates. movements based on prehistory.

[0162] Additionally, a configuration may be adopted in which the process of extracting an element having identical information from a list of potential motion vector predictor candidates based on history and preventing the element having identical information from being set as a normal motion vector predictor candidate candidate is not is performed when a candidate normal motion vector predictor candidate is derived using the history-based motion vector predictor candidate list HmvpCandList in the normal motion vector predictor identification mode.

[0163] A list of potentially eligible motion vector predictors based on history

Hereinafter, the configuration and operation of a history-based motion vector predictor candidate list is described.

[0164] As shown in FIG. 41, the inter prediction information used in inter prediction in the coding target block is set as an inter prediction information candidate candidate hMvpCand to be registered, and the inter prediction information is registered in the motion vector predictor candidate candidate list HmvpCandList based on backstory as the backstory used in the past. In FIG. 41, the history-based motion vector predictor candidate list has a list structure capable of storing six elements, and a first-in-first-out (FIFO) scheme in which elements are retrieved in order from the previously stored element is used as basic storage operation.

[0165] Here, although the following describes an example in which the maximum number of items that can be stored in the HmvpCandList is six as a negotiation between the encoding side and the decoding side, the maximum number is not specifically limited and may be six or more. In addition, the maximum number of items allowed to be stored in the HmvpCandList may be configured to be five or less. For example, the HmvpCandList may be configured with a maximum number of elements equivalent to the maximum number of elements of potentially suitable inter-prediction information cases, e.g. associations based on subblocks. In addition, the HmvpCandList may be configured in conjunction with the maximum number of elements of potentially suitable inter-prediction information options in each mode.

[0166] A configuration may be adopted in which the maximum number of HmvpCandList elements is included in the bitstream syntax element and thereby delivered from the encoding side to the decoding side.

[0167] As shown in FIG. 41, HmvpCandList can store six elements from position 0, which is the beginning of the list, to position 5, which is the end of the list, and can embed elements from position 0 to position 5. These positions 0-5 are managed as hMvpIdx indexes of predictor vectors. movements based on prehistory. For example, position 0 may be expressed as hMVpIdx[0], and position 5 may be expressed as hMVpIdx[5]. The number of stored elements of the HmvpCandList is controlled by NumHmvpCand, and the increase/decrease in the number of stored elements is controlled in the range of 0-6, which is the maximum number of elements.

[0168] The following describes in detail the case in which a new element is added from the end in a state in which the maximum number of elements is stored in the HmvpCandList. As shown in FIG. 42, when a potential candidate hMvpCand of the inter prediction information is intended to be newly registered as a history, the element at position 0, which is the start, is removed, and the positions of the respective elements are shifted one by one towards the start. As a result of the shift, the number of stored items is gradually reduced by one, as shown in FIG. 43, and because of this, a new element can be stored at end position 5. Therefore, a new element is added to the HmvpCandList as shown in FIG. 44 by storing a potentially suitable hMvpCand inter-prediction information at end position 5.

[0169] Modified example (part 1)

As a modified example of the first embodiment, a configuration is adopted in which a process is performed for a history-based list of potentially suitable motion vector predictors as shown below.

Retrieve identical potential matches in a list of potential candidates of motion vector predictors based on history

When the list HmvpCandList of potentially candidate history-based motion vector predictors is used in reference order, as shown in FIG. 45, an item (inter prediction information) is removed from the history-based motion vector predictor candidate list HmvpCandList when there is an inter prediction having a value identical to that of the inter prediction information candidate candidate hMvpCand to be registered in the inter prediction information. prediction registered in the HmvpCandList list of potentially suitable motion vector predictors based on history. The process of removing an identical candidate is hereinafter referred to as the "identical candidate removing process". A configuration is adopted in which the process of deleting identical candidates when an item is added to the HmvpCandList of potential candidates of motion vector predictors based on history is not performed. In addition, a configuration is adopted in which the process of deleting identical candidates is not performed when the history-based motion vector predictor candidate list HmvpCandList is used with reference in reverse order, which is the reverse of the storage order in which the new element is added. to the end of the list of potentially eligible motion vector predictors based on history, as shown in FIG. 46.

Fig. 38 is a flowchart for describing a processing procedure for retrieving potentially suitable motion vector predictor cases based on history, in the case of a configuration in which the reference is performed in reverse order, which is the reverse of the storage order in which motion information is added with end side, motion information is used that is a history-based motion vector predictor candidate stored in the history-based motion vector predictor candidate list HmvpCandList by reference in order from the oldest motion information to the newest added motion information, and a deletion process identical potentially suitable options are not met.

The inter prediction information used when inter prediction is performed in normal prediction vector mode or normal combining mode is set as inter prediction information candidates hMvpCand to be registered and inter prediction information candidates hMvpCand to be recorded. to be registered are added in order from the empty position closest to the start side, regardless of whether or not there is an inter prediction having a value identical to the value of a potentially suitable variant hMvpCand of the inter prediction information to be registered in the inter prediction information registered in a list HmvpCandList of potentially suitable motion vector predictors based on the history provided in the encoding information storage 111 and the encoding information storage 205 decoding rons. When HmvpCandList is filled until the maximum number of elements is reached, the element (inter-prediction information) at the head of the list HmvpCandList of candidate history-based motion vector predictor candidates is removed, and the candidate candidate inter-prediction information hMvpCand to be registered, is added at the end of the HmvpCandList list of potentially suitable motion vector predictors based on history. The number of elements of the list HmvpCandList of potentially suitable motion vector predictors based on history provided in the encoding-side encoding information storage 111 and in the decoding-side encoding information storage 205 of the present embodiment is assumed to be 6. First, the initial setting performed in slice units. The history-based motion vector predictor potential candidate list HmvpCandList is initialized at the beginning of the slice, and the value of the number NumHmvpCand of the history-based motion vector predictor potential candidates registered in the history-based motion vector predictor potential candidate list HmvpCandList is set to 0 (step S2401 according to Fig. 38).

Then, the process of updating the list HmvpCandList of potentially suitable motion vector predictor cases based on the history for each coding block in the slice is iteratively performed (steps S2402-S2407 of FIG. 38). It is determined whether or not there is a potential candidate hMvpCand of inter prediction information, which is the target of registering in the HmvpCandList of potential candidate motion vector predictors based on history (step S2404 in FIG. 38). When the encoding side prediction method determination unit 105 determines that the mode is the normal motion vector predictor identification mode or normal combining mode, or when the decoding side bit string decoding unit performs decoding in the normal motion vector predictor identification mode or normal combining mode, the inter -prediction is presumably hMvpCand. When the encoding side prediction method determination unit 105 determines that the mode is an intra prediction mode, a sub-block based motion vector predictor identification mode, or a sub-block combining mode, or when the decoding side bit string decoding unit performs decoding in the intra prediction mode , the sub-block based motion vector predictor identification mode, or the sub-block based combining mode, there is no potential candidate hMvpCand of the inter-prediction information that is the target of registration without performing an update process of the list HmvpCandList of potential candidate historical motion vector predictor predictors. When there is no potential candidate hMvpCand of the inter prediction information that is the registration target, step S2406 is skipped (step S2404 of FIG. 38: "No"). When there is a potential candidate hMvpCand of the inter prediction information that is the registration target, the processing from step S2406 is performed (step S2404 of FIG. 38: "Yes").

Then, a process of shifting and adding a list item HmvpCandList of potentially suitable motion vector predictor options based on the history is performed (step S2406 of FIG. 38).

When the process, as shown in FIG. 38 is performed, it is possible to omit the process of checking and deleting the identical potential candidate and the process of setting or comparing variables to execute the process and reduce the amount of processing.

In addition, even if the process of removing identical potential candidates is not performed on the history-based motion vector predictor potential candidate list, when an element is to be added to the history-based motion vector predictor candidate list HmvpCandList, it is possible to reduce the probability that the identical potential the candidate should be added to the list of potentially candidate motion vector predictor candidates by referring to the motion information in order from the oldest motion information to the newest added motion information.

[0170] Modified example (part 2)

As a modified example of the first embodiment, the procedure for processing the shift and adding a list item HmvpCandList of potentially suitable motion vector predictor options based on history in step S2406 of FIG. 38 has the following configuration.

Fig. 39 is a flowchart for the shift processing procedure and adding an item HmvpCandList of potentially suitable history-based motion vector predictor options in step S2406 of FIG. 38.

First, it is determined whether or not a new element should be added after the element at the beginning stored in the HmvpCandList list of potentially suitable motion vector predictors based on history is removed, or whether or not a new element should be added without deleting the element. Specifically, a comparison is made as to whether NumHmvpCand is equal to 6 or not (step S2541 of FIG. 39). When NumHmvpCand is equal to 6 (step S2541 of FIG. 39: "Yes"), a new element is to be added after the element at the beginning stored in the HmvpCandList of potentially suitable history-based motion vector predictor candidates is removed. An initial value of index i is set to a value of 1. The elementwise shift process of step S2543 is iteratively performed from this initial value to NumHmvpCand (steps S2542 to S2544 of FIG. 39). By copying the elements of HmvpCandList[i] to HmvpCandList[i-1], the elements are shifted forward (step S2543 of FIG. 39) and i is gradually increased by 1 (steps S2542 and S2544 of FIG. 39). Then, the inter-prediction information candidate hMvpCand is added to the (NumHmvpCand-1)th element of HmvpCandList[NumHmvpCand-1] counting from 0, which corresponds to the end of the history-based motion vector predictor candidate candidate list (step S2545 of FIG. 39), and the process of shifting and adding an element of the list HmvpCandList of potentially suitable motion vector predictors based on the history is completed. On the other hand, when NumHmvpCand is not 6 (step S2541 of FIG. 39: No), a new element is added without deleting an element stored in the list HmvpCandList of potentially suitable motion vector predictors based on history. The inter prediction information potential candidate hMvpCand is added to the (NumHmvpCand-1)th element of HmvpCandList[numHmvpCand] when counting from 0, which corresponds to the end of the history-based motion vector predictor potential candidate list, and NumHmvpCand is gradually increased by 1 (step S2546 in Fig. 39), and the process of shifting and adding an element of the list HmvpCandList of potentially suitable variants of motion vector predictors based on the history is completed.

By executing the process as shown in FIG. 39, it is possible to omit the process of checking and deleting the identical potential candidate and the process of setting or comparing variables to perform the process and reduce the amount of processing.

[0171] Modified example (part 3)

In the present embodiment, a case of a configuration in which the identical candidate candidate deletion process is not performed is described, the history-based motion vector predictor candidate list HmvpCandList is used with reference in reverse order, which is the reverse of the storage order in which the new motion information is added from the end side, and motion information is used, which is a potentially suitable history-based motion vector predictor stored in HmvpCandList by reference in order from oldest motion information to newly added motion information.

In a modified example of the first embodiment, a configuration is adopted in which a direct mode is provided in which the reference from the new added motion information to the old motion information is made in the direction in which the reference order is identical to the storage order in which the motion information is added from the end, and reverse mode in which reference from the oldest motion information to the newly added motion information is made in the direction in which the reference order is reversed with respect to the storage order in which new motion information is added from the end side, and the mode is switched between forward mode and reverse a mode using the history-based reference flag information to switch the reference order of the list HmvpCandList of potentially suitable history-based motion vector predictors. The history-based reference flag in the history-based reference flag information is included in the bitstream syntax element and transmitted to the decoding side, and the decoding side is configured to receive and decode the bitstream including the history-based reference flag into the syntax element.

[0172] Second Embodiment

As a second embodiment, the following configuration is adopted when the mvpListLX list of potentially suitable motion vector predictor options is retrieved. In particular, a configuration is adopted in which a process of determining whether or not an identical motion vector predictor is included in a motion vector predictor candidate candidate list is not performed in a process of adding a motion vector predictor candidate from a history-based motion vector predictor candidate candidate list. to a motion vector predictor candidate list using a history-based motion vector predictor candidate list HmvpCandList with reference to information from the beginning in reverse order, which is the reverse of the storage order in which information is added anew from its end when a list mvpListLX of potentially suitable motion vector predictor variants is retrieved.

[0173] Hereinafter, this document describes a process for extracting potentially suitable motion vector predictor variants based on the history of the present embodiment.

[0174] The process of extracting potentially candidate motion vector predictor candidates based on history, which does not involve the process of removing identical potential candidates

Hereinafter, a process for extracting potentially candidate motion vector predictor candidates based on history is described, which does not include a process for removing identical potential candidates.

[0175] FIG. 47 is a flowchart for describing a history-based motion vector predictor candidate extraction processing procedure in the case of a configuration in which the identical candidate candidate removal process is not performed using the history-based motion vector predictor potential candidate list HmvpCandList. by reference in reverse order, which is the reverse of the storage order when the mvpListLX list of potentially candidate motion vector predictors is retrieved.

[0176] When the current number numCurrMvpCand of potential candidate motion vector predictor candidates is greater than or equal to the maximum number of elements (here, 2) in the motion vector predictor list mvpListLX, or the value of the number NumHmvpCand of potential candidate motion vector predictor candidates based on history is 0 (step S2601 of Fig. 47: "No"), the processing of steps S2602-S2609 in Fig. 47 is omitted, and the history-based motion vector predictor candidate extraction process is terminated. When the current number numCurrMvpCand of potentially suitable motion vector predictor candidates is less than 2, which is the maximum number of elements of the motion vector predictor list mvpListLX (step S2601 of FIG. 47: Yes), the processing of steps S2602 to S2609 of FIG. 47 is being implemented.

[0177] Next, the processing of steps S2603-S2608 in FIG. 47 is iteratively executed until index i changes from 0 to the value obtained by subtracting 1 from the smaller value from 4 and the number NumHmvpCand of potentially suitable history-based motion vector predictor candidates (steps S2602-S2609 in FIG. 47). When the current number numCurrMvpCand of potentially suitable motion vector predictor candidates is greater than or equal to 2, which is the maximum number of elements of the motion vector predictor list mvpListLX (step S2603 of FIG. 47: "No"), the processing of step S2604-S2609 of FIG. 47 is omitted, and the history-based motion vector predictor candidate extraction process is terminated. When the current number of candidate motion vector predictor candidates numCurrMvpCand is less than 2, which is the maximum number of elements of the candidate motion vector predictor candidate list mvpListLX (step S2603 of FIG. 47: "Yes"), the processing from step S2604 of FIG. 47 is being implemented.

[0178] Next, the processing of steps S2605-S2607 is performed with respect to the Y index, which is 0 and 1 (L0 and L1) (steps S2604-S2608 of FIG. 47). When the current number numCurrMvpCand of potentially suitable motion vector predictor options is greater than or equal to 2, which is the maximum number of elements of the motion vector predictor list mvpListLX (step S2605 of FIG. 47: "No"), the processing of step S2606-S2609 of FIG. 47 is omitted, and the history-based motion vector predictor candidate extraction process is terminated. When the current number numCurrMvpCand of potentially suitable motion vector predictor candidates is less than 2, which is the maximum number of elements of the motion vector predictor list mvpListLX (step S2605 of FIG. 47: Yes), the processing from step S2606 of FIG. 47 is being implemented.

[0179] Then, when the reference index entry LY in the history-based motion vector predictor candidate list HmvpCandList[i] is a reference index entry that is identical to the encoding/decoding target motion vector reference index refIdxLX (step S2606 of FIG. 47 : "Yes"), the motion vector LY of the motion vector predictor candidate candidate HmvpCandList[i] based on the history is added to the numCurrMvpCand th element of mvpListLX[numCurrMvpCand] when counted from 0 for the motion vector predictor candidate candidate list (step S2607 in FIG. 47), and the current number numCurrMvpCand of candidate motion vector predictor candidates is incremented by 1. When the reference index entry LY in the history-based motion vector predictor candidate list HmvpCandList[i] is not a reference index entry that is identical to the reference index su refIdxLX of the motion vector of the encoding/decoding target (step S2606 of FIG. 47: "No"), the addition process of step S2607 is skipped.

[0180] The above processing of steps S2605-S2607 in FIG. 47 is executed in L0 and L1 (steps S2604-S2608 in FIG. 47).

[0181] When index i is gradually increased by 1 (steps S2602 and S2609 of FIG. 47), and index i is less than or equal to the value obtained by subtracting 1 from the smaller value from 4 and the number NumHmvpCand of potentially suitable motion vector predictor cases based on the history , the processing from step S2603 is performed again (steps S2602 to S2609 in FIG. 47).

It is possible to omit the process of checking and deleting an identical candidate and the process of setting or comparing variables to execute the process, and reduce the amount of processing by preventing the process of deleting identical candidates in the process of adding from the list of potential candidates of history-based motion vector predictors to the list. motion vector predictor candidate candidates by performing a history-based motion vector predictor candidate candidate extraction process as described above.

In addition, it is possible to add various types of motion information to the list of motion vector predictor candidates by enabling reference to motion information that is a history-based motion vector predictor candidate stored in the HmvpCandList in order from oldest motion information to new added traffic information.

In addition, even if the process of deleting identical candidates is not performed on the history-based motion vector predictor candidate list, in the process of performing adding from the history-based motion vector predictor candidates list to the motion vector predictor candidate list, it is possible to reduce a probability that an identical candidate is to be added to a list of motion vector predictor candidates by referring to motion information in order from oldest motion information to newly added motion information.

[0182] Modified example (part 1)

In the second embodiment, a case of a configuration is described in which the identical candidate candidate removal process is not performed using the history-based motion vector predictor candidate list HmvpCandList with reference from the beginning in reverse order, which is the reverse of the storage order, in in which the information is re-added from the end when the mvpListLX list of potentially eligible motion vector predictors is retrieved.

In the modified example of the second embodiment, as shown below, a configuration is adopted in which the process of deleting identical potential candidates is performed as is, so that the reference from the oldest motion information to the newly added motion information is performed by referring to the motion information from the beginning in reverse order. , which is the reverse of the storage order, in which motion information is re-added from the end only with respect to the reference order of the list HmvpCandList of potentially suitable motion vector predictors based on history.

[0183] FIG. 48 is a flowchart for describing a history-based motion vector predictor candidate extraction processing procedure in the case of a configuration in which the identical candidate candidate removal process is not performed using the history-based motion vector predictor potential candidate list HmvpCandList. reference in reverse order, which is the reverse of the storage order.

[0184] When the current number numCurrMvpCand of potential candidate motion vector predictor candidates is greater than or equal to the maximum number of elements (here, 2) in the motion vector predictor list mvpListLX, or the value of the number NumHmvpCand of potential candidate motion vector predictor candidates based on history is 0 (step S2801 to Fig. 48: "No"), the processing of steps S2802-S2809 in Fig. 48 is omitted, and the history-based motion vector predictor candidate extraction process is terminated. When the current number numCurrMvpCand of potentially suitable motion vector predictor candidates is less than 2, which is the maximum number of elements of the motion vector predictor list mvpListLX (step S2801 of FIG. 48: Yes), the processing of steps S2802 to S2809 of FIG. 48 is being implemented.

[0185] Next, the processing of steps S2803-S2808 in FIG. 48 is iteratively executed until index i changes from 0 to the value obtained by subtracting 1 from the smaller value from 4 and the number NumHmvpCand of potentially suitable history-based motion vector predictor candidates (steps S2802-S2809 in FIG. 48). When the current number of candidate motion vector predictor candidates numCurrMvpCand is greater than or equal to 2, which is the maximum number of elements of the candidate motion vector predictor candidate list mvpListLX (step S2803 of FIG. 48: "No"), the processing of steps S2804-S2809 of FIG. 48 is omitted, and the history-based motion vector predictor candidate extraction process is terminated. When the current number numCurrMvpCand of potentially suitable motion vector predictor candidates is less than 2, which is the maximum number of elements of the motion vector predictor list mvpListLX (step S2803 of FIG. 48: Yes), the processing from step S2804 of FIG. 48 is being implemented.

[0186] Next, the processing of steps S2805-S2807 is performed with respect to the Y index, which is 0 and 1 (L0 and L1) (steps S2804-S2808 of FIG. 48). When the current number of candidate motion vector predictor candidates numCurrMvpCand is greater than or equal to 2, which is the maximum number of elements of the candidate motion vector predictor candidate list mvpListLX (step S2805 of FIG. 48: "No"), the processing of steps S2806-S2809 of FIG. 48 is omitted, and the history-based motion vector predictor candidate extraction process is terminated. When the current number of candidate motion vector predictor candidates numCurrMvpCand is less than 2, which is the maximum number of elements of the candidate motion vector predictor candidate list mvpListLX (step S2805 of FIG. 48: "Yes"), the processing from step S2806 of FIG. 48 is being implemented.

[0187] Then, when the reference index LY element of the list HmvpCandList[i] of potentially suitable history-based motion vector predictors is an element that has a reference index that is identical to the encoding/decoding target motion vector reference index refIdxLX, and that is different from any element of the motion vector predictor index mvpListLX (step S2806 of FIG. 48: "Yes"), the motion vector LY of the historically applicable motion vector predictor candidate HmvpCandList[i] is added to the numCurrMvpCand th element of mvpListLX[numCurrMvpCand] when counted from 0 for the list of potential candidate motion vector predictor candidates (step S2807 of FIG. 48), and the current number of potential candidate motion vector predictor candidates numCurrMvpCand is gradually increased by 1. represented is an element that has a reference index identical to the encoding/decoding target motion vector reference index refIdxLX, and which is different from any element of the motion vector predictor index mvpListLX (step S2806 of FIG. 48: "No"), the addition process of step S2807 is skipped.

[0188] The above processing of steps S2805-S2807 in FIG. 48 is executed in L0 and L1 (steps S2804-S2808 in FIG. 48).

[0189] When the index i is gradually increased by 1 and the index i is less than or equal to the value obtained by subtracting 1 from the smaller value from 4 and the number NumHmvpCand of potentially suitable history-based motion vector predictor cases, the processing from step S2803 is performed again (steps S2802 -S2809 of Fig. 48).

The identical candidate candidate removal process is configured to be performed as is by performing the history-based motion vector predictor potential candidate extraction process as described above, and being able to refer to motion information that is the history-based motion vector predictor candidate candidate. stored in HmvpCandList, in order from oldest motion information to newly added motion information, so that more different types of motion information can be added to the list of potentially suitable motion vector predictor candidates.

[0190] Modified example (part 2)

As a modified example of the second embodiment, when the motion vector predictor candidate candidate list mvpListLX is retrieved, the procedure for processing shift and adding the history-based motion vector predictor candidate candidate list HmvpCandList is configured in step S2406 of FIG. 38 as shown in FIG. 39. Since the description of FIG. 39 is similar to the above description, the description of FIG. 39 is dropped.

By executing the process as shown in FIG. 39, it is possible to omit the process of checking and deleting the identical potential candidate and the process of setting or comparing variables to perform the process and reduce the amount of processing.

[0191] Modified example (part 3)

In the second embodiment, a case of a configuration in which a process of deleting identical candidates is not performed is described, a history-based motion vector predictor candidate list HmvpCandList is used with reference in reverse order, which is the reverse of the storage order in which the new motion information is added at the end side when the mvpListLX list of potentially candidate motion vector predictor candidates is retrieved, and motion information is used that is a historical motion vector predictor candidate stored in the HmvpCandList by reference in order from oldest motion information to newest added motion information. In a modified example of the second embodiment, a configuration is adopted in which a direct mode is provided in which the reference from the new added motion information to the old motion information is made in the direction in which the reference order is identical to the storage order in which the motion information is added from the end, and the reverse a mode in which reference from the oldest motion information to the newly added motion information is made in the direction in which the reference order is reverse of the storage order in which new motion information is added from the end side, and the mode is switched between forward mode and reverse mode using the history-based reference flag information to switch the reference order of the list HmvpCandList of potentially suitable history-based motion vector predictors. The history-based reference flag in the history-based reference flag information is included in the bitstream syntax element and transmitted to the decoding side, and the decoding side is configured to receive and decode the bitstream including the history-based reference flag into the syntax element.

[0192] Third embodiment

As a third embodiment, the following configuration is adapted when a list of potentially suitable join options is retrieved. In particular, the reference order, when the history-based motion vector predictor candidate list of potential candidates is used, is changed such that the history-based motion vector predictor candidate list is used with reference in order from the oldest motion information to the newly added motion information. In addition, when a new element is added to the list of candidate joins, the change is made such that the process of extracting an element having identical information from the list of potential candidates of motion vector predictors based on history and deleting the element is not performed. Further, in the normal join mode, when a candidate normal join is retrieved using the history-based motion vector predictor candidate list HmvpCandList, a change is made such that the process of extracting an element having identical information from the list of motion vector predictor candidates potentially based on history, and preventing an element having identical information from being set as a potential candidate for a normal join is not performed.

The process of extracting potentially matching candidates based on history, which does not involve the process of removing identical candidates

Hereinafter, a process for extracting potentially suitable merges based on history is described, which does not include an identical process for deleting a potentially suitable variant.

Fig. 40 is a flowchart for describing a history-based merge candidate extraction processing procedure in the case of a configuration in which the process of deleting identical candidates is not performed using the history-based motion vector predictor candidate list HmvpCandList with reference to motion information in order from the oldest motion information to the newly added motion information.

First, an initialization process is performed (step S2701 of FIG. 40). The number numCurrMergeCand of elements registered in the current merge candidate list is given as the numOrigMergeCand variable.

The hMvpIdx index initial value is set to 0, and the addition process of steps S2703 to S2710 in FIG. 40 is iteratively executed from this start value to NuMHmvpCand-1 (steps S2702-S2711 in FIG. 40). If the number of numCurrMergeCand elements registered in the current merge candidate list is not less than or equal to (MaxNumMergeCand maximum merge candidate list is 1), candidate merges are added to all elements of the merge candidate list, so that the current retrieval process is potentially matching options based on the history is completed ("No" at step S2703 of Fig. 40). "Not"). When the number numCurrMergeCand of the elements registered in the current merge candidate list is less than or equal (the maximum number of potential merge candidates MaxNumMergeCand is 1), the processing from step S2710 is executed (step S2703 of FIG. 40: Yes). The HMvpIdx th element HmvpCandList[hMvpIdx] of the list of potentially suitable motion vector predictors based on history is added to the numCurrMergeCand th element of mergeCandList[numCurrMergeCand] of the list of potentially suitable merge candidates, and numCurrMergeCand is incremented by 1 (step S2710 of FIG. 40). The hMvpIdx index is gradually increased by 1 (step S2702 of FIG. 40), and the iterative processing of steps S2702 to S2711 of FIG. 40 is being implemented.

When the confirmation of all elements of the history-based potential motion vector predictor candidate list is completed, or the candidate joins are added to all elements of the candidate join list, the current history-based matching candidate extraction process is terminated.

It is prevented from performing the process of deleting identical candidates in the process of performing addition from the history-based motion vector predictor candidates candidate list to the pooling candidate list by performing the history-based pooling potential candidates extraction process as described above, so that the process can be omitted. checking and removing an identical potentially suitable option; and the process of setting or comparing variables to perform the process and reduce the amount of processing.

In addition, it is possible to add various types of motion information to a list of potentially matching merges by allowing reference to motion information that is a history-based motion vector predictor candidate stored in HmvpCandList in order from oldest motion information to newest added. motion information.

In addition, even if the process of deleting identical candidates is not performed on the history-based motion vector predictor candidate candidate list in the process of performing adding from the history-based motion vector predictor candidate candidate list to the pooling candidate list, it is possible to reduce the probability that that the identical candidate candidate should be added to the list of candidate joins by referring to the motion information in order from the oldest motion information to the newest added motion information.

[0193] Modified example (part 1)

As a modified example of the third embodiment, a configuration is adopted in which the process of deleting identical candidates when an item is added to the HmvpCandList list of potential candidates of motion vector predictors based on history, as shown in FIG. 38 is not executed when a list of potentially matching merge options is retrieved. Since the description of FIG. 38 is similar to the above description, the description of FIG. 38 is dropped.

By executing the process as shown in FIG. 38, it is possible to omit the process of checking and removing the identical candidate and the process of setting or comparing variables to perform the process and reduce the amount of processing.

In addition, even if the process of removing identical potential candidates is not performed on the history-based motion vector predictor potential candidate list, when an element is to be added to the history-based motion vector predictor candidate list HmvpCandList, it is possible to reduce the probability that the identical potential the candidate must be added to the list of potentially candidate merges by allowing the motion information to be referenced in order from the oldest motion information to the newly added motion information.

[0194] Modified example (part 2)

As a modified example of the third embodiment, a procedure for processing the shift and adding the list item HmvpCandList of potentially suitable motion vector predictor options based on the history of step S2406 of FIG. 38 when a list of potentially suitable joins is retrieved as shown in FIG. 39. Since the description of FIG. 39 is similar to the above description, the description of FIG. 39 is dropped.

By executing the process as shown in FIG. 39, it is possible to omit the process of checking and deleting the identical potential candidate and the process of setting or comparing variables to perform the process and reduce the amount of processing.

[0195] Modified example (part 3)

In the third embodiment, a configuration case is described in which the identical candidate removal process is not performed using motion information that is a history-based motion vector predictor candidate stored in the HmvpCandList, with reference to the motion information in order from the most old motion information to new added motion information when a list of potentially suitable joins is retrieved.

In the modified example of the third embodiment, a configuration is adopted in which a forward mode in which a link from the new added motion information to the old motion information is made and a reverse mode in which a link from the oldest motion information to the new added motion information is provided are provided, and the mode is switched between the direct mode and the reverse mode using the history-based reference flag information to switch the reference order of the history-based list of potentially suitable motion vector predictors HmvpCandList. The history-based reference flag in the history-based reference flag information is included in the bitstream syntax element and transmitted to the decoding side, and the decoding side is configured to receive and decode the bitstream including the history-based reference flag into the syntax element.

[0196] Two or more of all the embodiments described above may be combined.

[0197] In all the embodiments described above, the bitstream output by the frame encoder has a specific data format such that the bitstream can be decoded according to the encoding method used in the embodiment. In addition, the frame decoding device corresponding to the frame encoding device can decode a bit stream of a specific data format.

[0198] When a wired or wireless network is used to exchange a bit stream between a frame encoder and a frame decoder, the bit stream can be converted to a data format suitable for the transmission form of the communication path and transmitted. In this case, a transmission device is provided for converting the bit stream output from the frame encoding device into coded data of a data format suitable for the transmission form of the communication path, and transmitting the coded data to the network, and a receiver for receiving the coded data from the network, recovering the coded data into a bitstream and providing the bitstream to a frame decoder. The transmission device includes a memory that buffers a bit stream output by the frame encoder, a packet processing module that packetizes the bit stream, and a transmission module that transmits packetized encoded data via a network. The receiving device includes a receiving module that receives packetized coded data via a network, a storage device that buffers the received coded data, and a packet processing module that generates a bit stream by performing packet processing on the coded data and provides the bit stream to a frame decoding device.

[0199] In addition, a display device can be provided by adding a display module that displays a frame decoded by the frame decoding device to a configuration. In this case, the display unit reads the decoded frame signal generated by the frame signal overlay unit 207 for decoding and stored in the decoded frame memory 208, and displays the decoded frame signal on a screen.

[0200] In addition, an imaging apparatus can be provided by adding an imaging module that inputs a captured frame to a frame encoding apparatus to a configuration. In this case, the imaging module inputs the captured frame signal to the block splitter 101 .

[0201] FIG. 37 shows an example of a hardware configuration of an encoder/decoder according to the present embodiment. The encoding/decoding apparatus includes a configuration of a frame encoding apparatus and a frame decoding apparatus according to an embodiment of the present invention. The associated encoding/decoding apparatus 9000 includes a CPU 9001, a codec IC 9002, an I/O interface 9003, a storage device 9004, an optical disc drive 9005, a network interface 9006, and a video interface 9009, and the respective parts are connected via a bus 9010.

[0202] Frame encoding module 9007 and frame decoding module 9008 are typically implemented as codec IC 9002. The frame encoding process of the frame encoding apparatus according to the embodiment of the present invention is performed by the frame encoding unit 9007, and the frame decoding process in the frame decoding apparatus according to the embodiment of the present invention is performed by the frame decoding unit 9008. The input/output interface 9003 is implemented by, for example, a USB interface and is connected to an external keyboard 9104, mouse 9105, and the like. The CPU 9001 controls the encoder/decoder 9000 such that the operation requested by the user is performed based on the user operation input via the I/O interface 9003. User operations using 9104 keyboard, 9105 mouse, etc. including selecting an encoding or decoding function to be performed, specifying an encoding quality, specifying an I/O destination of a bitstream, specifying an I/O destination of a frame, and the like.

[0203] When a user requires a playback operation of a frame recorded on the disc recording medium 9100, the optical disc drive 9005 reads the bitstream from the disc recording medium 9100 that is inserted, and transmits the read bitstream to the frame decoding module 9008 of the codec IC 9002 via the bus 9010 The frame decoding unit 9008 performs a frame decoding process for an input bitstream in a frame decoding apparatus according to an embodiment of the present invention, and transmits the decoded frame to an external monitor 9103 via a video interface 9009. The encoding/decoding apparatus 9000 includes a network interface 9006 and can be connected to an external the distribution server 9106 and the portable terminal 9107 via the network 9101. When the user wants to play back a frame recorded on the distribution server 9106 or the portable terminal 9107 instead of the frame recorded on the disc recording medium 9100, the network interface 9006 receives a bitstream from the network 9101 instead of reading the bitstream from the input disk recording medium 9100. When the user wants to play back a frame recorded in the memory 9004, a frame decoding process in the frame decoding apparatus according to an embodiment of the present invention is performed on the bitstream recorded in the memory 9004.

[0204] When the user wants to perform an operation of encoding a frame captured by the external camera 9102 and writing the encoded frame to the storage device 9004, the video interface 9009 inputs the frame from the camera 9102 and transmits the frame to the frame encoding module 9002 of the codec IC 9002 via the bus 9010. The encoding module 9007 frames performs a frame encoding process for inputting a frame through the video interface 9009 in the frame encoding device according to an embodiment of the present invention to create a bit stream. Then, the bitstream is transmitted to the storage device 9004 via the bus 9010. When the user wants to record the bitstream on the disk recording medium 9100 instead of the storage device 9004, the optical disc drive 9005 records the bitstream on the disk recording medium 9100 that is inserted.

[0205] It is also possible to implement a hardware configuration that includes a frame encoder without including a frame decoder, or a hardware configuration that includes a frame decoder without including a frame encoder. Such a hardware configuration is implemented, for example, by replacing the codec IC 9002 with a frame encoding module 9007 or a frame decoding module 9008.

[0206] The above processes related to encoding and decoding may be implemented as a data transmission, storage, and reception device using hardware, and implemented by firmware stored in read only memory (ROM), flash memory, or the like. or as computer software, etc. The firmware and firmware therefor may be provided by writing programs on a recording medium capable of being read by a computer or the like, or may be provided from a server via a wired or wireless network, or may be provided as data broadcasts according to terrestrial or satellite digital broadcast.

[0207] The present invention is described above based on the embodiments. Embodiments are examples, and those skilled in the art will appreciate that various modifications are possible in combinations of appropriate components and processing processes, and such modifications are within the scope of the present invention.

Explanation of reference numbers

[0208]

100 - frame encoder

101 - block breaking module

102 - inter-prediction module

103 - intra-prediction module

104 - decoded frame memory

105 - prediction method determination module

106 - residue formation module

107 - orthogonal transformation/quantization module

108 - bit string encoding module

109 - inverse quantization/inverse orthogonal transform module

110 - frame overlay module for decoding

111 - storage device for storing encoding information

200 - frame decoder

201 - bit string decoding module

202 - block breaking module

203 - inter-prediction module

204 - intra-prediction module

205 - storage device for storing encoding information

206 - inverse quantization/inverse orthogonal transform module

207 - frame overlay module for decoding

208 is a storage device for decoded frames.

Claims (16)

1. A moving frame encoding device, comprising: a spatial motion information candidate extractor, configured to extract a candidate spatial motion information from the motion information of a block adjacent to the coding target block in a spatial domain; and a history-based motion information candidate extraction module, configured to extract motion information candidates based on history from a memory storing encoded block motion information, wherein the historical motion information candidate extraction module preferably extracts the old motion information without comparing the motion information with the potential motion spatial information candidate. 2. A method for encoding moving frames, comprising: a first process in which a potentially suitable case of spatial motion information is extracted from the motion information of a block adjacent to the coding target block in a spatial domain; and a second process that extracts a potentially suitable case of motion information based on the history from a storage device that stores the motion information of the encoded block, wherein the second process includes the step of preferably extracting the old motion information without comparing the motion information with a potentially suitable variant of the spatial motion information. 3. A moving frame decoding device, comprising: a candidate spatial motion information candidate extractor, configured to extract a candidate spatial motion information from the motion information of a block adjacent to the decoding target block in a spatial domain; and a history-based motion information candidate extractor, configured to extract motion information candidates based on history from a memory storing motion information of the decoded block, wherein the historical motion information candidate extraction module preferably extracts the old motion information without comparing the motion information with the potential motion spatial information candidate. 4. A method for decoding moving frames, comprising: a first process in which a potentially suitable case of spatial motion information is extracted from the motion information of a block adjacent to the decoding target block in a spatial domain; and a second process that extracts a potentially suitable case of motion information based on the history from a storage device that stores the motion information of the decoded block, wherein the second process includes the step of preferably extracting the old motion information without comparing the motion information with a potentially suitable variant of the spatial motion information.

Images