US11438625B2 - Image decoding device, image encoding device, image processing system, and program - Google Patents

Image decoding device, image encoding device, image processing system, and program Download PDF

Info

Publication number
US11438625B2
US11438625B2 US16/981,744 US201916981744A US11438625B2 US 11438625 B2 US11438625 B2 US 11438625B2 US 201916981744 A US201916981744 A US 201916981744A US 11438625 B2 US11438625 B2 US 11438625B2
Authority
US
United States
Prior art keywords
block
prediction
candidate
reference position
candidate positions
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
US16/981,744
Other languages
English (en)
Other versions
US20210203981A1 (en
Inventor
Kyohei UNNO
Kei Kawamura
Sei Naito
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
KDDI Corp
Original Assignee
KDDI Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by KDDI Corp filed Critical KDDI Corp
Assigned to KDDI CORPORATION reassignment KDDI CORPORATION ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: NAITO, SEI, KAWAMURA, Kei, Unno, Kyohei
Publication of US20210203981A1 publication Critical patent/US20210203981A1/en
Application granted granted Critical
Publication of US11438625B2 publication Critical patent/US11438625B2/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation
    • H04N19/53—Multi-resolution motion estimation; Hierarchical motion estimation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/103—Selection of coding mode or of prediction mode
    • H04N19/105—Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation
    • H04N19/513—Processing of motion vectors
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation
    • H04N19/513—Processing of motion vectors
    • H04N19/517—Processing of motion vectors by encoding
    • H04N19/52—Processing of motion vectors by encoding by predictive encoding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation
    • H04N19/57—Motion estimation characterised by a search window with variable size or shape
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/136—Incoming video signal characteristics or properties
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/182—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a pixel
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation
    • H04N19/533—Motion estimation using multistep search, e.g. 2D-log search or one-at-a-time search [OTS]

Definitions

  • the present invention relates to an image decoding device, an image encoding device, an image processing system, and a program.
  • HEVC High Efficiency Video Coding
  • DMVR Decoder-side Motion Vector Refinement
  • the image decoding device specifies a reference position based on the motion vector, and specifies a corrected reference position having the smallest predetermined cost from a search range set based on the specified reference position.
  • the image decoding device corrects the motion vector based on the corrected reference position (for example, Algorithm Description of Joint Exploration Test Model 7 (JEM 7), JVET-G1001).
  • an object of the present invention is to provide an image decoding device, an image encoding device, an image processing system, and a program capable of reducing a processing load associated with correction of a motion vector without lowering correction accuracy of the motion vector.
  • An image decoding device including: a prediction unit configured to generate a prediction signal included in a prediction block based on a motion vector, wherein the prediction unit is configured to perform refinement processing of setting a search range based on a reference position specified by the motion vector, specifying a corrected reference position having the smallest predetermined cost from the search range, and correcting the motion vector based on the corrected reference position; and when a block size of the prediction block is larger than a predetermined block size, the prediction unit is configured to divide the prediction block into sub-block groups and perform the refinement processing for each sub-block.
  • An image encoding device including: a prediction unit configured to generate a prediction signal included in a prediction block based on a motion vector, wherein the prediction unit is configured to perform refinement processing of setting a search range based on a reference position specified by the motion vector, specifying a corrected reference position having the smallest predetermined cost from the search range, and correcting the motion vector based on the corrected reference position; and when a block size of the prediction block is larger than a predetermined block size, the prediction unit is configured to divide the prediction block into sub-block groups and performs the refinement processing for each sub-block.
  • An image processing system including an image encoding device and an image decoding device, wherein the image encoding device and the image decoding device include a prediction unit configured to generate a prediction signal included in a prediction block based on a motion vector; the prediction unit is configured to perform refinement processing of setting a search range based on a reference position specified by the motion vector, specifying a corrected reference position having the smallest predetermined cost from the search range, and correcting the motion vector based on the corrected reference position; and when a block size of the prediction block is larger than a predetermined block size, the prediction unit divides the prediction block into sub-block groups and performs the refinement processing for each sub-block.
  • a program that causes a computer to perform: a predicting step of generating a prediction signal based on a motion vector, wherein the predicting step includes a step of performing refinement processing of setting a search range based on a reference position specified by the motion vector, specifying a corrected reference position having the smallest predetermined cost from the search range, and correcting the motion vector based on the corrected reference position; and in the step, when a block size of the prediction block is larger than a predetermined block size, the prediction block is divided into sub-block groups and the refinement processing is performed for each sub-block.
  • an image decoding device an image encoding device, an image processing system, and a program capable of reducing a processing load associated with correction of a motion vector without lowering correction accuracy of the motion vector.
  • FIG. 1 is a diagram illustrating an image processing system 10 according to an embodiment.
  • FIG. 2 is a diagram illustrating an image encoding device 100 according to an embodiment.
  • FIG. 3 is a diagram illustrating an inter prediction unit 111 according to an embodiment.
  • FIG. 4 is a diagram for describing first processing according to an embodiment.
  • FIG. 5 is a diagram for describing second processing according to an embodiment.
  • FIG. 6 is a diagram for describing third processing according to an embodiment.
  • FIG. 7 is a diagram for describing the third processing according to the embodiment.
  • FIG. 8 is a diagram illustrating an image decoding device 200 according to an embodiment.
  • FIG. 9 is a diagram illustrating an inter prediction unit 241 according to an embodiment.
  • FIG. 10 is a diagram for describing third processing according to Modified Example 1.
  • FIG. 11 is a diagram for describing third processing according to Modified Example 2.
  • FIG. 12 is a diagram for describing a method of specifying a target search range according to Modified Example 3.
  • drawings are schematic and ratios of dimensions may be different from actual ones. Therefore, specific dimensions and the like should be determined in consideration of the following description.
  • drawings may include portions having different dimensional relationships or ratios.
  • An image decoding device includes a prediction unit that generates a prediction signal included in a prediction block based on a motion vector.
  • the prediction unit performs refinement processing of setting a search range based on a reference position specified by the motion vector, specifying a corrected reference position having the smallest predetermined cost from the search range, and correcting the motion vector based on the corrected reference position.
  • the prediction unit performs first processing of calculating the predetermined cost for four first candidate positions that define an outer frame of a target search range constituting at least a part of the search range and the reference position, and second processing of specifying two second candidate positions that define an outer frame of a divided search range in which the target search range is divided and calculating the predetermined cost for the two second candidate positions, based on a criterion first candidate position and the reference position having the smallest predetermined cost of the four first candidate positions.
  • the first processing of calculating the predetermined cost for the four first candidate positions that define the outer frame of the target search range is performed, and the second processing of specifying the two second candidate positions that define the outer frame of the divided search range in which the target search range is divided and calculating the predetermined cost for the two second candidate positions is performed. That is, by searching for the corrected reference position from outside the target search range, the corrected reference position can be appropriately searched while suppressing the number of pixels for which the predetermined cost is to be calculated. In other words, a processing load associated with correction of the motion vector can be reduced without lowering a correction accuracy of the motion vector.
  • the image encoding device includes a prediction unit that generates a prediction signal included in a prediction block based on a motion vector.
  • the prediction unit performs refinement processing of setting a search range based on a reference position specified by the motion vector, specifying a corrected reference position having the smallest predetermined cost from the search range, and correcting the motion vector based on the corrected reference position.
  • the prediction unit performs first processing of calculating the predetermined cost for four first candidate positions that define an outer frame of a target search range constituting at least a part of the search range and the reference position, and second processing of specifying two second candidate positions that define an outer frame of a divided search range in which the target search range is divided and calculating the predetermined cost for the two second candidate positions, based on a criterion first candidate position and the reference position having the smallest predetermined cost of the four first candidate positions.
  • the first processing of calculating the predetermined cost for the four first candidate positions that define the outer frame of the target search range is performed, and the second processing of specifying the two second candidate positions that define the outer frame of the divided search range in which the target search range is divided and calculating the predetermined cost for the two second candidate positions is performed. That is, by searching for the corrected reference position from outside the target search range, the corrected reference position can be appropriately searched while suppressing the number of pixels for which the predetermined cost is to be calculated. In other words, a processing load associated with correction of the motion vector can be reduced without lowering a correction accuracy of the motion vector.
  • an image decoding method according to the operation of the image decoding device described above may be provided, and an image encoding method according to the operation of the image encoding device described above may be provided.
  • an image processing system including the image decoding device and the image encoding device described above may be provided.
  • a program according to the operation of the image decoding device described above may be provided, or a program according to the operation of the image encoding device described above may be provided.
  • FIG. 1 is a diagram illustrating an image processing system 10 according to an embodiment.
  • an image processing system 10 includes an image encoding device 100 and an image decoding device 200 .
  • the image encoding device 100 generates encoded data by encoding an input image signal.
  • the image decoding device 200 generates an output image signal by decoding the encoded data.
  • the encoded data may be transmitted from the image encoding device 100 to the image decoding device 200 via a transmission path.
  • the encoded data may be provided from the image encoding device 100 to the image decoding device 200 after being stored in a storage medium.
  • FIG. 2 is a diagram illustrating the image encoding device 100 according to the embodiment.
  • the image encoding device 100 includes an inter prediction unit 111 , an intra prediction unit 112 , a subtractor 121 , an adder 122 , a transform/quantization unit 131 , an inverse transform/inverse quantization unit 132 , an encoding unit 140 , an in-loop filter processing unit 150 , and a frame buffer 160 .
  • the inter prediction unit 111 generates a prediction signal by inter prediction (inter-frame prediction). Specifically, the inter prediction unit 111 specifies a reference block included in a reference frame by comparing a frame to be encoded (hereinafter, a target frame) with a reference frame stored in the frame buffer 160 , and determines a motion vector for the specified reference block. The inter prediction unit 111 generates a prediction signal included in a prediction block for each prediction block based on the reference block and the motion vector. The inter prediction unit 111 outputs the prediction signal to the subtractor 121 and the adder 122 .
  • the reference frame is a frame different from the target frame.
  • the intra prediction unit 112 generates a prediction signal by intra prediction (intra-frame prediction). Specifically, the intra prediction unit 112 specifies a reference block included in a target frame, and generates a prediction signal for each prediction block based on the specified reference block. The intra prediction unit 112 outputs the prediction signal to the subtractor 121 and the adder 122 .
  • the reference block is a block that is referred to for a block to be predicted (hereinafter, a target block). For example, the reference block is a block adjacent to the target block.
  • the subtractor 121 subtracts the prediction signal from an input image signal, and outputs a prediction residual signal to the transform/quantization unit 131 .
  • the subtractor 121 generates a prediction residual signal that is a difference between the prediction signal generated by intra prediction or inter prediction and the input image signal.
  • the adder 122 adds the prediction signal to the prediction residual signal output from the inverse transform/inverse quantization unit 132 , and outputs a decoded signal before filtering to the intra prediction unit 112 and the in-loop filter processing unit 150 .
  • the decoded signal before filtering constitutes the reference block used in the intra prediction unit 112 .
  • the transform/quantization unit 131 performs processing of transforming the prediction residual signal, and acquires a coefficient level value. Further, the transform/quantization unit 131 may perform quantization of the coefficient level value.
  • the transform processing is processing of transforming the prediction residual signal into a frequency component signal. In the transform processing, a base pattern (transform matrix) corresponding to a discrete cosine transform (DCT) may be used, and a base pattern (transform matrix) corresponding to a discrete sine transform (DST) may be used.
  • DCT discrete cosine transform
  • DST discrete sine transform
  • the inverse transform/inverse quantization unit 132 performs inverse transform processing of the coefficient level value output from the transform/quantization unit 131 .
  • the inverse transform/inverse quantization unit 132 may perform inverse quantization of the coefficient level value prior to the inverse transform processing.
  • the inverse transform processing and the inverse quantization are performed in a procedure reverse to the transform processing and the quantization performed by the transform/quantization unit 131 .
  • the encoding unit 140 encodes the coefficient level value output from the transform/quantization unit 131 , and outputs encoded data.
  • encoding is entropy encoding that assigns codes of different lengths based on a probability of occurrence of the coefficient level value.
  • the encoding unit 140 encodes control data used in the decoding processing, in addition to the coefficient level value.
  • the control data may include size data such as an encoded block size, a predicted block size, and a transformed block size.
  • the in-loop filter processing unit 150 performs filter processing on the decoded signal before filtering output from the adder 122 , and outputs a decoded signal after filtering to the frame buffer 160 .
  • the filtering processing is deblocking filtering processing that reduces distortion generated at a boundary between blocks (encoded block, predicted block, or transformed block).
  • the frame buffer 160 accumulates reference frames used by the inter prediction unit 111 .
  • the decoded signal after filtering constitutes the reference frame used by the inter prediction unit 111 .
  • FIG. 3 is a diagram illustrating the inter prediction unit 111 according to the embodiment.
  • the inter prediction unit 111 includes a motion vector search unit 111 A, a refinement unit 111 B, and a prediction signal generation unit 111 C.
  • the inter prediction unit 111 is an example of a prediction unit that generates a prediction signal included in a prediction block based on a motion vector.
  • the motion vector search unit 111 A specifies a reference block included in the reference frame by comparing the target frame with the reference frame, and searches for the motion vector for the specified reference block. Since a known method can be adopted as a method of searching for the motion vector, details thereof are omitted.
  • the refinement unit 111 B performs refinement processing of setting a search range based on a reference position specified by the motion vector, specifying a corrected reference position having the smallest predetermined cost from the search range, and correcting the motion vector based on the corrected reference position.
  • the refinement unit 111 B may execute the refinement processing when a predetermined condition is satisfied.
  • the predetermined condition may include a condition that the prediction block is a block that performs bidirectional prediction, one reference frame is a frame that is temporally earlier than the target frame, and the other reference frame is a frame that is temporally future than the target frame.
  • the predetermined condition may include a condition that a block size of the prediction block is smaller than a predetermined size.
  • the block size may be the number of pixels of the prediction block in a horizontal direction or the number of pixels of the prediction block in a vertical direction.
  • the block size may be the smaller number of pixels in the horizontal direction and the vertical direction.
  • the block size may be the total number of pixels in the block (that is, a value obtained by multiplying the number of pixels in the horizontal direction and the number of pixels in the vertical direction).
  • the predetermined condition may include a condition that the motion vector is encoded in a merge mode.
  • the merge mode is a mode in which only an index of the motion vector of the encoded block adjacent to the prediction block is transmitted.
  • the predetermined condition may include a condition that the motion vector is not applied with motion compensation prediction using an affine transformation.
  • the refinement unit 111 B performs refinement processing in the following procedure.
  • a case where displacements of motion vectors in a past direction and a future direction are linked to each other in the refinement processing will be exemplified.
  • a sign of the displacement in the past direction is opposite to the sign of the displacement in the future direction.
  • the displacement in the past direction is ( ⁇ 2,2)
  • the displacement in the future direction is (2, ⁇ 2).
  • the predetermined cost is the sum of absolute values of differences between values of pixels included in the reference block (displaced) in the past direction and values of pixels included in the reference block (displaced) in the future direction.
  • the difference is a difference between the values of the pixels existing at the positions corresponding to each other. Note that this is only an example, and if the index (value) can be calculated from the value of the pixel included in the reference block in the past direction and the value of the pixel included in the reference block in the future direction, such as the sum of squares of the difference, an index other than the above can be used as the predetermined cost.
  • the target search range is the same as the search range and the target search range is a range of integer pixels of ⁇ 2 in the horizontal direction and the vertical direction will be exemplified.
  • the target search range has a square shape.
  • the refinement unit 111 B performs first processing of calculating a predetermined cost for four first candidate positions defining an outer frame (for example, four corners) of the target search range that constitutes at least a part of the search range, and a reference position.
  • the refinement unit 111 B specifies the reference position (P0 in FIG. 4 ) based on the motion vector.
  • the refinement unit 111 B sets the target search range based on the reference position.
  • the refinement unit 111 B calculates the predetermined cost for the four first candidate positions (P1 to P4 in FIG. 4 ) defining the outer frame of the target search range, and the reference position (P0 in FIG. 4 ).
  • the refinement unit 111 B performs second processing of specifying two second candidate positions defining an outer frame (for example, four corners) of a divided search range in which the target search range is divided based on a criterion first candidate position having the smallest predetermined cost of the four first candidate positions and the reference position, and calculating a predetermined cost for the two second candidate positions.
  • the refinement unit 111 B specifies the divided search range in which the target search range is divided based on the criterion first candidate position (P2 in FIG. 5 ) and the reference position (P0 in FIG. 5 ).
  • the divided target region has a square shape having a straight line from P0 to P2 as a diagonal line.
  • the refinement unit 111 B specifies two second candidate positions (P5 and P6 in FIG. 5 ) defining the outer frame of the divided search range, and calculates a predetermined cost for the two second candidate positions.
  • the refinement unit 111 B specifies a position having the smallest predetermined cost among the calculated candidate positions (P0, P2, P5, and P6) as a corrected reference position, and corrects (displaces) the motion vector based on the corrected reference position.
  • the refinement unit 111 B continues the following processing.
  • the refinement unit 111 B performs third processing of specifying a third candidate position based on a criterion second candidate position having a small predetermined cost of the four calculated candidate positions defining the outer frame (for example, four corners) of the divided search range, and calculating a predetermined cost for the third candidate position.
  • a case will be described in which two criterion second candidate positions are selected as the criterion second candidate position, and pixels arranged on a straight line sandwiched between the two criterion second candidate positions are specified as the third candidate position.
  • the refinement unit 111 B specifies pixels (P7, P8, and P9 in FIG. 6 ) arranged on a straight line sandwiched between the two criterion second candidate positions as the third candidate positions.
  • the refinement unit 111 B calculates a predetermined cost for the third candidate position (P7, P8, and P9 in FIG. 6 ).
  • P7 and P9 are 1 ⁇ 2 pixels and have values interpolated by integer pixels. That is, the third candidate position includes a 1 ⁇ 2 pixel.
  • the refinement unit 111 B specifies pixels (P7, P8, and P9 in FIG. 7 ) arranged on a straight line sandwiched between the two criterion second candidate positions as the third candidate positions.
  • the refinement unit 111 B calculates a predetermined cost for the third candidate position (P7, P8, and P9 in FIG. 7 ).
  • P7 and P9 are 1 ⁇ 2 pixels and have values interpolated by integer pixels. That is, the third candidate position includes a 1 ⁇ 2 pixel.
  • the refinement unit 111 B specifies a position having the smallest predetermined cost among the calculated candidate positions (P0, P2, P5 to P9) as a corrected reference position, and corrects (displaces) the motion vector based on the corrected reference position.
  • the prediction signal generation unit 111 C generates a prediction signal based on the motion vector. Specifically, when the motion vector is not corrected, the prediction signal generation unit 111 C generates the prediction signal based on the motion vector input from the motion vector search unit 111 A. On the other hand, when the motion vector is corrected, the prediction signal generation unit 111 C generates the prediction signal based on the corrected motion vector input from the refinement unit 111 B.
  • FIG. 8 is a diagram illustrating the image decoding device 200 according to the embodiment.
  • the image decoding device 200 includes a decoding unit 210 , an inverse transform/inverse quantization unit 220 , an adder 230 , an inter prediction unit 241 , an intra prediction unit 242 , an in-loop filter processing unit 250 , and a frame buffer 260 .
  • the decoding unit 210 decodes the encoded data generated by the image encoding device 100 , and decodes the coefficient level value.
  • the decoding is entropy decoding in a procedure reverse to the entropy encoding performed by the encoding unit 140 .
  • the decoding unit 210 may acquire control data by decoding the encoded data.
  • the control data may include size data such as an encoded block size, a predicted block size, and a transformed block size.
  • the control data may include an information element indicating an input source used to generate a predicted sample of a second component.
  • the inverse transform/inverse quantization unit 220 performs inverse transform processing of the coefficient level value output from the decoding unit 210 .
  • the inverse transform/inverse quantization unit 220 may perform inverse quantization of the coefficient level value prior to the inverse transform processing.
  • the inverse transform processing and the inverse quantization are performed in a procedure reverse to the transform processing and the quantization performed by the transform/quantization unit 131 .
  • the adder 230 adds the prediction signal to the prediction residual signal output from the inverse transform/inverse quantization unit 220 , and outputs a decoded signal before filtering to the intra prediction unit 262 and the in-loop filter processing unit 250 .
  • the decoded signal before filtering constitutes the reference block used in the intra prediction unit 262 .
  • the inter prediction unit 241 generates a prediction signal by inter prediction (inter-frame prediction), similarly to the inter prediction unit 111 . Specifically, the inter prediction unit 241 generates the prediction signal for each prediction block based on the motion vector decoded from the encoded data and the reference signal included in the reference frame. The inter prediction unit 241 outputs the prediction signal to the adder 230 .
  • the intra prediction unit 262 generates a prediction signal by intra prediction (intra-frame prediction), similarly to the intra prediction unit 112 . Specifically, the intra prediction unit 262 specifies a reference block included in a target frame, and generates the prediction signal for each prediction block based on the specified reference block. The intra prediction unit 262 outputs the prediction signal to the adder 230 .
  • the in-loop filter processing unit 250 performs filter processing on the decoded signal before filtering output from the adder 230 , and outputs a decoded signal after filtering to the frame buffer 260 , similarly to the in-loop filter processing unit 150 .
  • the filtering processing is deblocking filtering processing that reduces distortion generated at a boundary between blocks (encoded block, predicted block, or transformed block).
  • the frame buffer 260 accumulates reference frames used by the inter prediction unit 241 , similarly to the frame buffer 160 .
  • the decoded signal after filtering constitutes the reference frame used by the inter prediction unit 241 .
  • FIG. 9 is a diagram illustrating the inter prediction unit 241 according to the embodiment.
  • the inter prediction unit 241 includes a motion vector decoding unit 241 A, a refinement unit 241 B, and a prediction signal generation unit 241 C.
  • the inter prediction unit 241 is an example of a prediction unit that generates a prediction signal included in a prediction block based on a motion vector.
  • the motion vector decoding unit 241 A acquires the motion vector by decoding the control data received from the image encoding device 100 .
  • the refinement unit 241 B performs refinement processing of setting a search range based on a reference position specified by the motion vector, specifying a corrected reference position having the smallest predetermined cost from the search range, and correcting the motion vector based on the corrected reference position, similarly to the refinement unit 111 B.
  • the prediction signal generation unit 241 C generates a prediction signal based on the motion vector, similarly to the prediction signal generation unit 111 C.
  • the first processing of calculating the predetermined cost for the four first candidate positions that define the outer frame of the target search range is performed, and the second processing of specifying the two second candidate positions that define the outer frame of the divided search range in which the target search range is divided and calculating the predetermined cost for the two second candidate positions is performed. That is, by searching for the corrected reference position from outside the target search range, the corrected reference position can be appropriately searched while suppressing the number of pixels for which the predetermined cost is to be calculated. In other words, a processing load associated with correction of the motion vector can be reduced without lowering a correction accuracy of the motion vector.
  • the third processing of specifying a third candidate position based on a criterion second candidate position having a small predetermined cost of the four calculated candidate positions defining the outer frame (for example, four corners) of the divided search range, and calculating a predetermined cost for the third candidate position is performed.
  • the third candidate position is a pixel arranged on a straight line sandwiched between two criterion second candidate positions. That is, by calculating the predetermined cost only at the position where the predetermined cost is assumed to be small, the corrected reference position can be appropriately searched while suppressing the number of pixels for which the predetermined cost is to be calculated.
  • the embodiment has exemplified the case in which the two criterion second candidate positions are selected as the criterion second candidate position as a result of the second processing.
  • a case will be described in which one criterion second candidate position is selected as the criterion second candidate position as a result of the second processing.
  • the refinement unit (refinement unit 111 B and refinement unit 241 B) specifies a pixel adjacent to the reference position in the divided search range as the third candidate position when selecting the reference position as the criterion second candidate position in the third processing.
  • the refinement unit specifies pixels (P7, P8, and P9 in FIG. 10 ) adjacent to the reference position in the divided search range as the third candidate position.
  • P7 to P9 are 1 ⁇ 2 pixels and have values interpolated by integer pixels. That is, the third candidate position includes a 1 ⁇ 2 pixel.
  • FIG. 10 illustrates a case where the third candidate position is a 1 ⁇ 2 pixel, but Modified Example 1 is not limited thereto.
  • the third candidate position adjacent to the reference position in the divided search range may be an integer pixel.
  • the predetermined cost is calculated also for the 1 ⁇ 2 pixels (for example, P7 and P9 illustrated in FIG. 6 ) in the third processing.
  • the refinement unit does not calculate the predetermined cost for the 1 ⁇ 2 pixels, but calculates the predetermined cost only for the integer pixels.
  • the cost of the 1 ⁇ 2 pixel may not be calculated, and the predetermined cost of the integer pixel (P7 in FIG. 11 ) may be calculated, on the straight line from P0 to P2.
  • the cost of the 1 ⁇ 2 pixel may not be calculated, and the predetermined cost of the integer pixel (P7 in FIG. 11 ) may be calculated, on the straight line from P5 to P6.
  • the embodiment has exemplified the case in which the search range is the same as the target search range.
  • the search range for example, a range of ⁇ 4 integer pixels in the horizontal direction and the vertical direction
  • the refinement unit performs processing of specifying the target search range from the search range in the refinement processing. Such processing is performed before the above-described first processing.
  • the refinement unit may divide the search range into target search ranges A to D and perform the same processing as in the embodiment for each target search range.
  • the refinement unit may specify the target search range having the smallest predetermined cost of the target search ranges A to D, and perform the same processing as in the embodiment for the specified target search range.
  • the predetermined cost of the target search range may also be a predetermined cost of a representative point (for example, a center point (P4, P5, P8, and P9) or four corners (P1, P3, P10, and P12)) of each target search range.
  • the predetermined cost of the target search range may also be robust to the predetermined costs of two or more extraction points (for example, P0, P1, P2, P4, and P6 in the case of the target search range A) of each target search range.
  • the above-described embodiment can be applied to a case in which the search range is larger than the target search range.
  • the above-described embodiment can be applied even when the search range is an integer pixel range of ⁇ 4 or more in the horizontal direction and the vertical direction.
  • the prediction block When the block size of the prediction block is larger than a predetermined block size, the prediction block may be divided into sub-block groups having a size smaller than the predetermined size, and the refinement processing of the motion vector may be performed for each sub-block.
  • a square As a dividing method, a square may be divided into four parts, and the other may be divided into two parts.
  • the target search range is a range of ⁇ 2 integer pixels in the horizontal direction and the vertical direction.
  • the target search range may be a range of ⁇ 1 integer pixels in the horizontal and vertical directions.
  • the first processing and the second processing calculate the predetermined costs of all the integer pixels included in the divided search range.
  • the third processing a predetermined cost of one 1 ⁇ 2 pixel selected among the five 1 ⁇ 2 pixel candidates is calculated.
  • the target search range may be a range of ⁇ 3 integer pixels in the horizontal direction and the vertical direction.
  • the embodiment has been described in which the displacements of the motion vector in the past direction and the future direction are linked to each other in the refinement processing.
  • the embodiment is not limited thereto.
  • the displacements of the motion vector in the past direction and the future direction may not be linked to each other.
  • a program that causes a computer to execute each processing performed by the image encoding device 100 and the image decoding device 200 may be provided.
  • the program may be recorded on a computer readable medium. If a computer-readable medium is used, the program can be installed on the computer.
  • the computer-readable medium on which the program is recorded may be a non-transitory recording medium.
  • the non-transitory recording medium is not particularly limited, but may be a recording medium such as a CD-ROM or a DVD-ROM.
  • a chip configured by a memory that stores a program for executing each processing performed by the image encoding device 100 and the image decoding device 200 and a processor that executes the program stored in the memory may be provided.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
US16/981,744 2018-09-21 2019-08-29 Image decoding device, image encoding device, image processing system, and program Active US11438625B2 (en)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
JP2018178101A JP6976916B2 (ja) 2018-09-21 2018-09-21 画像復号装置、画像符号化装置、画像処理システム及びプログラム
JPJP2018-178101 2018-09-21
JP2018-178101 2018-09-21
PCT/JP2019/034012 WO2020059459A1 (ja) 2018-09-21 2019-08-29 画像復号装置、画像符号化装置、画像処理システム及びプログラム

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2019/034012 A-371-Of-International WO2020059459A1 (ja) 2018-09-21 2019-08-29 画像復号装置、画像符号化装置、画像処理システム及びプログラム

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/860,485 Continuation US11627336B2 (en) 2018-09-21 2022-07-08 Image decoding device, image encoding device, image processing system, and program

Publications (2)

Publication Number Publication Date
US20210203981A1 US20210203981A1 (en) 2021-07-01
US11438625B2 true US11438625B2 (en) 2022-09-06

Family

ID=69888732

Family Applications (4)

Application Number Title Priority Date Filing Date
US16/981,744 Active US11438625B2 (en) 2018-09-21 2019-08-29 Image decoding device, image encoding device, image processing system, and program
US17/860,485 Active US11627336B2 (en) 2018-09-21 2022-07-08 Image decoding device, image encoding device, image processing system, and program
US18/179,900 Active US11882308B2 (en) 2018-09-21 2023-03-07 Image decoding device, image encoding device, image processing system, and program
US18/541,515 Pending US20240114165A1 (en) 2018-09-21 2023-12-15 Image decoding device, image encoding device, image processing system, and program

Family Applications After (3)

Application Number Title Priority Date Filing Date
US17/860,485 Active US11627336B2 (en) 2018-09-21 2022-07-08 Image decoding device, image encoding device, image processing system, and program
US18/179,900 Active US11882308B2 (en) 2018-09-21 2023-03-07 Image decoding device, image encoding device, image processing system, and program
US18/541,515 Pending US20240114165A1 (en) 2018-09-21 2023-12-15 Image decoding device, image encoding device, image processing system, and program

Country Status (5)

Country Link
US (4) US11438625B2 (ja)
EP (1) EP3855736A4 (ja)
JP (1) JP6976916B2 (ja)
CN (1) CN111837386B (ja)
WO (1) WO2020059459A1 (ja)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112954331B (zh) * 2019-03-11 2022-07-29 杭州海康威视数字技术股份有限公司 一种编解码方法、装置及其设备

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040252766A1 (en) 2003-06-11 2004-12-16 Daeyang Foundation (Sejong University) Motion vector search method and apparatus
KR20040106202A (ko) 2003-06-11 2004-12-17 학교법인 대양학원 움직임 벡터 탐색 방법 및 그 장치
US20160286232A1 (en) * 2015-03-27 2016-09-29 Qualcomm Incorporated Deriving motion information for sub-blocks in video coding
WO2018110180A1 (ja) 2016-12-15 2018-06-21 シャープ株式会社 動きベクトル生成装置、予測画像生成装置、動画像復号装置、および動画像符号化装置
EP3343924A1 (en) 2017-01-03 2018-07-04 Thomson Licensing Method and apparatus for encoding and decoding motion information
EP3547687A2 (en) 2016-11-28 2019-10-02 Electronics and Telecommunications Research Institute Image encoding/decoding method and device, and recording medium having bitstream stored thereon

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2081386A1 (en) * 2008-01-18 2009-07-22 Panasonic Corporation High precision edge prediction for intracoding
US8094714B2 (en) * 2008-07-16 2012-01-10 Sony Corporation Speculative start point selection for motion estimation iterative search
CN101699865B (zh) * 2009-10-28 2011-06-01 山东大学 一种块运动自适应的亚像素快速搜索方法
CA2788952A1 (en) * 2010-02-09 2011-08-18 Nippon Telegraph And Telephone Corporation Motion vector predictive encoding method, motion vector predictive decoding method, moving picture encoding apparatus, moving picture decoding apparatus, and programs thereof
US9967558B1 (en) * 2013-12-17 2018-05-08 Google Llc Adaptive motion search control for variable block size partitions in video coding
CN111971966A (zh) * 2018-03-30 2020-11-20 韩国电子通信研究院 图像编码/解码方法和设备以及存储比特流的记录介质

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040252766A1 (en) 2003-06-11 2004-12-16 Daeyang Foundation (Sejong University) Motion vector search method and apparatus
KR20040106202A (ko) 2003-06-11 2004-12-17 학교법인 대양학원 움직임 벡터 탐색 방법 및 그 장치
US20160286232A1 (en) * 2015-03-27 2016-09-29 Qualcomm Incorporated Deriving motion information for sub-blocks in video coding
EP3547687A2 (en) 2016-11-28 2019-10-02 Electronics and Telecommunications Research Institute Image encoding/decoding method and device, and recording medium having bitstream stored thereon
WO2018110180A1 (ja) 2016-12-15 2018-06-21 シャープ株式会社 動きベクトル生成装置、予測画像生成装置、動画像復号装置、および動画像符号化装置
EP3343924A1 (en) 2017-01-03 2018-07-04 Thomson Licensing Method and apparatus for encoding and decoding motion information

Non-Patent Citations (10)

* Cited by examiner, † Cited by third party
Title
Chen, J., et al. "Algorithm Description of Joint Exploration Test Model 7 (JEM 7)" Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 50 pages submitted (Jul. 13-21, 2017).
Extended European Search Report for European Patent Application No. 19862930.5 dated Jun. 30, 2022.
International Search Report issued in corresponding International Patent Application No. PCT/JP2019/034012 dated Nov. 19, 2019, with English translation.
ITU-T H.265 v5 "Series H: Autiovisual and Multimedia Systems Infrastructure of audiovisual services—Coding of moving video High Efficiency Video Coding" International Telecommunication Union , 692 pages (Feb. 2018).
Liu, H., et al., "CE9-related: Simplification of Decoder Side Motion Vector Derivation" Joint Video Experts Team (JVET) of ITU0T SG 16 WP 3 and ISO/EDC JTC 1/SC 29/WG 11, 4 pages (Jul. 10-18, 2018).
S CHEN et al., "Further Improvements to HMKTA-1.0," Qualcomm Incorporated, ITU—Telecommunications Standardization Sector (Jun. 19-26, 2015); pp. 1-8.K0041-vLJVET-K0041-vl.zip JVET-K0041-vl.docx.
Sethuraman et al., "Decoder Side MV Refinement/Derivation with CTB-Level Concurrency and Other Normative Complexity Reduction Techniques," Ittiam Systems Pvt. Ltd., Joint Video Experts Team (Jul. 10-18, 2018); pp. 1-8.
Ueda, M., "Bi-prediction by single motion vector using decoder-side inter-reference ME" Joint Collaborative Team on Video-Coding (JCT-VC) of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11, 22 pages (Mar. 16-23, 2011).
Unno, K., et al., "CE9:-related : DMVR with Coarse-to-Fine Search and Block Size Limit" Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 5 pages (Oct. 3-12, 2018).
Zhou, M., et al., "Non-CE9: A computational complexity analysis for DMVR" Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 5 pages (Jul. 10-18, 2018).

Also Published As

Publication number Publication date
CN111837386A (zh) 2020-10-27
EP3855736A1 (en) 2021-07-28
EP3855736A4 (en) 2022-08-03
US20230209086A1 (en) 2023-06-29
JP6976916B2 (ja) 2021-12-08
US20220345740A1 (en) 2022-10-27
US11882308B2 (en) 2024-01-23
WO2020059459A1 (ja) 2020-03-26
US11627336B2 (en) 2023-04-11
CN111837386B (zh) 2024-09-20
US20240114165A1 (en) 2024-04-04
US20210203981A1 (en) 2021-07-01
JP2020053723A (ja) 2020-04-02

Similar Documents

Publication Publication Date Title
JP5277257B2 (ja) 動画像復号化方法および動画像符号化方法
US20160073107A1 (en) Method and apparatus for video encoding/decoding using intra prediction
JP5216710B2 (ja) 復号化処理方法
US11627311B2 (en) Image decoding device, image decoding method, and program
US20240114165A1 (en) Image decoding device, image encoding device, image processing system, and program
JP2024029090A (ja) 画像復号装置、画像符号化装置、画像処理システム及びプログラム
US8223832B2 (en) Resolution increasing apparatus
JP7026286B1 (ja) 画像復号装置、画像符号化装置、画像処理システム及びプログラム
CN115280778A (zh) 帧间预测方法、编码器、解码器以及存储介质
JP6914462B2 (ja) 画像復号装置、画像復号方法及びプログラム
JP5281597B2 (ja) 動きベクトル予測方法，動きベクトル予測装置および動きベクトル予測プログラム
JP2020053725A (ja) 予測画像補正装置、画像符号化装置、画像復号装置、及びプログラム
CN118827973A (zh) 图像解码装置、图像编码装置、图像处理系统和程序产品
JP7034363B2 (ja) 画像復号装置、画像復号方法及びプログラム
US20220312024A1 (en) Image decoding device, image decoding method, and program
WO2020184262A1 (ja) 画像復号装置、画像復号方法及びプログラム
WO2019187396A1 (ja) 画像復号装置、画像符号化装置、画像処理システム、画像復号方法及びプログラム
JP2021118525A (ja) 符号化装置、復号装置、及びプログラム

Legal Events

Date Code Title Description
AS Assignment

Owner name: KDDI CORPORATION, JAPAN

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:UNNO, KYOHEI;KAWAMURA, KEI;NAITO, SEI;SIGNING DATES FROM 20200821 TO 20200824;REEL/FRAME:053799/0725

FEPP Fee payment procedure

Free format text: ENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: BIG.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

STPP Information on status: patent application and granting procedure in general

Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER

STPP Information on status: patent application and granting procedure in general

Free format text: FINAL REJECTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION

STPP Information on status: patent application and granting procedure in general

Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS

STPP Information on status: patent application and granting procedure in general

Free format text: AWAITING TC RESP., ISSUE FEE NOT PAID

STPP Information on status: patent application and granting procedure in general

Free format text: PUBLICATIONS -- ISSUE FEE PAYMENT VERIFIED

STCF Information on status: patent grant

Free format text: PATENTED CASE