WO2018041005A1 - 运动补偿预测方法和设备 - Google Patents
运动补偿预测方法和设备 Download PDFInfo
- Publication number
- WO2018041005A1 WO2018041005A1 PCT/CN2017/098894 CN2017098894W WO2018041005A1 WO 2018041005 A1 WO2018041005 A1 WO 2018041005A1 CN 2017098894 W CN2017098894 W CN 2017098894W WO 2018041005 A1 WO2018041005 A1 WO 2018041005A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- pixel
- image
- polyhedron
- point
- reference pixel
- Prior art date
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
- H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
- H04N19/51—Motion estimation or motion compensation
Definitions
- the present invention relates to the field of image processing and, more particularly, to a method and apparatus for motion compensated prediction.
- the spherical image When processing a spherical panoramic image (hereinafter referred to as a spherical image), the spherical image is generally first projected into a two-dimensional planar panoramic image of a polyhedral format (hereinafter referred to as a two-dimensional image), and then the two-dimensional image of the polyhedral format is edited. Decoding operation.
- the position of the reference pixel point of the current pixel point is first determined, and then the pixel value of the current pixel point is predicted according to the pixel value of the reference pixel point.
- the pixel value of the current pixel point is predicted according to the pixel value of the position of the reference pixel point regardless of whether the reference pixel point is at the face where the current pixel point is located.
- the reference pixel When the reference pixel is not in the plane where the current pixel is located, since the faces of the two-dimensional image in the polyhedral format are not in the same projection plane, the adjacent regions of the faces are deformed, resulting in a position according to the position of the reference pixel at this time.
- the pixel value is less effective in predicting the pixel value of the current pixel.
- the invention provides a motion compensation prediction method and a motion prediction compensation device to improve the motion compensation prediction effect.
- a motion compensation prediction method comprising: determining a position of an initial reference pixel of a current pixel in a reference image, the current pixel being located within a first sub-image of the current image Determining a target reference of the current pixel point according to a position of the initial reference pixel point when the initial reference pixel point is outside a second sub-image of the reference image corresponding to the first sub-image position a position of a pixel in the reference image, wherein a line connecting the position of the target reference pixel at a surface of the polyhedron corresponding to the reference image and a position of the initial reference pixel at a first plane passes a center point of the polyhedron, the position of the target reference pixel at the surface of the polyhedron is determined according to a position of the initial reference pixel point and layout information of the polyhedron corresponding to the reference image, the first plane a plane in which the face corresponding to the second sub-image in the polyhedron is located; according to the
- the present invention finds a target reference pixel point that can actually serve as a reference according to the position of the initial reference pixel point, and then predicts the pixel value of the current pixel point according to the target reference pixel point, thereby improving the motion compensation prediction. The accuracy.
- the second sub-image is a sub-image in the reference image corresponding to the first sub-image in the current image
- the current image and the reference image are all polyhedrons are regular hexahedron
- the current image and the reference image The format of the current image is 4 ⁇ 3.
- the first sub-image of the current image corresponds to the Bottom plane of the regular hexahedron corresponding to the current image
- the second sub-image of the reference image also corresponds to the regular hexahedron corresponding to the reference image. Bottom face.
- the reference image corresponding polyhedron may be disposed on the reference image, wherein the anchoring surface of the polyhedron is the surface of the region of the second sub-image in the reference image, and the anchoring surface is the surface as the reference when the polyhedron is expanded, for example, in the unfolding
- the Bottom face of the polyhedron is placed on a plane, and then the other faces in the polyhedron are folded onto the plane.
- the position of the Bottom face on the plane is constant, and the other faces of the polyhedron The face folds into other areas of the plane, and Bottom is the anchoring face of the polyhedron.
- the method further includes: determining, according to the layout information of the reference image and the location of the initial reference pixel, whether the initial reference pixel is located Outside of the second sub-image in the reference image.
- the pixel value of the current pixel can be directly predicted according to the initial reference pixel when the initial reference pixel is not within the second sub-image, It is no longer necessary to determine the target reference pixel based on the initial reference pixel.
- the determining, according to a location of the initial reference pixel point, a target reference pixel point of the current pixel point Positioning in the reference image includes: determining a position of the target reference pixel on a surface of the polyhedron according to a position of the initial reference pixel point and layout information of the reference image, wherein the target reference pixel a position of a surface of the polyhedron at an intersection of a line connecting the initial reference pixel point and a center point of the polyhedron with a surface of the polyhedron; according to the target reference pixel point on a surface of the polyhedron A location and a layout information of the reference image determine a location of the target reference pixel in the reference image.
- determining, according to the location of the initial reference pixel point and the layout information of the reference image, the target reference pixel point Determining a position of the surface of the polyhedron comprising: determining a position of the initial reference pixel in the first plane according to a position of the initial reference pixel point and layout information of the reference image; according to the initial reference pixel point A position of the target reference pixel at a surface of the polyhedron is determined at a position of the first plane and layout information of the reference image.
- the layout information includes the quantity information of the face of the polyhedron And at least one of arrangement information of the sub-images of the reference image, arrangement order information of the sub-images of the reference image, and rotation information of the sub-images of the reference image.
- the pixel value according to the target reference pixel point And determining a predicted value of the pixel value of the current pixel point, including: a pixel value of the target reference pixel point and a proximity of the target reference pixel point
- the pixel value of the pixel is subjected to weighting processing; the pixel value of the position at which the target reference pixel is processed by the weighting is determined as a predicted value of the pixel value of the current pixel.
- the pixel value according to the target reference pixel point and And a pixel value of a neighboring pixel of the target reference pixel determining a predicted value of the pixel value of the current pixel, including a pixel according to a neighboring pixel of the target reference pixel
- the value is subjected to an interpolation operation at the target reference pixel position; the pixel value obtained by the interpolation operation is determined as a predicted value of the pixel value of the current pixel point.
- the current image and the reference image are both two-dimensional images.
- determining whether the initial reference pixel point is located outside the second sub-image in the reference image according to the layout information of the reference image and the position of the initial reference pixel point comprises: Determining, according to a location of the initial reference pixel point and an area where the second sub image of the reference image is located, whether the initial reference pixel point is located outside the second sub image in the reference image, wherein The area in which the second sub-image of the reference image is located is determined based on the layout information of the reference image.
- the polyhedron corresponding to the two-dimensional image is a regular polyhedron.
- the positive polyhedron may include a regular tetrahedron, a regular hexahedron, a regular octahedron, a regular dodecahedron, and an icosahedron.
- the initial reference pixel when the initial reference pixel is located in a second sub-image of the reference image corresponding to the first sub-image, determining, according to the pixel value of the initial reference pixel The predicted value of the pixel value of the current pixel.
- the pixel value of the current pixel may be directly predicted according to the pixel value of the initial reference pixel. Only when the initial reference pixel is not located in the second sub-image (at this time, since the adjacent region of the sub-image in the two-dimensional image is deformed, the pixel value of the current pixel cannot be directly predicted according to the initial reference pixel. ), it is necessary to find the target reference pixel according to the position of the initial reference pixel.
- a motion compensation prediction method comprising: determining a position of an initial reference pixel of a current reference pixel in a reference image, the current pixel being located at a first sub-image of the current image Determining that the current pixel point is in an extended region of the second sub-image when the initial reference pixel is located outside a second sub-image of the reference image corresponding to the first sub-image a target reference pixel position, wherein the extended region of the second sub-image is located outside the second sub-image, the extended region includes a plurality of pixel points, and any one of the first pixels in the extended region
- the pixel value is determined according to a pixel value of a second pixel point in the reference image, the position of the second pixel point on a surface of the polyhedron formed by the reference image and the first pixel point being in a first plane a line connecting the position of the center point of the polyhedron, the position of the second pixel point on the face of the polyhedron is according to
- the present invention directly searches for the target reference pixel point that can be used as a reference directly in the extended area according to the current pixel point and the motion information of the current pixel point, and then performs pixel value of the current pixel point according to the target reference pixel point.
- the prediction can quickly find the target reference pixel and improve the accuracy of the motion prediction estimation.
- the extended area of the second sub-image described above may be determined in advance according to the method in the above first aspect.
- the method further includes: determining, according to a location of the first pixel point and layout information of the reference image, that the second pixel is located a position of a surface of the polyhedron at a position of a surface of the polyhedron at an intersection of a line connecting the first pixel point and a center point of the polyhedron and a surface of the polyhedron; Determining a position of the second pixel at a position of the surface of the polyhedron and layout information of the reference image to determine a position of the second pixel in the reference image.
- the method further includes: determining, according to the layout information of the reference image and the location of the initial reference pixel point, the initial Whether the reference pixel is located outside of the second sub-image in the reference image.
- a motion compensation prediction method comprising: determining an initial reference pixel position of a current pixel in a reference image, the current pixel being located in a first sub-image of the current image; When the initial reference pixel is located outside the second sub-image of the reference image corresponding to the first sub-image, determining the target reference pixel of the current pixel according to the initial reference pixel position a position of a surface of the polyhedron corresponding to the reference image, wherein a line connecting the position of the target reference pixel at the surface of the polyhedron and the position of the initial reference pixel at the first plane passes through a center point of the polyhedron
- the first plane is a plane in which the plane corresponding to the second sub-image of the polyhedron is located; and the target reference pixel is determined in the reference image according to a position of the target reference pixel at a surface of the polyhedron a location according to a pixel value of the target reference pixel in the reference image and/or an adjacent
- the present invention finds a target reference pixel point that can actually serve as a reference according to the position of the initial reference pixel point, and then predicts the pixel value of the current pixel point according to the target reference pixel point, thereby improving the motion compensation prediction. The accuracy.
- the determining, according to the initial reference pixel position, a location of a target reference pixel of the current pixel point on a surface of a polyhedron corresponding to the reference image includes: determining, according to a location of the initial reference pixel point and layout information of the reference image, a position of the initial reference pixel in the first plane; according to the initial reference pixel point in the first plane And a position information of the reference image to determine a position of the target reference pixel on a surface of the polyhedron.
- the determining, according to the location of the target reference pixel on the surface of the polyhedron, the target reference pixel is Determining a position in the reference image, comprising: determining a position of the target reference pixel in the reference image according to a position of the initial reference pixel and layout information of the reference image, where the target reference pixel is The position of the surface of the polyhedron is at the intersection of the line connecting the initial reference pixel point and the center point of the polyhedron with the surface of the polyhedron.
- a motion compensated prediction apparatus comprising means for performing the method of the first aspect.
- a motion compensated prediction apparatus comprising means for performing the method of the second aspect.
- a motion compensated prediction apparatus comprising means for performing the method of the third aspect.
- a codec including a nonvolatile storage medium, and a central processing unit, the nonvolatile storage medium storing an executable program, the central processing unit and the nonvolatile The storage medium is connected, and the executable program is executed to implement the motion compensation prediction method and the extended content thereof as provided by the first aspect of the present invention.
- a codec including a nonvolatile storage medium, and a central processing unit, the nonvolatile storage medium storing an executable program, the central processing unit and the nonvolatile The storage medium is connected, and the executable program is executed to implement the motion compensation prediction method and the extended content thereof as provided by the second aspect of the present invention.
- a codec including a nonvolatile storage medium, and a central processing unit, the nonvolatile memory
- the storage medium stores an executable program
- the central processor is coupled to the non-volatile storage medium, and executes the executable program to implement the motion compensation prediction method and the extended content thereof provided by the third aspect of the present invention .
- a computer readable medium storing program code for execution by an image processing apparatus, the program code comprising instructions for performing the method of the first aspect.
- a computer readable medium storing program code for execution by an image processing apparatus, the program code comprising instructions for performing the method of the second aspect.
- a computer readable medium storing program code for execution by an image processing apparatus, the program code comprising instructions for performing the method of the third aspect.
- the target reference pixel point that can actually serve as a reference is found according to the position of the initial reference pixel point, and then the current pixel point is determined according to the target reference pixel point.
- the pixel values are predicted to improve the accuracy of the motion compensation prediction.
- Figure 1 is a latitude and longitude view of a spherical image.
- Figure 2 is a two-dimensional image of a polyhedral format.
- Figure 3 is a schematic illustration of a spherical image projected onto a regular hexahedron.
- FIG. 4 is a schematic diagram of a two-dimensional image obtained by developing a regular hexahedron and a regular hexahedron.
- FIG. 5 is a schematic flowchart of a motion compensation prediction method according to an embodiment of the present invention.
- Fig. 6 is a schematic diagram of a current image and a reference image.
- Figure 7 is a schematic diagram of reference images of different layout formats.
- FIG. 8 is a schematic diagram of a current image and a reference image.
- Fig. 9 is a schematic diagram of a regular hexahedron corresponding to a reference image.
- Fig. 10 is a schematic diagram of a regular hexahedron corresponding to a reference image.
- Figure 11 is a schematic illustration of other polyhedrons corresponding to the reference image.
- Figure 12 is a reference image of different layout formats.
- Figure 13 is a schematic illustration of adjacent pixel points around a target reference pixel point.
- FIG. 14 is a schematic flowchart of a motion compensation prediction method according to an embodiment of the present invention.
- Figure 15 is a schematic illustration of an extended area.
- FIG. 16 is a schematic flowchart of a motion compensation prediction method according to an embodiment of the present invention.
- Figure 17 is a schematic block diagram of a motion compensation prediction apparatus according to an embodiment of the present invention.
- Figure 18 is a schematic block diagram of a motion compensation prediction apparatus according to an embodiment of the present invention.
- Figure 19 is a schematic block diagram of a motion compensation prediction apparatus according to an embodiment of the present invention.
- 20 is a schematic flow chart of a decoding process of a panoramic video decoder.
- 21 is a schematic flow chart of an encoding process of a panoramic video decoder.
- Figure 22 is a schematic block diagram of an image encoder in accordance with an embodiment of the present invention.
- VR video images In order to support video image content presentation in all directions, VR video images usually contain 360-degree omnidirectional visual information in three dimensions, up and down, and the VR video image can be imagined as a map of the globe from the inner center of a globe.
- VR video images panoramic video images (which may be referred to simply as spherical images).
- the spherical image cannot be conveniently represented, stored, and indexed, before the spherical image is processed, the spherical image is usually developed to obtain a two-dimensional image, and then the two-dimensional image is compressed, processed, stored, transmitted, and the like. Among them, the process of expanding a spherical image to obtain a two-dimensional image is called projection.
- the common two-dimensional image is called the latitude and longitude image.
- the image adjacent to the north and south pole regions is stretched to a large extent, and there is serious distortion and data redundancy.
- the spherical image can be projected into a regular polyhedron to convert the spherical image into a two-dimensional image in a polyhedral format.
- the spherical image can be projected to a regular tetrahedron (Fig. 2(a)), a regular hexahedron (Fig. 2(b)), a regular octahedron (Fig. 2(c)), a regular dodecahedron (Fig. 2).
- an icosahedron (Fig. 2(e)) and a two-dimensional planar image obtained by projecting a spherical image into each of the above polyhedrons is sequentially shown in Fig. 2(f) - Fig. 2(k).
- the specific process of projecting a spherical image into a polyhedron is: placing a spherical image in a polyhedron to make it a polyhedral inscribed ball; connecting a spherical or polyhedral body to a point on the spherical surface and extending it compared to a polyhedron,
- the pixel at the position at the intersection of the polyhedron is the pixel of the corresponding point on the spherical image.
- the spherical surface is inscribed in the regular hexahedron ABCDEFGH, and the spherical center O and M' are connected in order to obtain the pixel value at the M' point on the regular hexahedron.
- the pixel at the M point is the pixel at the M' point.
- all the pixels in the ABCD region on the plane A'B'C'D' can be obtained in the same way, wherein the pixels in the ABCD region form an ABCD face, and the plane A'B'C'D' is The projection plane of the ABCD surface.
- each of the planar images on the surface of the polyhedron becomes an image of one region or a sub-image of the spherical image in the two-dimensional image.
- the surface of the regular hexahedron in Fig. 4(a) is expanded into the image of Fig. 4(b), and the surface image of the top surface on the surface of the hexahedron will become the sub-image in the upper left corner of Fig. 4(b).
- the image is a surface image of the top surface of the spherical image, and the top surface in the spherical image refers to the area covered by the surface image of the Top surface.
- the Top face is called the face of the pixel.
- the face and the sub-image in the embodiment of the present invention correspond to each other.
- the Bottom face is a face of a certain two-dimensional image
- the image in the Bottom face is the first sub-image
- the Bottom face is the first sub-image.
- the corresponding sub-image, the first sub-image is a sub-image corresponding to the Bottom surface.
- each of the small rectangular areas is a face of the two-dimensional image
- the image composed of the pixels in each rectangular area is a sub-image of the two-dimensional image, that is, the face is The concept of a region
- the sub-image is an image.
- the image shown in FIG. 2(f) to FIG. 2(k) may be directly processed, or the smallest area rectangle surrounded by the image may be selected.
- the image in the area is used as a processing target, and the area other than the face containing the two-dimensional image in the rectangular area is filled with the default content or the like, for example, all gray, all black, or all White and so on.
- the image when encoding and decoding an image, the image is often divided into a plurality of image blocks of equal size, and then a reference block is found for each image block, and there may be different reference cases in the process of finding a reference block for the current image block. According to the reference direction, it can be divided into one-way prediction and two-way prediction.
- Unidirectional prediction means that there is a reference image set in the current block (the elements in the set are selected reference images in the reconstructed image), and the coding block can select any one of the reference images in the set.
- Bidirectional prediction means that there are two reference image sets in the current block (the elements of the two sets are respectively independently selected from the reconstructed image, the reference images in the two sets may be partially or completely the same), and the coding block may be from two Each of the sets selects a reference image.
- the method of constructing the bidirectional or unidirectional and reference image sets is commonly agreed by the codec, or the encoding end transmits the used method to the decoding end, and the decoding end determines the used method according to the decoding result.
- the bidirectional prediction method there will be two reference blocks for the current coding block, each of which requires an indication of motion information.
- the decoding end needs to determine two reference blocks according to the two sets of motion information decoded.
- the predicted value of the pixel value of the pixel in the current block is determined according to the pixel value of the pixel within the two reference blocks.
- FIG. 5 is a schematic flowchart of a motion compensation prediction method according to an embodiment of the present invention. The method includes:
- the above current image and reference image are two-dimensional images of a polyhedral format obtained by spherical image conversion.
- the current image when the current image is subjected to prediction processing, the current image may be divided into a plurality of image blocks, and then each image block is processed, and the current pixel point may be a pixel point in an image block in the current image. . Specifically, the current pixel to be processed may be located in a certain image block within the first sub-image.
- the position of the initial reference pixel in the reference image may be determined according to the position of the current pixel point and the motion information of the current pixel point obtained by decoding the motion information code stream.
- the reference image is determined based on the reference image indication information in the motion information
- the position of the initial reference pixel point in the reference image is determined according to the motion vector information in the motion information and the position of the current pixel point.
- the position of the current reference pixel is the initial reference pixel position.
- the polyhedron corresponding to the current image and the reference image is a regular hexahedron
- FIG. 6(a) is the current image (the current pixel is on the current image)
- FIG. 6(b) is the reference image (the initial reference pixel is on the reference image)
- the current image and the reference image are each composed of six sub-images, namely Top, Front, Right, Rear, Left, and Bottom.
- the sub-image here can be thought of as an array of pixels that are in the same projection plane when the spherical surface is projected onto the polyhedron.
- each face of the polyhedron becomes a part of the two-dimensional image.
- the current pixel is P
- P is within the Bottom sub-image in the current image
- T is within the Front sub-image in the reference image, that is, T is not in the Bottom sub-image in the reference image.
- the first sub-image is a Bottom sub-image
- the second sub-image is a Bottom sub-image
- the first sub-image and the second sub-image are current images and A sub-image at the corresponding position in the reference image.
- determining a position of the target reference pixel of the current pixel in the reference image according to the position of the initial reference pixel, wherein, the connection of the target reference pixel to the position of the surface of the polyhedron corresponding to the reference image and the position of the initial reference pixel at the first plane pass through multiple faces
- the center point of the body, the position of the target reference pixel on the surface of the polyhedron is determined according to the position of the initial reference pixel point and the layout information of the polyhedron corresponding to the reference image, the first plane being corresponding to the second sub-image in the polyhedron The plane where the face is located.
- the current pixel may be predicted according to the initial reference pixel directly, instead of finding the target reference pixel. point.
- the polyhedron corresponding to the reference image may be a polyhedron composed of the reference image, that is, a polyhedron formed by folding each sub-image of the reference image according to a certain rule.
- the method further includes: determining, according to the layout information of the reference image and the position of the initial reference pixel, whether the initial reference pixel is located outside the second sub-image in the reference image.
- the position of the initial reference pixel in the reference image may be determined based on the position of the current pixel and the motion information of the current pixel obtained by decoding the motion information bitstream.
- the decoder may determine the reference image based on the reference image indication information in the motion information, and determine the position of the initial reference pixel in the reference image based on the motion vector information in the motion information and the position of the current pixel.
- the position of the current reference pixel is the position of the initial reference pixel.
- the layout information of the polyhedron corresponding to the reference image includes the information about the number of faces of the polyhedron corresponding to the reference image, the arrangement information of the sub-images of the reference image, the arrangement order information of the sub-images of the reference image, and the reference At least one of rotation information of a sub-image of the image.
- the number information of the faces of the polyhedron corresponding to the reference image may specifically be a polyhedron corresponding to the reference image.
- the number information of the faces of the polyhedron may indicate that the corresponding image of the reference image is a regular hexahedron.
- the arrangement mode information of the sub-images of the reference image refers to the arrangement manner of each sub-image in the reference image.
- the reference image corresponds to a regular hexahedron, and the reference image includes 6 sub-images, and the 6 sub-images
- the arrangement may be 4 ⁇ 3 type (Fig. 7(b)), 3 ⁇ 2 type (Fig. 7(c), Fig. 7(d)), 6 ⁇ 1 type (Fig. 7(e), Fig. 7(f) ).
- the arrangement order information of the sub-images of the reference image refers to the arrangement order of the respective sub-images of the reference image in the reference image.
- FIG. 7(c) and FIG. 7(d) are images of the 3 ⁇ 2 type, in the figure.
- the sub-images corresponding to the Front face are arranged in the lower left corner of the image
- the sub-images corresponding to the Front face are arranged in the middle of the first column.
- the rotation information of the sub-image of the reference image may refer to the rotation angle of the sub-image of the reference image, assuming that the placement position of each sub-image in FIG. 7(c) is used as a reference, then in FIG. 7(d), the Front surface corresponds to The sub image has a rotation angle of -90 degrees.
- the target reference pixel point that can actually serve as a reference is found according to the position of the initial reference pixel point, and then the current pixel is selected according to the target reference pixel point.
- the pixel values of the points are predicted to improve the accuracy of the motion compensation prediction.
- determining, according to the location of the initial reference pixel point and the layout information of the reference image, the location of the target reference pixel of the current pixel in the reference image including: according to the position and reference of the initial reference pixel Layout information of the image, determining a position of the target reference pixel on the surface of the polyhedron, wherein the position of the target reference pixel on the surface of the polyhedron is at the intersection of the line connecting the initial reference pixel and the center point of the polyhedron with the surface of the polyhedron; The position of the target reference pixel in the reference image is determined according to the position of the target reference pixel at the surface of the polyhedron and the layout information of the reference image.
- the position of the target pixel on the surface of the polyhedron is determined in detail based on the position of the initial reference pixel and the layout information of the reference image in combination with the first and second examples.
- Fig. 8(a) is the current image
- Fig. 8(b) is the reference image
- P is the current pixel point in the current image
- P is located on the Bottom side of the current image
- P' is in the reference image with P The pixel of the same position
- P' is located on the Bottom face of the reference image
- T 1 , T 2 and T 3 are the initial reference pixel points of the current pixel, which are all located outside the Bottom face of the reference image.
- the regular hexahedron in FIG. 9 is a polyhedron corresponding to the reference image constructed with the Bottom surface of the reference image as a bottom surface, and FIG. 9 shows three current reference pixel points T 1 , T 2 and T 3 in FIG. 8( b ).
- the corresponding pixel points of the three target reference pixel points on the regular hexahedron are H 1 , H 2 and H 3 , respectively, which are located at T 1 , T 2 and T 3 and the O point line and the regular hexahedron.
- the position of the intersection point, so that the positions of H 1 , H 2 and H 3 can be determined from the positions of T 1 , T 2 and T 3 .
- T 1 , T 2 , and T 3 only show the case where the initial reference pixel points are at different positions, and in fact, only one reference pixel point exists at a time.
- how to determine the position of the target reference pixel at the corresponding pixel point of the polyhedron is determined according to the position of the initial reference pixel.
- the position of the projection pixel point on which the target pixel point is projected onto the surface of the polyhedron is determined in detail based on the position of T 1 .
- the regular hexahedron in FIG. 10 is a polyhedron corresponding to the reference image constructed with the Bottom surface of the reference image as the bottom surface, that is, the anchoring surface of the regular hexahedron (the anchoring surface can be understood as the bottom surface when constructing the polyhedron, for example, for reference)
- the anchoring surface can be understood as the bottom surface when constructing the polyhedron, for example, for reference
- One side of the image serves as the bottom surface of the constructed polyhedron, then the bottom surface is the anchoring surface of the polyhedron) the Bottom surface of the reference image.
- the hexahedron has a rib length of a
- O is a body of a regular hexahedron
- the T position is the position of the initial reference pixel point
- the ABCD surface is the face of the reference image in the same position as the current pixel point
- O' is O Projecting the vertical projection on the ABCD surface along the plane normal direction on the projection plane where the current pixel point ABCD is located
- J is the midpoint of the BC side
- K is the vertical projection of T on the O'J extension line
- OT cross BCGF The surface is at point H (the position where the H point is located at the position of the projected pixel point of the initial reference pixel), the vertical projection of the H point on the BC is S, and I is the vertical projection of T on the BC side.
- the lengths of OO' and O'J are both Assuming that the length of the JK line segment is x and the length of the KT line segment is y, it is obtained according to a similar triangle:
- L SJ is the length of SJ
- L SH is the length of SH.
- the polyhedron corresponding to the reference image includes two faces ACB and DBC (the polyhedron may also include other faces, which are not listed here), wherein the face ACB is in the same position as the current pixel in the reference image.
- the face where the pixel is located O is the body of the polyhedron, O' is the vertical projection of O on the ABC surface, the T position is the position of the initial reference pixel point, O'T and BC are intersected at the S point, OT and Face BCD is at point H, H" is the vertical projection of H on O'K, L is H" vertical projection on O'K, and I is the vertical projection of T on BC.
- the length of OO' is L OO '
- the length of O'J is L O'J
- the length of JK is L JK
- the length of KT is L KT
- the face ACB and the face BCD The angle is ⁇ ( ⁇ is greater than 90°)
- ⁇ O'SH ⁇
- the facet angle of the adjacent faces of the tetrahedron is less than 90°
- the face angle of the adjacent faces of the regular hexahedron is 90°
- the face angle of the adjacent faces of the regular octahedron and the more complex polyhedron is greater than 90°.
- the corresponding parameters can be obtained in a similar way, and finally L SJ and L H"J are obtained , thereby determining the position of the projected pixel point of the target reference pixel projected onto the surface of the polyhedron.
- the projection pixel here
- the position of the point refers to the position of the intersection of the initial reference pixel point and the line of the polyhedral body and the polyhedral surface.
- determining, according to the location of the initial reference pixel point and the layout information of the reference image, the location of the target reference pixel on the surface of the polyhedron includes: according to the location of the initial reference pixel point and the layout information of the reference image, Determining a position of the initial reference pixel in the first plane; determining a position of the target reference pixel on the surface of the polyhedron according to the position of the initial reference pixel at the first plane and the layout information of the reference image.
- a point at which the target reference pixel is located at the surface of the polyhedron may be understood as a projected pixel point of the target reference pixel, such that the target reference pixel of the current pixel is determined in the reference image according to the position of the initial reference pixel.
- the position of the medium includes: determining the position of the projected pixel point of the initial reference pixel on the surface of the polyhedron according to the position of the initial reference pixel point and the layout information of the reference image, and the position of the projected pixel point on the surface of the polyhedron is initially referenced The intersection of the line point with the center point of the polyhedron and the surface of the polyhedron; determining the position of the target reference pixel point in the reference image according to the position of the projected pixel point on the surface of the polyhedron and the layout information of the reference image.
- a two-dimensional image in a regular hexahedron format is taken as an example to specifically describe how to determine the position of a target reference pixel in a reference image, as shown in FIG. 12(a), assuming a side length of a regular hexahedron corresponding to the reference image.
- the layout format of the reference image is 4x3
- the target reference pixel of the current pixel is projected to the H1 point of the front face of the regular hexahedron, and the distances of H1 from the apex of the upper left corner of the Front face are ⁇ x and ⁇ y, respectively.
- the position of the target reference pixel in the reference image in the two-dimensional image of the different partial format is determined below.
- FIG. 12 (c) Front face still in the lower left corner of FIG. 12 (b) the same position, but rotated 90 degrees clockwise, the H 1 in the reference image with respect to the reference image is the upper left corner vertex Q (a- ⁇ y, 2a+ ⁇ x).
- determining a predicted value of a pixel value of the current pixel point according to the pixel value of the target reference pixel point and/or the pixel value of the adjacent pixel point of the target reference pixel point, including: the target reference pixel point
- the pixel value is determined as the predicted value of the pixel value of the current pixel.
- the pixel value of the target reference pixel may be a pixel value of the location of the target reference pixel, and the pixel value of the target reference pixel is directly determined as the predicted value of the current pixel, which can reduce the calculation process.
- determining a predicted value of a pixel value of the current pixel point according to the pixel value of the target reference pixel point and/or the pixel value of the adjacent pixel point of the target reference pixel point including:
- the weighting process may be a smoothing process on the target reference pixel and surrounding pixels, that is, averaging the pixel values of the plurality of pixels including the target reference pixel, and averaging the obtained pixel values.
- the predicted value of the pixel value of the current pixel is a smoothing process on the target reference pixel and surrounding pixels, that is, averaging the pixel values of the plurality of pixels including the target reference pixel, and averaging the obtained pixel values.
- the pixel value of the target reference pixel point may be directly used as the predicted value of the pixel value of the current pixel point, or the pixel value of the current pixel point is not separately applied to the pixel of the current pixel point.
- the value is predicted, and the pixel values of the current pixel are predicted together according to the target reference pixel and the adjacent pixel around the target reference pixel.
- determining a predicted value of a pixel value of the current pixel point according to a pixel value of the target reference pixel point and/or a pixel value of a neighboring pixel point of the target reference pixel point including: according to the target reference pixel point The pixel value of the adjacent pixel is interpolated at the target reference pixel position; the pixel value obtained by the interpolation operation is determined as the predicted value of the pixel value of the current pixel.
- the accuracy of the pixel value of the target reference pixel may be first determined.
- the accuracy of the pixel value of the target reference pixel is smaller than the motion vector accuracy of the current pixel, interpolation is needed.
- the algorithm calculates the pixel value of the pixel around the target pixel, and uses the calculated pixel value as the pixel value of the target reference pixel.
- the accuracy of the pixel value of the target reference pixel is smaller than that of the current pixel The accuracy of the motion vector, so the interpolation algorithm is needed to recalculate the pixel value of the target reference pixel.
- the coordinates of the projection point H are (i+u, j+v), where i, j are non-negative integers, and u and v are floating point numbers of the [0, 1) interval.
- the position coordinates of the four integer pixels adjacent to H are P (i, j) , P (i, j+1) , P (i, j+1), and P (i+1, j+1), respectively .
- P H P (i,j) (1-u)(1-v)+P (i,j+1) (u)(1-v)+P (i,j+1) (1-u) (v)+P (i+1,j+1) (u)(v) (19)
- the coordinates of the projection point H are still (i+u, j+v), and the coordinates of the 16 adjacent points around the H point are respectively P (i-1, j-1) , P. (i-1, j+0) , P (i-1, j+1) , P (i-1, j+2) , P (i+0, j-1) , P (i+0, j +0) , P (i+0,j+1) , P (i+0,j+2) , P (i+1,j-1) , P (i+1,j+0) , P (i+1, j+1) , P (i+1, j+2) , P (i+2, j-1) , P (i+2, j+0) , P (i+2, j+ 1) and P (i+2, j+2) .
- the above only uses the double bilinear interpolation algorithm and the bicubic interpolation algorithm as an example to describe how to use the interpolation algorithm to determine the pixel at the position of the projection pixel in the embodiment of the present invention.
- the Lanczos interpolation algorithm, the nearest neighbor interpolation algorithm, and the like can also be used.
- Some non-analytic interpolation methods based on information such as image structure.
- the pixel value of the target reference pixel may be directly determined as the pixel value of the current pixel, regardless of whether the pixel value of the target reference pixel satisfies the corresponding requirement. .
- the pixel value of the target reference pixel is determined to meet the preset requirement. When the pixel value of the target reference pixel does not meet the requirement, the target reference may be used according to the target reference.
- Pixel values of adjacent pixel points of the pixel are interpolated to the target reference pixel point, and the obtained pixel value is used as a predicted value of the pixel value of the current pixel point, or the pixel value of the target reference pixel point and the target may be
- the pixel value of the adjacent pixel of the reference pixel is weighted, and the result of the weighting process is used as the predicted value of the pixel value of the current pixel.
- FIG. 14 is a schematic flowchart of a motion compensation prediction method according to an embodiment of the present invention. The method shown in Figure 14 includes:
- the position of the point is determined by the layout information of the polyhedron corresponding to the reference image, and the first plane is a plane in which the face corresponding to the second sub-image in the polyhedron is located.
- the initial reference pixel point when the initial reference pixel point is located outside the second sub-image, directly searching for the target reference pixel point that can be used as a reference function directly in the extended area according to the current pixel point and the motion information of the current pixel point. Then, the pixel value of the current pixel point is predicted according to the target reference pixel, and the target reference pixel point can be quickly found, and the accuracy of the motion prediction estimation can also be improved.
- the pixel value of the pixel in the extended area may be calculated according to the method shown in FIG. 5 above, so that when the pixel value of the current pixel is predicted, the target reference pixel can be directly found from the extended area. The prediction of the pixel value of the current pixel can be quickly realized.
- the method shown in FIG. 14 further includes: determining a position of the second pixel on the surface of the polyhedron according to the position of the first pixel and the layout information of the reference image, the second pixel being in the polyhedron
- the position of the surface is at the intersection of the line connecting the first pixel point and the center point of the polyhedron with the surface of the polyhedron; determining the second pixel point according to the position of the surface of the polyhedron of the second pixel and the layout information of the reference image The position in the reference image.
- the method shown in FIG. 14 further includes:
- the motion compensation prediction method shown in FIG. 14 is to first determine an extended region of a certain sub-image of the reference image, and then The position of the reference pixel in the extended area is determined, so that the reference pixel can be directly searched from the extended area when the current pixel is predicted, and the current pixel is predicted according to the pixel value of the reference pixel.
- the above process will be described in detail below with reference to FIG. 15, and the specific steps are as follows:
- the layout information of the polyhedron corresponding to the reference image (generally, the layout information of the polyhedron corresponding to the reference image is the same as the current image, and therefore, can be used to determine the plane of the pixel in the current image) and the position of the current pixel
- the face of the current pixel is determined, the face corresponding to the previous pixel in the reference image is found, and an extended region is constructed for the corresponding face, and the pixel value of the pixel in the extended region is determined.
- the specific effect of the extended area is similar to expanding or filling the surface around the surface. As shown in Fig. 15(a), the current pixel point P is on the Bottom plane, then it is necessary to construct an extended area for the Bottom plane in the reference image.
- the range of the extended area is as shown by the dotted area outside the Bottom plane in 15(b).
- a Bottom of the reference image may be used as the bottom surface as shown in FIG. 15(c).
- the polyhedron corresponding to the reference image is shown, if the polyhedron is unfolded, the positions of the polygon face, the Front face, the Left face, the Right face, and the like of the polyhedron respectively fall on the extended region are as shown in Fig. 15(d).
- the layout information of the polyhedron may finally determine the position of the second pixel corresponding to the first pixel in the current reference image, and determine the pixel value at the second pixel position, and finally the pixel value of the second pixel position as the first
- the pixel value of a pixel is similar to the initial reference pixel point in the motion compensation prediction method of the embodiment of the present invention shown in FIG. 5, and the second pixel point is similar to the motion compensation prediction method of the embodiment of the present invention shown in FIG.
- the target reference pixel point is similar to the motion compensation prediction method of the embodiment of the present invention shown in FIG.
- the reference pixel of the current pixel is determined, and whether the reference pixel is in the same position as the current pixel is determined according to the surface of the current pixel and the position of the reference pixel. If it is, then the pixel value of the reference pixel position can be directly used as the predicted value of the current pixel point pixel value. Otherwise, the following operations are required: first, the corresponding position of the reference pixel in the extended area is determined according to the relative positional deviation of the reference pixel from the current pixel. As shown in FIG.
- the reference pixel point T is not at the same position as the current pixel point P, and according to the relative positional deviation of the reference pixel point from the current pixel point, it can be known that the reference pixel point T should be located in the extended area.
- the area corresponding to the Rear surface If the extended area is placed according to the actual relative spatial position, as shown in Fig. 15(d), the position of the reference pixel in the extended area is determined according to the relative positional shift. Otherwise, it is necessary to determine the position of the reference pixel in the extended area according to the placement method of the specific extended area.
- the pixel value at this position is taken as the predicted value of the current pixel pixel value.
- the pixel points in the extended area may be integer pixels or fractional pixels. If it is a fractional pixel, the specific pixel value may be 1/2 precision, 1/4 precision, 1/8 precision, and the like.
- the reference pixel After determining that the reference pixel is in the extended region position, if there is no pixel value at the location, the reference pixel needs to be interpolated.
- Specific interpolation operations can use DCT interpolation, bilinear interpolation or other interpolation methods.
- the method shown in FIG. 14 is different from the method shown in FIG. 5 in that, in FIG. 14, the position of the current pixel point in the extended area is first calculated, so that when the current pixel point is predicted, it can be more Conveniently finding the target reference pixel eliminates the computational process of determining the position of the target reference pixel based on the position of the initial reference pixel.
- the method shown in FIG. 5 is to determine the position of the target reference pixel point according to the position of the initial reference pixel point when predicting the pixel value of the current pixel point. There is one more calculation process than the method shown in FIG.
- the other steps of the method shown in FIG. 14 are substantially the same as the other steps of the method shown in FIG. 5, and the duplicated description is appropriately omitted for brevity.
- FIG. 16 is a schematic flowchart of a motion compensation prediction method according to an embodiment of the present invention. The method shown in Figure 16 includes:
- the target reference pixel of the current pixel is on the surface of the polyhedron corresponding to the reference image. a position in which a position of the target reference pixel on the surface of the polyhedron and a position of the initial reference pixel at the position of the first plane pass through a center point of the polyhedron, and the first plane is a face of the polyhedron corresponding to the second sub image Plane
- the target reference pixel point that can actually serve as a reference is found according to the position of the initial reference pixel point, and then the current pixel is selected according to the target reference pixel point.
- the pixel values of the points are predicted to improve the accuracy of the motion compensation prediction.
- the motion compensation prediction method shown in FIG. 5 is different from the motion compensation prediction method shown in FIG. 16 in that FIG. 5 is that the target reference pixel point is directly determined according to the initial reference pixel position.
- the position in the image, and FIG. 15 is to first determine the position of the target reference pixel on the surface of the polyhedron corresponding to the reference image according to the position of the initial reference pixel, and then determine the target reference pixel according to the position of the target reference pixel on the surface of the polyhedron.
- the position in the reference image The explanation and explanation of the motion compensation prediction method shown in FIG. 5 are equally applicable to the motion compensation prediction method shown in FIG. 16, and therefore, for the sake of brevity, the repeated description is appropriately omitted herein.
- determining, according to the initial reference pixel position, a position of the target reference pixel of the current pixel point on a surface of the polyhedron corresponding to the reference image including: according to the position of the initial reference pixel point and the layout information of the reference image. Determining a position of the initial reference pixel in the first plane; determining a position of the target reference pixel on the surface of the polyhedron according to the position of the initial reference pixel at the first plane and the layout information of the reference image.
- determining a location of the target reference pixel in the reference image according to the location of the target reference pixel at the polyhedral surface includes: determining the target reference according to the location of the initial reference pixel and the layout information of the reference image. The position of the pixel in the reference image, the position of the target reference pixel on the surface of the polyhedron at the intersection of the line connecting the initial reference pixel and the center point of the polyhedron with the surface of the polyhedron.
- the motion compensation prediction method of the embodiment of the present invention is applicable to two-dimensional images of various polyhedral formats, and that the surface of the two-dimensional image of the polyhedral format is rotated, the order of the faces is changed, and the spatial layout of the faces is changed.
- the motion compensation prediction method of the embodiment of the present invention is also applicable to the case where the format is changed.
- the pixel value of the position of the target reference pixel point may be calculated when the pixel value of the current pixel point is predicted.
- the pixel value of the target reference pixel of the pixel to be processed may also be calculated in advance, so that the pixel value of the target reference loudness point of the pixel to be processed may be directly obtained when the pixel to be processed is processed, thereby saving the time for processing the image.
- the motion compensation prediction method of the embodiment of the present invention is described in detail in the above two-dimensional image in the 4x3 format.
- the motion compensation prediction method of the embodiment of the present invention is also applicable.
- two-dimensional images The motion compensation prediction method of the embodiment of the present invention is also applicable to the case where the surface is partially or completely rotated, the arrangement order of the different faces, and the arrangement method.
- the pixel value of the reference pixel in the reference image may be directly used as the predicted value of the pixel value of the current pixel point, and the pixel value of the reference pixel point and the weight value of the pixel value of the surrounding pixel point may also be utilized. Or the pixel value obtained by other operations is used as the predicted value of the pixel value of the current pixel.
- the motion compensation prediction method of the embodiment of the present invention is described in detail above with reference to FIG. 1-16.
- the motion compensation prediction apparatus of the embodiment of the present invention will be described in detail below with reference to FIGS. 17-22. It should be understood that the motion compensation prediction apparatus in FIGS. 17-22 can perform the respective steps of the motion compensation prediction method in FIGS. 1-16, and the repeated description is appropriately omitted in order to avoid redundancy.
- FIG. 17 is a schematic block diagram of a motion compensation prediction apparatus according to an embodiment of the present invention.
- the motion compensation prediction apparatus 400 includes:
- the first determining unit 410 is configured to determine a position of an initial reference pixel of the current pixel in the reference image, where the current pixel is located in the first sub-image of the current image.
- a second determining unit 420 configured to determine, according to a location of the initial reference pixel point, when the initial reference pixel point is located outside a second sub-image of the reference image corresponding to the first sub-image position Determining a position of a target reference pixel of the current pixel in the reference image, wherein a position of the target reference pixel at a surface of the polyhedron corresponding to the reference image and the initial reference pixel are in a first plane A line connecting the position passes through a center point of the polyhedron, and a position of the target reference pixel on a surface of the polyhedron is determined according to a position of the initial reference pixel point and layout information of the polyhedron corresponding to the reference image.
- the first plane is a plane in which the face corresponding to the second sub-image of the polyhedron is located;
- the prediction unit 430 is configured to determine a predicted value of the pixel value of the current pixel point according to the pixel value of the target reference pixel point and/or the pixel value of the adjacent pixel point of the target reference pixel point.
- the present invention finds a target reference pixel point that can actually serve as a reference according to the position of the initial reference pixel point, and then predicts the pixel value of the current pixel point according to the target reference pixel point, Improve the accuracy of motion prediction estimates.
- the motion compensation prediction apparatus 400 further includes:
- the determining unit 440 is configured to determine, according to the layout information of the reference image and the position of the initial reference pixel, whether the initial reference pixel is located outside the second sub image in the reference image.
- the second determining unit 420 is specifically configured to:
- the second determining unit 420 is specifically configured to:
- the layout information includes quantity information of a face of the polyhedron, arrangement information of a sub-image of the reference image, and arrangement order information of the sub-image of the reference image.
- Rotation of the sub-image of the reference image At least one of the information.
- the prediction unit 430 is specifically configured to:
- a pixel value of the target reference pixel is determined as a predicted value of a pixel value of the current pixel.
- the prediction unit 430 is specifically configured to:
- the pixel value of the position at which the target reference pixel is processed by the weighting is determined as a predicted value of the pixel value of the current pixel.
- the prediction unit 430 is specifically configured to:
- the pixel value obtained by the interpolation operation is determined as a predicted value of a pixel value of the current pixel point.
- FIG. 18 is a schematic block diagram of a motion compensation prediction apparatus according to an embodiment of the present invention.
- the motion compensation prediction apparatus 500 includes:
- a first determining unit 510 configured to determine a position of an initial reference pixel of a current reference pixel in a reference image, where the current pixel is located in a first sub-image of the current image;
- a second determining unit 520 configured to determine, when the initial reference pixel point is located outside the second sub-image of the reference image corresponding to the first sub-image, in the sub-image The position of the target reference pixel in the extended area
- the extended area of the second sub-image is located outside the second sub-image, the extended area includes a plurality of pixel points, and the pixel value of any one of the first pixel points in the extended area is according to the reference Determining, by the pixel value of the second pixel in the image, the connection of the position of the second pixel to the surface of the polyhedron formed by the reference image and the position of the first pixel at the first plane passes through a central point of the polyhedron, wherein a position of the second pixel at a face of the polyhedron is determined according to a position of the first pixel and a layout information of the polyhedron corresponding to the reference image, the first plane a plane in which the face corresponding to the second sub-image in the polyhedron is located;
- the prediction unit 530 is configured to determine a predicted value of a pixel value of the current pixel point according to a pixel value of the target reference pixel point and/or a pixel value of a neighboring pixel point of the target reference pixel point.
- the present invention directly searches for the target reference pixel point that can be used as a reference directly in the extended area according to the current pixel point and the motion information of the current pixel point, and then according to the target reference pixel point pair.
- the pixel values of the pixels are predicted, the target reference pixels can be quickly found, and the accuracy of the motion prediction estimation can be improved.
- the motion compensation prediction apparatus further includes:
- the third determining unit 540 is specifically configured to:
- Determining a position of the second pixel in the reference image according to a position of the second pixel at a position of a surface of the polyhedron and layout information of the reference image.
- the motion compensation prediction apparatus further includes:
- the determining unit 550 is configured to determine, according to the layout information of the reference image and the position of the initial reference pixel point, whether the initial reference pixel point is outside the second sub image in the reference image.
- FIG. 19 is a schematic block diagram of a motion compensation prediction apparatus according to an embodiment of the present invention.
- the motion compensation prediction apparatus 600 includes:
- a first determining unit 610 configured to determine an initial reference pixel position of the current pixel in the reference image, where the current pixel is located in the first sub-image of the current image;
- a second determining unit 620 when the initial reference pixel point is located outside the second sub-image of the reference image corresponding to the first sub-image, determining the current pixel according to the initial reference pixel position a position of the target reference pixel at a position of a surface of the polyhedron corresponding to the reference image, wherein a position of the target reference pixel at a surface of the polyhedron and a position of the initial reference pixel at a first plane a line passing through a center point of the polyhedron, the first plane being a plane of a face of the polyhedron corresponding to the second sub-image;
- a third determining unit 630 configured to determine, according to a position of the target reference pixel on the surface of the polyhedron, a position of the target reference pixel in the reference image;
- a prediction unit 640 configured to determine a predicted value of a pixel value of the current pixel point according to a pixel value of the target reference pixel point in the reference image and/or a pixel value of a neighboring pixel point of the target reference pixel point .
- the present invention finds a target reference pixel point that can actually serve as a reference according to the position of the initial reference pixel point, and then predicts the pixel value of the current pixel point according to the target reference pixel point, Improve the accuracy of motion prediction estimates.
- the second determining unit 620 is specifically configured to:
- the third determining unit is specifically configured to:
- the motion compensation prediction method and the motion compensation prediction apparatus according to the embodiment of the present invention are described in detail above with reference to FIG. 1-19.
- the motion compensation prediction method of the embodiment of the present invention can be regarded as one of encoding or decoding processes.
- the intermediate process or step, the panoramic video encoder or the panoramic video decoder may implement the motion compensation prediction method of the embodiment of the present invention.
- the decoding process of the panoramic video decoder and the encoding process of the panoramic video encoder will be described in detail below with reference to FIG. 20 and FIG. 21, respectively.
- the current image Before encoding the current image, the current image is generally divided into several blocks of equal size, and then each block is encoded.
- the layout information of the current image is generally transmitted as header information to the decoding end.
- the header information is the preamble information, and the decoder first decodes the header information after receiving the encoded code stream, and then decodes the subsequent code stream according to the header information. It is assumed that at the time of encoding, the current image is divided into a plurality of image blocks and encoded in sequence, and the decoding is performed in the same manner. The decoding is performed sequentially in order. For the current block, if it is determined that it is the inter-frame coding mode, in order to obtain its reconstruction information, the reconstructed value of the pixel value of the pixel in the current block is obtained.
- FIG. 20 is a schematic flow chart of a decoding process of a panoramic video decoder. Specifically include:
- the decoder can decode the motion information code stream and determine the current fast motion information according to the analysis result.
- the motion information of the current block may also be determined by decoding the indication information of the motion information and the difference information of the motion information and the motion information, that is, the code stream of the motion information may not be directly parsed. It is possible to determine the motion information of the current block.
- the decoding end may construct a motion information set (for example, the motion information set includes motion information of the reconstructed block adjacent to the time or space of the current block, and the construction method of the set is jointly agreed by the codec side), then decoding The terminal may parse the indication information of the predicted motion information, and then determine the predicted motion information from the set, and obtain the motion information of the current block according to the predicted motion information and the difference information of the predicted motion information and the motion information.
- the decoding end also uses the predicted motion information determined according to the indication information of the predicted motion information as the motion information of the current block, which method is specifically used by the codec terminal, or the encoding end transmits the mode information used. To the decoding end, the decoding end determines which manner is used to determine the motion information of the current block according to the received mode information.
- step 710 if it is a one-way reference, a set of motion information needs to be parsed. If it is a two-way reference, you need to parse two sets of motion information. Specifically, several sets of motion information are parsed together by the codec, or the encoder transmits the set number related information to the decoding end, and the decoding end determines according to the parsing result.
- the decoder may determine a reference image of the current image in which the current block is located according to the running information of the current block, and determine a position of the reference block in the reference image.
- determining the predicted value of the pixel value of the pixel in the current block is a prediction value of determining the pixel value of each pixel in the current block, and determining the predicted value of the pixel value of the pixel may be specifically according to the embodiment of the present invention.
- the motion compensation prediction method is determined.
- the prediction block of the current block may be obtained according to the predicted value of the pixel value of each pixel, and the prediction block includes the pixel of each pixel in the current block. The predicted value of the value.
- step 720 if it is a one-way reference, it is only necessary to determine the position of a reference block in the reference image. For a pixel in the current block, only the position of one reference pixel needs to be determined, and then according to the present invention.
- the motion compensated prediction method of an embodiment determines a predicted value of a pixel pixel value within a current block.
- the positions of the two reference pixels in the two reference blocks need to be determined separately.
- the two predicted values of the pixel values of the pixel points within the current block can then be determined in accordance with the motion compensated prediction method of an embodiment of the present invention.
- the two predicted values are then weighted or otherwise manipulated to obtain a predicted value of the pixel value of the pixel within the current block.
- the specific operation mode is that the codec side has a common agreement, or the coding end transmits the used method to the decoding end, and the decoding end determines the used method according to the analysis result.
- the predicted value obtained by the operation is used as the final predicted value of the pixel value of the current pixel.
- the obtained predicted value may be smoothed to obtain a predicted value as a final predicted value of the pixel value of the current pixel point.
- the pixel value of the current pixel point can be bi-directionally predicted, that is, the current pixel point can be predicted twice, and the obtained two predicted values are weighted to obtain a predicted value of the pixel value of the current pixel point.
- the decoder decodes the coded code stream of the residual information of the pixel values of the pixel points in the current block, and obtains the residual information of the pixel values of the pixel points in the current block by using the inverse quantization and inverse transform methods. That is to say, the encoder decodes the coded code stream of the residual block of the current block, and then obtains the residual block of the current block by using the inverse quantization and inverse transform methods.
- step 730 in order to determine the residual information of the pixel values of the pixel points in the current block, it is also possible to use only the inverse transform or only the inverse quantization method, which method is used for the codec end common agreement, or coding.
- the terminal transmits the used method to the decoding end, and the decoding end determines the used method according to the analysis result.
- the reconstructed value of the pixel value of each pixel in the current block may be obtained according to the residual value information of the pixel value of the pixel in the current block obtained in step 720 and the pixel value of the pixel in the current block obtained in step 730.
- the prediction block of the current block may be obtained according to step 720, and then the prediction block is added to the residual block obtained in step 730 to obtain a reconstructed block of the current block, where the reconstructed block includes each pixel in the current block.
- the reconstructed value of the pixel value may be obtained according to the residual value information of the pixel value of the pixel in the current block obtained in step 720 and the pixel value of the pixel in the current block obtained in step 730.
- step 740 in order to obtain the reconstructed value of the pixel value of the pixel point in the current block, after using the prediction information plus the residual information, some other operations, such as deblocking filtering, etc., may also be required. Whether the other operations and other operations are required to be jointly agreed by the codec terminal, or the encoding end transmits the used method to the decoding end, and the decoding end determines the used method according to the analysis result.
- the decoding process of the panoramic video decoder is described in detail above with reference to FIG. 20.
- the decoding process is generally performed on the encoded code stream obtained by using the panoramic video encoder as an example. Carry out detailed instructions.
- the layout information of the spherical image is generally determined after the acquisition or generation of the spherical image. Therefore, the encoder has already known the layout information of the spherical image before the encoder encodes the panoramic video.
- the layout information of the spherical image is generally transmitted as header information.
- the header information is a kind of preamble information, so that the decoder decodes the header information before receiving the encoded code stream, and after acquiring the layout information of the spherical image according to the header information, the subsequent code stream can be decoded.
- 21 is a schematic flow chart of an encoding process of a panoramic video encoder. Specifically include:
- the encoder first determines the motion information of the current block and then encodes the motion information. Specifically, the encoder selects a reference image for the current image in the reconstructed image, and searches for a matching block for the current block in a prescribed area of the reference image, and uses the matching block as a reference block of the current block, and in the process
- the reference image used in and the motion vector information indicating the positional offset of the reference block with respect to the current block is used as motion information and encoded.
- a plurality of blocks may be selected for the current block first, and which block is selected as the reference block at the end may be determined using a rate-distortion optimization criterion. For example, determining the number of bits of the motion information that needs to be encoded when the candidate reference block is used as the matching block, and the distortion value when using the pixel value of the pixel point in the current candidate reference block to predict the pixel value of the pixel point in the current block, using Lagrange The daily optimization method determines the cost of the candidate reference block. The candidate reference block with the least cost is then selected as the matching block. For the case of multiple reference images, it is necessary to perform the above operations for each image.
- the motion information may not be directly encoded, but the indication information of the motion information is predicted, or the difference information of the motion information and the operation information is encoded.
- the encoding end may construct a motion information set (for example, the motion information set includes motion information of the reconstructed block adjacent to the time or space of the current block, and the construction method of the set is jointly agreed by the codec side), then the encoding The terminal may select one of the motion information as the predicted motion information, and then obtain the difference information of the motion information of the current block and the predicted motion information, and encode the indication information of the motion information and the difference information.
- the decoding end can determine the motion information from the constructed motion information set according to the indication information, and use it as the motion information of the current block.
- Which method is used for the common agreement of the codec side can also be selected by the rate distortion method, and the selected method is transmitted to the decoding end.
- step 810 if it is a one-way reference, only one set of motion information is encoded. If it is a two-way reference, you need to encode two sets of motion information.
- the specific coding several sets of motion information is jointly agreed by the codec end, and can also be determined by the rate distortion method, and the corresponding information is transmitted by the encoding end to the decoding end.
- the unidirectional reference it is necessary to determine the position of a reference block in the reference image.
- the position of a reference pixel needs to be determined, and then the motion compensation prediction method according to an embodiment of the present invention is used.
- the predicted value of the pixel value of the reference pixel is determined. If it is a bidirectional reference, it is necessary to determine the position of the two reference blocks of the current block in the respective reference images, that is, for one pixel in the current block, it is necessary to separately determine two reference pixels in the two reference blocks. position.
- the two predicted values of the pixel values of the pixel points within the current block can then be determined in accordance with the motion compensated prediction method of an embodiment of the present invention.
- the two predicted values are then weighted or otherwise manipulated to obtain a predicted value of the pixel value of the pixel within the current block.
- the specific operation mode is that the codec side has a common agreement, or the coding end transmits the used method to the decoding end, and the decoding end determines the used method according to the analysis result.
- the encoder may determine a reference image of the current image in which the current block is located according to the running information of the current block, and determine a position of the reference block in the reference image.
- determining the predicted value of the pixel value of the pixel in the current block is a prediction value of determining the pixel value of each pixel in the current block, and determining the predicted value of the pixel value of the pixel may be specifically according to the embodiment of the present invention.
- the motion compensation prediction method is determined.
- the predicted value obtained by the operation is used as the final predicted value of the pixel value of the current pixel.
- the obtained predicted value may be smoothed to obtain a predicted value as a final predicted value of the pixel value of the current pixel point.
- the pixel value of the current pixel point can be bi-directionally predicted, that is, the current pixel point can be predicted twice, and the obtained two predicted values are weighted to obtain a predicted value of the pixel value of the current pixel point.
- the encoder subtracts the predicted value from the pixel value of the pixel in the current block to obtain the residual information of the pixel value of the pixel in the current block, and then processes the residual information of the pixel value of the pixel in the current block according to the transform quantization method. And encode the result after processing.
- the rate distortion method determines and transmits the information to the decoder.
- FIG. 22 is a schematic block diagram of an image encoder 900 according to an embodiment of the present invention, including a prediction module 910, a transform quantization module 920, an entropy encoding module 930, a reconstruction module 940, and a filtering module 950.
- the specific functions of each module are as follows:
- the prediction module 910 is configured to generate prediction data. Prediction module 910 can generate one or more prediction units (PUs) that each no longer partition the CU. Each PU of a CU may be associated with a different block of pixels within a block of pixels of the CU. Prediction module 910 can generate predictive pixel blocks for each PU of the CU. Prediction module 910 can use intra prediction or inter prediction to generate predictive pixel blocks for the PU. If prediction module 910 uses intra prediction to generate a predictive pixel block for the PU, prediction module 910 can generate a predictive pixel block of the PU based on the decoded pixels of the picture associated with the PU.
- PUs prediction units
- the prediction module 910 can generate a predictive pixel block of the PU based on the decoded pixels of the one or more pictures that are different from the picture associated with the PU. .
- Prediction module 910 can generate a residual pixel block of the CU based on the predictive pixel block of the PU of the CU.
- the residual pixel block of the CU may indicate the difference between the sampled value in the predictive pixel block of the PU of the CU and the corresponding sampled value in the initial pixel block of the CU.
- Transform quantization module 920 is for processing the predicted residual data.
- Image encoder 900 may perform recursive quadtree partitioning on the residual pixel blocks of the CU to partition the residual pixel blocks of the CU into one or more smaller residual pixel blocks associated with the transform units (TUs) of the CU. Because the pixels in the pixel block associated with the TU each correspond to one luma sample and two chroma samples, each TU can be associated with one luma residual sample block and two chroma residual sample blocks.
- Image encoder 900 may apply one or more transforms to the residual sample block associated with the TU to generate a coefficient block (ie, a block of coefficients).
- the transform can be a DCT transform or a variant thereof.
- the coefficient block is obtained by applying a one-dimensional transform in the horizontal and vertical directions to calculate a two-dimensional transform.
- Image encoder 900 may perform a quantization procedure for each of the coefficients in the coefficient block. Quantization generally refers to the process by which the coefficients are quantized to reduce the amount of data used to represent the coefficients, thereby providing further compression.
- Image encoder 900 may generate a set of syntax elements that represent coefficients in the quantized coefficient block.
- Image encoder 900 may apply an entropy encoding operation (eg, a context adaptive binary arithmetic coding (CABAC) operation) to some or all of the above syntax elements by entropy encoding module 930.
- CABAC context adaptive binary arithmetic coding
- entropy encoding module 930 can binarize the syntax elements to form a binary sequence that includes one or more bits (referred to as "binary").
- Entropy encoding module 930 can encode a portion of the binary using regular encoding, and can use bypass encoding to encode other portions of the binary.
- image encoder 900 may apply inverse quantization and inverse transform to the transformed coefficient block by reconstruction module 940 to reconstruct the residual sample block from the transformed coefficient block.
- Image encoder 900 may add the reconstructed residual sample block to a corresponding sample block of one or more predictive sample blocks to produce a reconstructed sample block.
- image encoder 900 can reconstruct the block of pixels associated with the TU. The pixel block of each TU of the CU is reconstructed in this way until the entire pixel block reconstruction of the CU is completed.
- image encoder 900 After image encoder 900 reconstructs the block of pixels of the CU, image encoder 900 performs a deblocking filtering operation through filtering module 950 to reduce the blockiness of the block of pixels associated with the CU. After the image encoder 900 performs the deblocking filtering operation, the image encoder 900 may use the sample adaptive offset (SAO) to modify the reconstructed pixel block of the CTB of the picture. After performing these operations, image encoder 900 may store the reconstructed blocks of pixels of the CU in a decoded picture buffer for use in generating predictive blocks of pixels for other CUs.
- SAO sample adaptive offset
- the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored on or transmitted as one or more instructions or code via a computer-readable medium and executed by a hardware-based processing unit.
- the computer readable medium can comprise a computer readable storage medium (which corresponds to a tangible medium such as a data storage medium) or a communication medium comprising, for example, any medium that facilitates transfer of the computer program from one place to another in accordance with a communication protocol. .
- computer readable media generally may correspond to (1) a non-transitory tangible computer readable storage medium, or (2) a communication medium such as a signal or carrier wave.
- Data storage media may be any available media that can be accessed by one or more computers or one or more processors to retrieve instructions, code and/or data structures for use in carrying out the techniques described herein.
- the computer program product can comprise a computer readable medium.
- certain computer-readable storage media may comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, disk storage or other magnetic storage device, flash memory, or may be used to store instructions or data structures. Any other medium in the form of the desired program code and accessible by the computer. Also, any connection is properly termed a computer-readable medium. For example, if you use coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technology (eg, infrared, radio, and microwave) to send commands from a website, server, or other remote source, coaxial cable , fiber optic cable, twisted pair, DSL, or wireless technologies (eg, infrared, radio, and microwave) are included in the definition of the media.
- coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technology eg, infrared, radio, and microwave
- a magnetic disk and an optical disk include a compact disk (CD), a laser disk, an optical disk, a digital video disk (DVD), a flexible disk, and a Blu-ray disk, wherein the disk usually reproduces data magnetically, and the disk is optically optically optical. Way to copy data. Combinations of the above should also be included within the scope of computer readable media.
- processors such as one or more digital signal processors (DSPs), general purpose microprocessors, application specific integrated circuits (ASICs), field programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuits
- DSPs digital signal processors
- ASICs application specific integrated circuits
- FPGAs field programmable logic arrays
- processors may refer to any of the foregoing structures or any other structure suitable for implementing the techniques described herein.
- the functionality described herein may be provided within dedicated hardware and/or software modules configured for encoding and decoding, or incorporated in a combined codec.
- the techniques can be fully implemented in one or more circuits or logic elements.
- the techniques of the present invention can be broadly implemented by a variety of devices or devices, including a wireless handset, an integrated circuit (IC), or a collection of ICs (eg, a chipset).
- IC integrated circuit
- Various components, modules or units are described in this disclosure to emphasize functional aspects of the apparatus configured to perform the disclosed techniques, but are not necessarily required to be implemented by different hardware units. Rather, as described above, various units may be combined in a codec hardware unit or combined with suitable software and/or by a collection of interoperable hardware units (including one or more processors as described above). Or firmware to provide.
- system and “network” are used interchangeably herein. It should be understood that the term “and/or” herein is merely an association relationship describing an associated object, indicating that there may be three relationships, for example, A and/or B, which may indicate that A exists separately, and A and B exist simultaneously. There are three cases of B alone. In addition, the character "/" in this article generally indicates that the contextual object is an "or" relationship.
- B corresponding to A means that B is associated with A, and B can be determined according to A.
- determining B from A does not mean that B is only determined based on A, and that B can also be determined based on A and/or other information.
- the disclosed systems, devices, and methods may be implemented in other manners.
- the device embodiments described above are merely illustrative.
- the division of the unit is only a logical function division.
- there may be another division manner for example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored or not executed.
- Another point, the mutual coupling shown or discussed Or a direct coupling or communication connection may be an indirect coupling or communication connection through some interface, device or unit, and may be electrical, mechanical or otherwise.
- the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of the embodiment.
- each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
- the functions may be stored in a computer readable storage medium if implemented in the form of a software functional unit and sold or used as a standalone product.
- the technical solution of the present invention which is essential or contributes to the prior art, or a part of the technical solution, may be embodied in the form of a software product, which is stored in a storage medium, including
- the instructions are used to cause a computer device (which may be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention.
- the foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, and the like. .
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Image Analysis (AREA)
Abstract
Description
Claims (28)
- 一种运动补偿预测方法,其特征在于,包括:确定当前像素点的初始参考像素点在参考图像中的位置,所述当前像素点位于所述当前图像的第一子图像内;当所述初始参考像素点位于所述参考图像中与所述第一子图像对应位置的第二子图像之外时,根据所述初始参考像素点的位置确定所述当前像素点的目标参考像素点在所述参考图像中的位置,其中,所述目标参考像素点在所述参考图像对应的多面体的表面的位置与所述初始参考像素点在第一平面的位置的连线经过所述多面体的中心点,所述目标参考像素点在所述多面体的表面的位置是根据所述初始参考像素点的位置和所述参考图像对应的所述多面体的布局信息确定的,所述第一平面为所述多面体中与所述第二子图像对应的面所在的平面;根据所述目标参考像素点的像素值和/或所述目标参考像素点的临近像素点的像素值,确定所述当前像素点的像素值的预测值。
- 如权利要求1所述的方法,其特征在于,所述方法还包括:根据所述参考图像的布局信息和所述初始参考像素点的位置判断所述初始参考像素点是否位于所述参考图像中的所述第二子图像之外。
- 如权利要求1或2所述的方法,其特征在于,所述根据所述初始参考像素点的位置确定所述当前像素点的目标参考像素点在所述参考图像中的位置,包括:根据所述初始参考像素点的位置和所述参考图像的布局信息,确定所述目标参考像素点在所述多面体的表面的位置,其中,目标参考像素点在所述多面体的表面的位置为所述初始参考像素点与所述多面体的中心点的连线与所述多面体的表面的交点处;根据所述目标参考像素点在所述多面体的表面的位置和所述参考图像的布局信息,确定所述目标参考像素点在所述参考图像中的位置。
- 如权利要求3所述的方法,其特征在于,根据所述初始参考像素点的位置和所述参考图像的布局信息,确定所述目标参考像素点在所述多面体的表面的位置,包括:根据所述初始参考像素点的位置和所述参考图像的布局信息,确定所述初始参考像素点在所述第一平面的位置;根据所述初始参考像素点在所述第一平面的位置和所述参考图像的布局信息,确定所述目标参考像素点在所述多面体的表面的位置。
- 如权利要求1-4中任一项所述的方法,其特征在于,所述布局信息包含所述多面体的面的数量信息,所述参考图像的子图象的排布方式信息,所述参考图像的子图象的排布顺序信息,所述参考图像的子图象的旋转信息中的至少一种。
- 如权利要求1-5中任一项所述的方法,其特征在于,所述根据所述目标参考像素点的像素值和/或所述目标参考像素点的临近像素点的像素值,确定所述当前像素点的像素值的预测值,包括:将所述目标参考像素点的像素值确定为所述当前像素点的像素值的预测值。
- 如权利要求1-5中任一项所述的方法,其特征在于,所述根据所述目标参考像素点的像素值和/或所述目标参考像素点的临近像素点的像素值,确定所述当前像素点的像素值的预测值,包括:对所述目标参考像素点的像素值以及所述目标参考像素点的临近像素点的像素值进行加权处 理;将所述加权处理到的所述目标参考像素点所在位置的像素值确定为所述当前像素点的像素值的预测值。
- 如权利要求1-5中任一项所述的方法,其特征在于,所述根据所述目标参考像素点的像素值和/或所述目标参考像素点的临近像素点的像素值,确定所述当前像素点的像素值的预测值,包括:根据目标参考像素点的临近像素点的像素值在所述目标参考像素点位置进行插值运算;将所述插值运算得到的像素值确定为所述当前像素点的像素值的预测值。
- 一种运动补偿预测方法,其特征在于,包括:确定当前参考像素点的初始参考像素点在参考图像中的位置,所述当前像素点位于所述当前图像的第一子图像内;当所述初始参考像素点位于所述参考图像中与所述第一子图像对应位置的第二子图像之外时,确定所述当前像素点在所述第二子图像的扩展区域中的目标参考像素点的位置,其中,所述第二子图像的扩展区域位于所述第二子图像之外,所述扩展区域包括多个像素点,所述扩展区域中任意一个第一像素点的像素值是根据所述参考图像中的第二像素点的像素值确定的,所述第二像素点在所述参考图像构成的多面体的表面的位置与所述第一像素点在第一平面的位置的连线经过所述多面体的中心点,所述第二像素点在所述多面体的面的位置是根据所述第一像素点的位置和所述参考图像对应的所述多面体的布局信息确定的,所述第一平面为所述多面体中与所述第二子图像对应的面所在的平面;根据所述目标参考像素点的像素值和/或所述目标参考像素点的临近像素点的像素值,确定所述当前像素点的像素值的预测值。
- 如权利要求9所述的方法,其特征在于,所述方法还包括:根据所述第一像素点的位置和所述参考图像的布局信息,确定所述第二像素点在所述多面体的表面的位置,所述第二像素点在所述多面体的表面的位置在所述第一像素点与所述多面体的中心点的连线与所述多面体的表面的交点处;根据所述第二像素点的在所述多面体的表面的位置和所述参考图像的布局信息,确定所述第二像素点在所述参考图像中的位置。
- 如权利要求9或10所述的方法,其特征在于,所述方法还包括:根据所述参考图像的布局信息和所述初始参考像素点的位置判断所述初始参考像素点是否位于所述参考图像中的所述第二子图像之外。
- 一种运动补偿预测方法,其特征在于,包括:确定当前像素点在参考图像中的初始参考像素点位置,所述当前像素点位于所述当前图像的第一子图像内;当所述初始参考像素点位于所述参考图像中与所述第一子图像对应位置的第二子图像之外时,根据所述初始参考像素点位置确定所述当前像素点的目标参考像素点在所述参考图像对应的多面体的表面的位置,其中,所述目标参考像素点在所述多面体的表面的位置与所述初始参考像素点在第一平面的位置的连线经过所述多面体的中心点,所述第一平面为所述多面体中与所述第二子图像对应的面所在的平面;根据所述目标参考像素点在多面体表面的位置确定所述目标参考像素点在所述参考图像中的 位置;根据所述参考图像中所述目标参考像素点的像素值和/或所述目标参考像素点的临近像素点的像素值,确定所述当前像素点的像素值的预测值。
- 如权利要求12所述的方法,其特征在于,所述根据所述初始参考像素点位置确定所述当前像素点的目标参考像素点在所述参考图像对应的多面体的表面的位置,包括:根据所述初始参考像素点的位置和所述参考图像的布局信息,确定所述初始参考像素点在所述第一平面的位置;根据所述初始参考像素点在所述第一平面的位置和所述参考图像的布局信息,确定所述目标参考像素点在所述多面体的表面的位置。
- 如权利要求12或13所述的方法,其特征在于,所述根据所述目标参考像素点在多面体表面的位置确定所述目标参考像素点在所述参考图像中的位置,包括:根据所述初始参考像素点的位置和所述参考图像的布局信息,确定所述目标参考像素点在所述参考图像中的位置,所述目标参考像素点在所述多面体的表面的位置在所述初始参考像素点与所述多面体的中心点的连线与所述多面体的表面的交点处。
- 一种运动补偿预测装置,其特征在于,包括:第一确定单元,用于确定当前像素点的初始参考像素点在参考图像中的位置,所述当前像素点位于所述当前图像的第一子图像内;第二确定单元,用于当所述初始参考像素点位于所述参考图像中与所述第一子图像对应位置的第二子图像之外时,根据所述初始参考像素点的位置确定所述当前像素点的目标参考像素点在所述参考图像中的位置,其中,所述目标参考像素点在所述参考图像对应的多面体的表面的位置与所述初始参考像素点在第一平面的位置的连线经过所述多面体的中心点,所述目标参考像素点在所述多面体的表面的位置是根据所述初始参考像素点的位置和所述参考图像对应的所述多面体的布局信息确定的,所述第一平面为所述多面体中与所述第二子图像对应的面所在的平面;预测单元,用于根据所述目标参考像素点的像素值和/或所述目标参考像素点的临近像素点的像素值,确定所述当前像素点的像素值的预测值。
- 如权利要求15所述的运动补偿预测装置,其特征在于,所述运动补偿预测装置还包括:判断单元,用于根据所述参考图像的布局信息和所述初始参考像素点的位置判断所述初始参考像素点是否位于所述参考图像中的所述第二子图像之外。
- 如权利要求15或16所述的运动补偿预测装置,其特征在于,所述第二确定单元具体用于:根据所述初始参考像素点的位置和所述参考图像的布局信息,确定所述目标参考像素点在所述多面体的表面的位置,所述目标参考像素点在所述多面体的表面的位置为所述初始参考像素点与所述多面体的中心点的连线与所述多面体的表面的交点处;根据所述目标参考像素点在所述多面体的表面的位置和所述参考图像的布局信息,确定所述目标参考像素点在所述参考图像中的位置。
- 如权利要求17所述的运动补偿预测装置,其特征在于,第二确定单元具体用于:根据所述初始参考像素点的位置和所述参考图像的布局信息,确定所述初始参考像素点在所述第一平面的位置;根据所述初始参考像素点在所述第一平面的位置和所述参考图像的布局信息,确定所述目标 参考像素点在所述多面体的表面的位置。
- 如权利要求15-18中任一项所述的运动补偿预测装置,其特征在于,所述布局信息包含所述多面体的面的数量信息,所述参考图像的子图象的排布方式信息,所述参考图像的子图象的排布顺序信息,所述参考图像的子图象的旋转信息中的至少一种。
- 如权利要求15-19中任一项所述的运动补偿预测装置,其特征在于,所述预测单元具体用于:将所述目标参考像素点的像素值确定为所述当前像素点的像素值的预测值。
- 如权利要求15-19中任一项所述的运动补偿预测装置,其特征在于,所述预测单元具体用于:对所述目标参考像素点的像素值以及所述目标参考像素点的临近像素点的像素值进行加权处理;将所述加权处理到的所述目标参考像素点所在位置的像素值确定为所述当前像素点的像素值的预测值。
- 如权利要求15-19中任一项所述的运动补偿预测装置,其特征在于,所述预测单元具体用于:根据目标参考像素点的临近像素点的像素值在所述目标参考像素点位置进行插值运算;将所述插值运算得到的像素值确定为所述当前像素点的像素值的预测值。
- 一种运动补偿预测装置,其特征在于,包括:第一确定单元,用于确定当前参考像素点的初始参考像素点在参考图像中的位置,所述当前像素点位于所述当前图像的第一子图像内;第二确定单元,用于当所述初始参考像素点位于所述参考图像中与所述第一子图像对应位置的第二子图像之外时,确定所述当前像素点在所述子图像的扩展区域中的目标参考像素点的位置,其中,所述第二子图像的扩展区域位于所述第二子图像之外,所述扩展区域包括多个像素点,所述扩展区域中任意一个第一像素点的像素值是根据所述参考图像中的第二像素点的像素值确定的,所述第二像素点在所述参考图像构成的多面体的表面的位置与所述第一像素点在第一平面的位置的连线经过所述多面体的中心点,所述第二像素点在所述多面体的面的位置是根据所述第一像素点的位置和所述参考图像对应的所述多面体的布局信息确定的,所述第一平面为所述多面体中与所述第二子图像对应的面所在的平面;预测单元,用于根据所述目标参考像素点的像素值和/或所述目标参考像素点的临近像素点的像素值,确定所述当前像素点的像素值的预测值。
- 如权利要求23所述的运动补偿预测装置,其特征在于,所述运动补偿预测装置还包括:第三确定单元,用于根据所述第一像素点的位置和所述参考图像的布局信息,确定所述第二像素点在所述多面体的表面的位置,所述第二像素点在所述多面体的表面的位置在所述第一像素点与所述多面体的中心点的连线与所述多面体的表面的交点处;所述第三确定单元还用于根据所述第二像素点的在所述多面体的表面的位置和所述参考图像的布局信息,确定所述第二像素点在所述参考图像中的位置。
- 如权利要求23或24所述的运动补偿预测装置,其特征在于,所述运动补偿预测装置还包括:判断单元,用于根据所述参考图像的布局信息和所述初始参考像素点的位置判断所述初始参考像素点是否位于所述参考图像中的所述第二子图像之外。
- 一种运动补偿预测装置,其特征在于,包括:第一确定单元,用于确定当前像素点在参考图像中的初始参考像素点位置,所述当前像素点位于所述当前图像的第一子图像内;第二确定单元,当所述初始参考像素点位于所述参考图像中与所述第一子图像对应位置的第二子图像之外时,根据所述初始参考像素点位置确定所述当前像素点的目标参考像素点在所述参考图像对应的多面体的表面的位置,其中,所述目标参考像素点在所述多面体的表面的位置与所述初始参考像素点在第一平面的位置的连线经过所述多面体的中心点,所述第一平面为所述多面体中与所述第二子图像对应的面所在的平面;第三确定单元,用于根据所述目标参考像素点在多面体表面的位置确定所述目标参考像素点在所述参考图像中的位置;预测单元,用于根据所述参考图像中所述目标参考像素点的像素值和/或所述目标参考像素点的临近像素点的像素值,确定所述当前像素点的像素值的预测值。
- 如权利要求26所述的运动补偿预测装置,其特征在于,所述第二确定单元具体用于:根据所述初始参考像素点的位置和所述参考图像的布局信息,确定所述初始参考像素点在所述第一平面的位置;根据所述初始参考像素点在所述第一平面的位置和所述参考图像的布局信息,确定所述目标参考像素点在所述多面体的表面的位置。
- 如权利要求26或27所述的运动补偿预测装置,其特征在于,所述第三确定单元具体用于:根据所述初始参考像素点的位置和所述参考图像的布局信息,确定所述目标参考像素点在所述参考图像中的位置,所述目标参考像素点在所述多面体的表面的位置在所述初始参考像素点与所述多面体的中心点的连线与所述多面体的表面的交点处。
Priority Applications (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2019531518A JP6690061B2 (ja) | 2016-08-30 | 2017-08-24 | 動き補償予測方法およびデバイス |
EP17845307.2A EP3499894A4 (en) | 2016-08-30 | 2017-08-24 | METHOD AND DEVICE FOR PREDICTING MOTION COMPENSATION |
KR1020197008671A KR102210609B1 (ko) | 2016-08-30 | 2017-08-24 | 움직임 보상 예측 방법 및 디바이스 |
US16/289,250 US11128874B2 (en) | 2016-08-30 | 2019-02-28 | Motion compensating prediction method and device |
Applications Claiming Priority (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201610783101.X | 2016-08-30 | ||
CN201610783101 | 2016-08-30 | ||
CN201610896027.2 | 2016-10-13 | ||
CN201610896027.2A CN107801039B (zh) | 2016-08-30 | 2016-10-13 | 运动补偿预测方法和装置 |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US16/289,250 Continuation US11128874B2 (en) | 2016-08-30 | 2019-02-28 | Motion compensating prediction method and device |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2018041005A1 true WO2018041005A1 (zh) | 2018-03-08 |
Family
ID=61300089
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2017/098894 WO2018041005A1 (zh) | 2016-08-30 | 2017-08-24 | 运动补偿预测方法和设备 |
Country Status (1)
Country | Link |
---|---|
WO (1) | WO2018041005A1 (zh) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP3515076A4 (en) * | 2016-09-30 | 2019-07-24 | Huawei Technologies Co., Ltd. | METHOD AND DEVICE FOR MOTION COMPENSATION FORECASTING |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101002474A (zh) * | 2004-08-13 | 2007-07-18 | 庆熙大学校产学协力团 | 用于二十面体全景图像的编码和解码的方法和设备 |
JP2013046270A (ja) * | 2011-08-25 | 2013-03-04 | Olympus Corp | 画像貼り合せ装置、撮影装置、画像貼り合せ方法、および画像処理プログラム |
CN105137705A (zh) * | 2015-08-14 | 2015-12-09 | 太微图影(北京)数码科技有限公司 | 一种虚拟球幕的创建方法和装置 |
CN105245838A (zh) * | 2015-09-29 | 2016-01-13 | 成都虚拟世界科技有限公司 | 一种全景视频播放方法及播放器 |
CN105898344A (zh) * | 2016-04-12 | 2016-08-24 | 乐视控股(北京)有限公司 | 一种全景视频的播放方法和装置 |
-
2017
- 2017-08-24 WO PCT/CN2017/098894 patent/WO2018041005A1/zh unknown
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101002474A (zh) * | 2004-08-13 | 2007-07-18 | 庆熙大学校产学协力团 | 用于二十面体全景图像的编码和解码的方法和设备 |
JP2013046270A (ja) * | 2011-08-25 | 2013-03-04 | Olympus Corp | 画像貼り合せ装置、撮影装置、画像貼り合せ方法、および画像処理プログラム |
CN105137705A (zh) * | 2015-08-14 | 2015-12-09 | 太微图影(北京)数码科技有限公司 | 一种虚拟球幕的创建方法和装置 |
CN105245838A (zh) * | 2015-09-29 | 2016-01-13 | 成都虚拟世界科技有限公司 | 一种全景视频播放方法及播放器 |
CN105898344A (zh) * | 2016-04-12 | 2016-08-24 | 乐视控股(北京)有限公司 | 一种全景视频的播放方法和装置 |
Non-Patent Citations (1)
Title |
---|
See also references of EP3499894A4 * |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP3515076A4 (en) * | 2016-09-30 | 2019-07-24 | Huawei Technologies Co., Ltd. | METHOD AND DEVICE FOR MOTION COMPENSATION FORECASTING |
US10779000B2 (en) | 2016-09-30 | 2020-09-15 | Huawei Technologies Co., Ltd. | Motion-compensated prediction method and apparatus |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US10779000B2 (en) | Motion-compensated prediction method and apparatus | |
JP6690061B2 (ja) | 動き補償予測方法およびデバイス | |
WO2018095087A1 (zh) | 一种去块滤波方法及终端 | |
CN111279673B (zh) | 具有电子卷帘快门校正的图像拼接的系统和方法 | |
US10986371B2 (en) | Sample adaptive offset filtering method for reconstructed projection-based frame that employs projection layout of 360-degree virtual reality projection | |
US20240098298A1 (en) | Segmentation-based parameterized motion models | |
CN114788264B (zh) | 用于发出虚拟边界和环绕运动补偿的信号的方法 | |
WO2022068682A1 (zh) | 图像处理方法及装置 | |
TWI688256B (zh) | 具有無效區域的vr 圖像的視訊編解碼方法和裝置 | |
US11115678B2 (en) | Diversified motion using multiple global motion models | |
WO2018107800A1 (zh) | 一种运动矢量解码方法及解码器 | |
WO2018068747A1 (zh) | 运动补偿预测方法和设备 | |
US11330268B2 (en) | Apparatus and methods for encoding and decoding a video signal | |
US20210150665A1 (en) | Image processing method and device | |
WO2018041005A1 (zh) | 运动补偿预测方法和设备 | |
TWI756526B (zh) | 用於重構的基於投影的幀的樣本適應性偏移濾波方法 | |
JP2023514215A (ja) | 画像およびビデオ圧縮のイントラ予測 | |
CN109151484B (zh) | 解码视频比特流的方法和装置 | |
WO2018059500A1 (zh) | 运动补偿预测方法和设备 | |
WO2023142132A1 (zh) | 基于分辨率的解码方法、编码方法、解码器以及编码器 | |
CN114792290A (zh) | 图像/视频处理 | |
CN110677673A (zh) | 帧间编码方法、帧间解码方法及其设备和存储介质 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 17845307 Country of ref document: EP Kind code of ref document: A1 |
|
ENP | Entry into the national phase |
Ref document number: 2019531518 Country of ref document: JP Kind code of ref document: A |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
ENP | Entry into the national phase |
Ref document number: 2017845307 Country of ref document: EP Effective date: 20190315 |
|
ENP | Entry into the national phase |
Ref document number: 20197008671 Country of ref document: KR Kind code of ref document: A |