WO2020042185A1 - 视频处理方法及相关装置 - Google Patents

视频处理方法及相关装置 Download PDF

Info

Publication number
WO2020042185A1
WO2020042185A1 PCT/CN2018/103676 CN2018103676W WO2020042185A1 WO 2020042185 A1 WO2020042185 A1 WO 2020042185A1 CN 2018103676 W CN2018103676 W CN 2018103676W WO 2020042185 A1 WO2020042185 A1 WO 2020042185A1
Authority
WO
WIPO (PCT)
Prior art keywords
expanded
area
video processing
region
target
Prior art date
Application number
PCT/CN2018/103676
Other languages
English (en)
French (fr)
Inventor
郑萧桢
Original Assignee
深圳市大疆创新科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市大疆创新科技有限公司 filed Critical 深圳市大疆创新科技有限公司
Priority to CN201880036750.0A priority Critical patent/CN110741644A/zh
Priority to PCT/CN2018/103676 priority patent/WO2020042185A1/zh
Publication of WO2020042185A1 publication Critical patent/WO2020042185A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/597—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding specially adapted for multi-view video sequence encoding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/146—Data rate or code amount at the encoder output
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/174—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a slice, e.g. a line of blocks or a group of blocks
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/44—Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/90—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using coding techniques not provided for in groups H04N19/10-H04N19/85, e.g. fractals
    • H04N19/91—Entropy coding, e.g. variable length coding [VLC] or arithmetic coding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/90—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using coding techniques not provided for in groups H04N19/10-H04N19/85, e.g. fractals
    • H04N19/96—Tree coding, e.g. quad-tree coding

Definitions

  • the present application relates to the field of image processing, and more particularly, to a video processing method and related devices.
  • HMD head mounted devices
  • the present application provides a video processing method and a related device.
  • a video processing method including: acquiring an image to be expanded, the image to be expanded is a planar image obtained by splicing a plurality of partitions obtained by mapping a curved surface image by a polyhedron, and a plurality of the partitions are formed together; A plurality of regions to be expanded, each of which includes at least one of the plurality of partitions, and the image content in each of the regions to be expanded is continuous; Expand one week to obtain a plurality of expanded regions corresponding to the plurality of to-be-expanded regions, wherein each side of each to-be-expanded region is parallel to the corresponding side of the corresponding expanded region, and each The shape of the area to be expanded is the same as that of the corresponding area to be expanded; and prediction is performed according to the target image.
  • a video processing device including: a memory for storing instructions; a processor for executing instructions stored in the memory to perform the following operations: acquiring an image to be expanded, the image to be expanded is A planar image obtained by splicing a plurality of partitions obtained by mapping a curved surface image through a polyhedron, and the plurality of partitions together form a plurality of regions to be expanded, and each of the regions to be expanded includes at least one of the plurality of partitions And the image content in each of the to-be-expanded areas is continuous; the boundaries of the to-be-expanded areas are extended outward for one week to obtain a plurality of expanded areas corresponding to the to-be-expanded areas one-to-one, Each side of each region to be expanded is parallel to the corresponding side of the corresponding expanded region, and each of the regions to be expanded has the same shape as the corresponding region to be expanded; prediction is performed according to the target image .
  • a machine-readable storage medium stores instructions for executing the method according to the first aspect.
  • a computer program product comprising instructions for performing the method according to the first aspect.
  • Figure 1 is a schematic diagram of a curved video encoding / compression process.
  • the left image in FIG. 2 is a schematic diagram of mapping between a spherical image and an tangent octahedron
  • the right image is a schematic diagram of an embodiment in which a plurality of triangles on the surface of the octahedron are merged into a two-dimensional rectangular image in the left image.
  • the left image is a schematic diagram of the mapping of a spherical image and an icosahedral icosahedron
  • the right image is an embodiment of merging multiple triangles on the surface of the icosahedron in the left image into a two-dimensional rectangular image.
  • FIG. 4 is a schematic diagram of an extended manner of a specific boundary.
  • FIG. 5 is a schematic flowchart of a video processing method according to an embodiment of the present application.
  • FIG. 6 is a schematic diagram of a boundary extension manner according to an embodiment of the present application.
  • FIG. 7 is a schematic diagram of a boundary extension manner provided by another embodiment of the present application.
  • FIG. 8 is a schematic flowchart of an implementation manner of step 520 in FIG. 5.
  • FIG. 9 is an exemplary diagram of a filling manner of a filling area provided by an embodiment of the present application.
  • FIG. 10 is a schematic flowchart of another specific implementation manner of step 520 in FIG. 5.
  • FIG. 11 is a diagram illustrating an example of a filling manner of a filling area provided by another embodiment of the present application.
  • FIG. 12 is a diagram illustrating an example of a filling manner of a filling area according to another embodiment of the present application.
  • FIG. 13 is an exemplary diagram of an identification manner of a filled area provided by an embodiment of the present application.
  • FIG. 14 is a schematic structural diagram of a video processing apparatus according to an embodiment of the present application.
  • each device or system can reduce the bandwidth occupied by storing and transmitting a curved video.
  • the curved video may be a picture projected on any smooth surface (for example, at least part of a spherical surface or an ellipsoidal surface).
  • a curved video includes multiple frames of curved images, where the content in the curved images includes content captured in different directions at the same time. Therefore, a curved video can cover a large field of view.
  • a spherical video or 360-degree panoramic video
  • a spherical video may have a horizontal field of view of 360 degrees and a vertical field of view of 180 degrees, which are usually presented in a three-dimensional spherical form.
  • a spherical video is used as an example to describe a curved video. Those skilled in the art can clearly determine that the curved video is not limited to the spherical video.
  • Figure 1 is a schematic diagram of a curved video encoding / compression process.
  • the encoding / compression process of the curved video includes the following steps: mapping 101, prediction 102, transform 103, quantization 104, and entropy encoding 105.
  • the encoding device maps a three-dimensional curved video into a two-dimensional video so as to be able to utilize various encoding / compression techniques.
  • the encoding device may use a two-dimensional rectangular format to store and transmit the curved video.
  • the encoding device may adopt a two-dimensional rectangular format to support digital image processing and codec operations.
  • mapping methods for curved video For example, map a curved video to a two-dimensional video.
  • the spherical image in the spherical video may be mapped into a two-dimensional image based on a method of longitude and latitude projection.
  • each meridian on a spherical surface may be mapped to a plurality of equally spaced vertical straight lines on a two-dimensional plane, and each weft line may be mapped to a plurality of equally spaced horizontal straight lines on the two-dimensional plane.
  • a spherical image in a spherical video may be mapped into a two-dimensional image based on a method of polyhedral projection. Specifically, the images on different areas of the sphere are respectively projected onto a plurality of polygons on the surface of the circumscribed polyhedron of the sphere, and then the two-dimensional images on the plurality of polygons are combined into a two-dimensional image.
  • the method of projection may be that a virtual ray emitted from the center of the sphere passes through a point on the sphere and a point on the surface of the polyhedron in order, and the pixel information of the point on the sphere is projected onto the point on the surface of the polyhedron.
  • the polyhedron may be any polyhedron.
  • the polyhedron may be a hexahedron, an octahedron, an icosahedron, or the like.
  • the hexahedron may be a cube, a cuboid, a rhombic hexahedron, a parallelepiped, etc .
  • the octahedron may be a regular octahedron, a hexagonal column, a triangular anti-prism, etc .
  • the icosahedron may be a regular icosahedron, a regular pentagonal prism. Wait, there are no restrictions here.
  • multiple ways to combine multiple polygons on a polyhedron into a two-dimensional image In some embodiments, multiple polygons on the surface of the polyhedron are merged into a two-dimensional rectangle.
  • merging for example, the positional relationship between the polygons is different in different merging methods; and for example, the sizes / shapes of the rectangles merged in different merging methods are different.
  • the left image in FIG. 2 is a schematic diagram of the mapping of a spherical image and an tangent octahedron
  • the right image is an implementation of merging multiple triangles on the surface of the octahedron into a two-dimensional rectangular image Schematic illustration.
  • Each triangle on the surface of the regular octahedron shown in FIG. 2 and each triangle in the rectangular image are marked with corresponding numbers for indicating the correspondence between the triangles in the rectangular image and the triangles on the surface of the regular octahedron. relationship.
  • FIG. 2 refers to the triangles labeled 3 in the regular octahedron are cut into two right-angled triangles labeled 3_1 and 3_2; other similar numbers are the same.
  • two triangles that share a side in a rectangular image also share the same side on the surface of a regular octahedron, so that the image content between the two triangles in a rectangular image is continuous, which can improve coding efficiency.
  • the rectangle shown in the right figure in FIG. 2 is only an example of a method of merging multiple triangles on a regular octahedron. There may be other ways to merge into a two-dimensional rectangular image. For example, you can change the Positional relationship, or changing the length and width of a rectangular image, is not limited here.
  • the left image of Figure 3 is a schematic diagram of the mapping of a spherical image and an icosahedron
  • the right image is a combination of multiple triangles on the surface of the icosahedron in the left image into a two-dimensional rectangular image.
  • a schematic diagram of an embodiment Each triangle in the icosahedron and rectangular images shown in FIG. 3 is marked with a corresponding number to indicate the correspondence between the triangles in the rectangular image and the triangles on the surface of the octahedron.
  • the rectangle shown in the right image in FIG. 3 is only an example of a combination of multiple triangles on the icosahedron. There can also be other ways to merge into a rectangle. For example, the position relationship between the triangles in the right image of FIG. 3 can be changed. Or change the length and width of the rectangular image, which is not limited here.
  • the mapped two-dimensional video may be a rectangular video, or a video of other shapes (for example, a circle, an oval, or a regular polygon).
  • the curved video may not be a spherical video, but other three-dimensional curved videos, such as a curved video (such as a hemispherical video) on a spherical surface, an ellipsoid video, and a part of a curved surface (such as a semi-ellipsoid).
  • Spherical video other regular or irregular 3D curved video.
  • the method of mapping a spherical video into a two-dimensional video can also be applied to mapping a three-dimensional curved video of an aspherical video into a two-dimensional video.
  • some commonly used video codec standards such as high-efficiency video encoding HEVC / H.265, H.264 / AVC, AVS1-P2, AVS2-P2, VP8, VP9, you can compress, encode, and decode 2D video .
  • the encoding device may further write mapping information for indicating a mapping method into a code stream.
  • the mapping information may be written in at least one of the following: a sequence header, a picture header, a slice header, a video parameter set (VPS), and a sequence parameter set (VPS) sequence parameter set (SPS), picture parameter set (PPS), additional enhancement information (SEI), extension data (extension data).
  • the two-dimensional image after mapping is hereinafter referred to as including a plurality of partitions, where each partition corresponds to a polygon on the surface of the polyhedron, for example, within each triangle in the rectangular images in the right image of FIGS. 2 and 3 Image as a partition.
  • each partition corresponds to a polygon on the surface of the polyhedron, for example, within each triangle in the rectangular images in the right image of FIGS. 2 and 3 Image as a partition.
  • some of the polygons on the polyhedron may be divided, and one polygon is divided into at least two polygons.
  • the triangle 7 in the left image of FIG. 2 is divided into two triangles (triangle 7_1 and triangle 7_2) in the rectangle in the right image; the triangle 4 in the left image of FIG.
  • triangle 3 is divided into two in the rectangle on the right image.
  • Triangle (triangle 4_1 and triangle 4_2).
  • an image in each of the at least two polygons in the two-dimensional image is used as a partition, respectively.
  • the triangle 7_1 in the rectangular figure on the right in FIG. 2 is one partition
  • the triangle 7_2 is another partition.
  • the shape of at least one partition in the mapped two-dimensional image may be regular and / or irregular.
  • the shape of at least one partition in the two-dimensional image may include a rectangle, a triangle, a diamond, a circle, a pentagon, a hexagon, or other shapes.
  • the triangle may include at least one of a regular triangle, a right-angled triangle, and an isosceles triangle.
  • the prediction step 102 is used to reduce redundant information in the image.
  • a prediction block refers to a basic unit for performing prediction. In some standards, the prediction block is also called a prediction unit (PU).
  • PU prediction unit
  • the image is divided into multiple image blocks. Further, each of the multiple image blocks can be divided into multiple image blocks again, and so on.
  • the number of division levels can be different, and the operation methods they undertake are also different.
  • the names of image blocks on the same level may be different.
  • each image block in a plurality of image blocks into which a frame of image is first divided is called a coding tree unit (CTU); each coding tree unit may contain an encoding
  • a coding unit (CU) may be divided into multiple coding units again; a coding unit may be divided into one, two, four, or other number of prediction units according to a prediction mode.
  • the coding tree unit is also called a maximum coding unit (LCU).
  • LCU maximum coding unit
  • the prediction unit is the smallest unit in the image, and the prediction unit will not continue to be divided into multiple image blocks.
  • Prediction refers to finding image data similar to the prediction block, which is also referred to as the reference block of the prediction block.
  • the difference between the prediction block and the reference block of the prediction block is encoded / compressed to reduce redundant information in the encoding / compression.
  • the difference between the prediction block and the reference block may be a residual obtained by subtracting corresponding pixel values of the prediction block and the reference block.
  • Prediction includes intra prediction and inter prediction. Intra prediction refers to finding a reference block of the prediction block in the frame where the prediction block is located, and inter prediction refers to finding the reference block of the prediction block in a frame other than the frame in which the prediction block is located.
  • the transforming step 103 can further reduce redundant information in the image block.
  • the residual between the prediction block and the reference block of the prediction block may be used to remove the correlation of the residual and improve the coding efficiency.
  • the transformation of the prediction block usually adopts a two-dimensional transformation, that is, in the encoding device, the residual information of the prediction block is multiplied with an N * M transformation matrix and its transposed matrix, and the transformation coefficient is obtained after the multiplication.
  • the transform coefficient may be quantized in the quantization step 104 to obtain a quantized coefficient.
  • the entropy coding step 105 the quantized coefficients are subjected to entropy coding.
  • the bit stream obtained by entropy encoding and the encoding mode information after encoding such as the intra prediction mode and motion vector information, are stored or sent to the decoding device.
  • the decoding device After receiving the data, the decoding device executes a process opposite to the encoding / compression process in the encoding device. For example, the following steps are performed sequentially: entropy decoding, inverse quantization, inverse transform, prediction, and mapping. Wherein, in the prediction step, the decoding device may perform the prediction using the same method as the prediction method described above. In the mapping step, the decoding device uses the opposite method to the mapping method described above, that is, in the encoding device, after the curved surface image is mapped onto the surface of the polyhedron, the multiple polygons on the surface of the polyhedron are merged into a two-dimensional A planar image, and the decoding device maps a two-dimensional planar image to a curved image.
  • the decoding device may first split multiple partitions in the two-dimensional planar image and then piece together at least part of a polyhedron. An image on at least part of the polyhedron is then mapped onto a curved surface to form a curved image.
  • the content of two adjacent partitions in the two-dimensional video may be discontinuous.
  • triangle 7 and triangle 4 are not adjacent
  • triangle 5 and triangle 6 are not adjacent
  • triangle 3 and triangle 0 are not adjacent
  • triangles 1 and 2 are not adjacent; therefore, in the right of FIG.
  • the image content between partition 7_1 and partition 4, partition 3_1 and partition 0, partition 5 and partition 6, and partition 1 and partition 2 are discontinuous.
  • the boundaries L1, L2 between the discontinuous image contents shown on the right For convenience of description, this type of boundary is referred to as a specific boundary below.
  • triangle 19 and triangle 2 are not adjacent, triangle 2 and triangle 18 are not adjacent, triangle 18 and triangle 0 are not adjacent, triangle 0 and triangle 16 are not adjacent, and triangle 16 and triangle are not adjacent.
  • 1 is not adjacent; therefore, in the rectangular image shown in the right image of FIG. 3, between partition 19 and partition 2, between partition 2 and partition 18, between partition 18 and partition 0, and between partition 0 and partition 16 The image content between partition 16 and partition 1 are not continuous, forming a specific boundary L3.
  • triangle 13 is not adjacent to triangle 6
  • triangle 6 is not adjacent to triangle 4
  • triangle 4 is not adjacent to triangle 3. Therefore, in the rectangular image shown in the right diagram of FIG. 3,
  • the image content between the partition 13_1 and the partition 6_1, between the partition 6_1 and the partition 4_1, and between the partition 4_1 and the partition 3_1 are discontinuous, forming a specific boundary L4 as shown in FIG. 3.
  • the to-be-encoded block is divided into at least one prediction block, and the reference block of the at least one prediction block is searched separately, and the at least one prediction block and the corresponding
  • the differences between the reference blocks are encoded / compressed to reduce redundant information in the encoding / compression.
  • the difference between the prediction block and the reference block may be a residual obtained by subtracting corresponding pixel values of the prediction block and the reference block. Therefore, the smaller the difference between the found reference block and the prediction block, the more redundant information is reduced, and the smaller the amount of data after encoding / compressing the prediction block.
  • the shape of the prediction block is rectangular, and there is a phenomenon that part of the prediction block crosses a specific boundary during encoding and decoding. Therefore, the content of the image on the two sides of a specific boundary in the partial prediction block is discontinuous, resulting in a large difference between the content of the found reference block and the prediction block, which will affect the prediction efficiency and reduce the efficiency and quality of video encoding and decoding.
  • solution one is to extend a specific boundary (or boundary pixel extension).
  • the expansion method is generally to fill some pixels outside a specific boundary, and the filled pixels are usually similar to the boundary content of the image area or are continuous in the image content.
  • the above method can be used to ensure that after the prediction block crosses a specific boundary, it is still possible to obtain continuous pixel information on the image content, so that the prediction efficiency can be improved, and the efficiency and quality of encoding can be improved.
  • the specific boundary as the specific boundary shown in the right figure in FIG. 3 as an example, it can be expanded in the manner shown in FIG. 4.
  • the method in FIG. 5 may include steps 510 to 530. Each step in FIG. 5 is described in detail below.
  • an image to be expanded is acquired.
  • the to-be-expanded image is a planar image formed by splicing a plurality of partitions obtained by mapping a curved surface image by a polyhedron.
  • the embodiment of the present application does not specifically limit the manner of forming the curved surface image to the image to be expanded, and may be implemented by referring to the conventional technology. For example, you can first determine the mapping method of the surface image to the polyhedron, such as tetrahedral mapping, hexahedral mapping, octahedral mapping, icosahedral mapping, and so on. After the polyhedron mapping method is determined, the corresponding mapping method can be used to map the curved surface image to each face of the polyhedron, thereby forming multiple partitions.
  • the mapping information may be written into the code stream, so that the decoding end obtains the mapping information from the code stream.
  • the mapping information can be written into at least one of a sequence header, an image header, a slice header, a video parameter set, a sequence parameter set, an image parameter set, additional enhancement information, and extended data of a code stream.
  • the multiple partitions obtained after the polyhedron mapping may form multiple regions to be expanded, where each region to be expanded may include at least one of the plurality of regions, and the image content in each region to be expanded is continuous.
  • one to-be-expanded area may be composed of one of the partitions, or may be formed by concatenating at least two adjacent partitions in an image to be extended, as long as the image content of the adjacent at least two partitions is continuous Just fine.
  • step 520 the boundaries of the plurality of areas to be expanded are extended outward for one week to obtain a plurality of expanded areas corresponding to the one to one.
  • Each side of each region to be expanded is parallel to the corresponding side of the corresponding expanded region. Further, the shape of each region to be expanded and the corresponding region to be expanded may be the same.
  • each to-be-expanded region and the corresponding to-be-expanded region may be the same and may be replaced by: a plurality of expanded regions (all expanded regions) formed by seamless stitching
  • the target image has the same shape as the image to be expanded.
  • step 520 may be replaced by: each area to be expanded is enlarged proportionally, and a plurality of expanded areas formed after the areas to be expanded having the same area are respectively enlarged proportionally still have the same area.
  • the multiple partitions include at least two right-angled triangular partitions and at least two equilateral-triangular partitions, where each equilateral-triangular partition is enlarged at the same ratio and each right-angled triangular partition is enlarged at the same ratio.
  • step 520 may be replaced by extending the boundary of multiple areas to be expanded outward for one week to obtain a one-to-one correspondence between the multiple areas to be expanded.
  • the area ratios of the expanded regions are equal, and the proportion of the corresponding sides between each expanded region and the corresponding region to be expanded is equal.
  • the foregoing “consistent shape” means that the shapes of the area to be expanded and the corresponding expanded area are similar. For example, if the area to be expanded is an equilateral triangle, the corresponding expanded area is still an equilateral triangle; for another example, if the area to be expanded is a right triangle, the corresponding expanded area is still a right triangle.
  • the expanded area is obtained by scaling up the area to be expanded.
  • step 520 may be replaced by expanding the boundaries of the plurality of regions to be expanded outward for one week to obtain a plurality of expanded regions corresponding to the plurality of regions to be expanded one by one, where the expanded regions and the The area to be expanded satisfies the following relationship: each edge of each area to be expanded is parallel to the corresponding edge of the corresponding expanded area, and each edge of each area to be expanded is corresponding to the corresponding edge of the corresponding expanded area.
  • the horizontal and / or vertical distances are equal.
  • step 530 prediction is performed based on the target image.
  • the target image may be used for intra prediction or inter prediction, which is not limited in this embodiment of the present application.
  • step 530 may include performing intra prediction using pixels of the target image as predicted pixels.
  • step 530 may include performing inter prediction using the target image as a reference image.
  • the specific prediction manner may be performed in a conventional manner. For example, reference may be made to the foregoing description of step 102 in FIG. 1.
  • all to-be-expanded regions in the to-be-expanded image are extended outward for one week, and the area ratio before and after each to-be-expanded region remains unchanged. Therefore, the expanded regions can be spliced together naturally to form a Target images with the same shape as the to-be-expanded image, there is no need to design a special expansion area for each mapping method as in the above solution 1, to ensure that the shape of the image before and after expansion is consistent, so the embodiment of this application improves prediction efficiency At the same time can simplify the implementation of codecs.
  • an area to be expanded may include only one partition or at least two partitions with continuous image content.
  • the above two implementations of the area to be extended are described in detail by taking the regular icosahedron mapping method shown in FIG. 3 as an example.
  • each partition obtained after the mapping can be used as a region to be expanded.
  • each partition can be expanded as shown in FIG. 6.
  • the area enclosed by the dashed line in FIG. 6 represents the area to be expanded, and the area enclosed by the solid line represents the area after expansion.
  • each region to be expanded includes two shapes of an equilateral triangle and a right triangle, and the areas of the equilateral triangles are equal, and the areas of the right triangles are equal.
  • the obtained target image still includes two shapes of an equilateral triangle and a right triangle, and the areas of the equilateral triangles are equal, and the areas of the right triangles are equal.
  • partition 15 is an equilateral triangle, which is still an equilateral triangle after expansion
  • partition 3_1 is a right-angled triangle, which is still a right-angled triangle after expansion.
  • each partition is uniformly expanded, so that the area of the partitions before and after expansion, and the ratio of the length of the sides of the sides remain unchanged.
  • Each partition is treated as a The method of expanding the extended area does not need to determine the position of the specific boundary formed after the polyhedron mapping, which improves the ease of use of the image expansion method and further simplifies the implementation of encoding and decoding.
  • the adjacent multiple partitions in the icosahedron can also be divided into a region to be expanded, as long as the image content of the adjacent multiple partitions is continuous.
  • partition 6_1 and partition 4_1 can be divided into a region to be expanded. After the partitions included in each area to be expanded are determined, the areas to be expanded may be uniformly expanded according to the method described in FIG. 5.
  • the expansion results of partition 6_1 and partition 4_1 are shown in FIG. 7.
  • the difference from the expansion method shown in FIG. 6 is that partition 6_1 and partition 4_1 are expanded as a whole.
  • partition 3_2, partition 19, partition 5, partition 11, partition 7, partition 13_1, partition 18, partition 10, partition 12, partition 14, and partition 16 may be divided into a region to be expanded. In short, no matter which partitions are expanded as a region to be expanded as a whole, the expanded regions can be stitched into a rectangle.
  • each area to be expanded is extended outward for one week to form a corresponding expanded area.
  • pixels near the boundary of the area to be expanded may be directly used as pixels in the filled area.
  • a pixel having a neighboring relationship with a pixel in the region to be expanded may be selected from the curved surface image as a pixel in the filled region.
  • step 520 may include steps 810 and 820.
  • the target to-be-expanded region may be any one of the multiple to-be-expanded regions
  • the target to-be-expanded region may be any one of the multiple to-be-expanded regions
  • the target partition and the partition forming the target to-be-expanded region have two vertices in common (or have common edges) in the mapping region of the polyhedron.
  • step 820 the boundary of the target to-be-expanded area is extended outward by one circle, wherein at least some pixels in the target filling area between the expanded area corresponding to the target to-be-expanded area and the target to-be-expanded area are based on the pixels in the target partition. owned.
  • the spherical surface is mapped by icosahedron to obtain twenty faces (each face is a triangle), and the correspondence between each face and its vertex is shown in Table 1.
  • Number of icosahedron faces Number of vertices that make up the face 0 ⁇ V 8 , V 9 , V 0 ⁇ 1 ⁇ V 2 , V 9 , V 8 ⁇ 2 ⁇ V 0 , V 9 , V 6 ⁇ 3 ⁇ V 7 , V 6 , V 9 ⁇ 4 ⁇ V 6 , V 1 , V 0 ⁇ 5 ⁇ V 11 , V 1 , V 6 ⁇ 6 ⁇ V 4 , V 0 , V 1 ⁇ 7 ⁇ V 10 , V 4 , V 1 ⁇ 8 ⁇ V 0 , V 4 , V 8 ⁇ 9 ⁇ V 5 , V 8 , V 4 ⁇ 10 ⁇ V 3 , V 10 , V 11 ⁇
  • the pixels in the target partition can be used to fill the target filling area completely; or, the pixels in the target partition can be used to fill a part of the target filling area, and the remaining areas can be filled in other ways.
  • the partial area may be a common area of a planar figure spliced by the target partition and the target to-be-expanded area according to a common vertex and the target filling area.
  • the target partitions having two common vertices with the partition 16 can be found as the partition 14, the partition 17 and the partition 18.
  • partitions 14 and 16 have V 2 and V 3 as common vertices; partitions 17 and 16 have V 2 and V 7 as common vertices; partitions 18 and 16 have V 7 and V 3 as common vertices.
  • the partition 14, the partition 16, the partition 17, and the partition 18 can be spliced into a planar figure as shown in FIG.
  • the flat area and the filled area of the partition 16 (the area surrounded by the dashed line is the extended area of the partition 16 and the area between the extended area and the partition 16 is the filled area) (that is, the shadow in FIG. 9)
  • Areas a, b, and c) identified by the lines are the above-mentioned partial areas, and pixels of the target partition can be used for filling.
  • step 820 The following describes the implementation manner of step 820 with reference to FIG. 9.
  • step 820 may include: using pixels in the target partition to fill a part of the target filling area; using default pixels to fill the remaining areas of the target filling area other than the above-mentioned partial area To fill.
  • the pixels in the target partitions 18, 17, and 14 can be used to fill the areas a, b, and c in the area, and the remaining areas d, e, and f can be filled with the default pixels.
  • step 820 may include: using pixels in the target partition to fill a part of the target filling area; and using the periphery of the remaining area in the target filling area except for the above partial area The pixels fill the remaining area.
  • the pixels of the area d, the area e, and the area f can be selected from the surrounding filled areas.
  • FIG. 10 illustrates another possible implementation manner of step 520 in FIG. 5.
  • the region to be expanded includes a first region to be expanded and a second region to be expanded.
  • the first region to be expanded includes a first region
  • the second region to be expanded includes a second region.
  • the first region and the second region are adjacent regions in the image to be expanded.
  • step 520 may Step 1010 includes: interpolating pixels of the first region and the second region to obtain pixels of a filling region between the first region and the second region.
  • the shapes of the first region to be expanded and the second region to be expanded may be, for example, rectangular. Taking FIG. 11 as an example, it is assumed that the surface image is mapped by using a hexahedral mapping method to obtain 6 partitions.
  • the first to-be-expanded region and the second to-be-expanded region may be two adjacent ones of the six regions, as One area to be expanded is the left side in FIG. 11, and the second area to be expanded is the front side in FIG. 11.
  • the width of the first region in the first to-be-expanded region is identified by WH 1 as shown in FIG. 11.
  • the width of the area to be filled to the right of the first area to be expanded is identified by WH 2 as shown in FIG. 11.
  • the second region in the second to-be-expanded region and the region on the left side of the second to-be-expanded region that needs to be filled can be identified in the same manner (not shown in FIG. 11).
  • the filled area between the first area and the second area is an area between the left-hand right dotted line and the front-left-hand dotted line in FIG. 11.
  • the pixels in this region can be obtained by interpolation of the pixels in the first region and the second region.
  • the shapes of the first region to be expanded and the second region to be expanded may also be triangles.
  • the first area to be expanded as partition 3_1 as shown in FIG. 12 the second area to be expanded as partition 4_1 as shown in FIG. 12 as an example, the first area is a right-angled trapezoidal area in partition 3_1, and WH 1 is used for identification.
  • the rectangular trapezoidal area is high; the area above the first partition that needs to be filled by interpolation is a rectangular area, and its width is identified by WH 2 .
  • the remaining areas other than the interpolation area can be filled by other methods mentioned above, such as filling by default pixels.
  • Determining the pixels in the filled area by interpolation can ensure the smoothness of the pixels when the prediction block crosses the partition boundary, thereby improving the coding efficiency.
  • the codec may use the same rules to determine the filling area, and perform pixel filling on the determined filling area.
  • the encoding end may write the extension parameters into the code stream; the decoding end may obtain the extension parameters from the code stream, and then use the extension parameters to expand the area to be extended.
  • the expansion parameter may be used to identify a position of a filling region between each region to be expanded and a corresponding expanded region.
  • the extended parameter may include, for example, one or more of the following parameters: an area parameter and a distance parameter.
  • the area parameter can be used to identify the area ratio of each region to be expanded to the corresponding expanded region.
  • the distance parameter can be used to identify the parallel line distance between the edge of each area to be expanded and the corresponding edge of the corresponding expanded area.
  • the embodiment of the present application does not limit the specific implementation of the distance parameter, and any parameter that can identify the parallel line distance between two sides can be used in this application.
  • the distance parameter may include a horizontal distance parameter and a vertical distance parameter.
  • the horizontal distance parameter can be used to identify the moving distance required for the remaining edges of each extended area except the edge extending in the horizontal direction to move to the corresponding edge of the corresponding extended area in the horizontal direction;
  • the vertical distance parameter can be used to identify each The moving distance required for the side of the extended area extending in the horizontal direction to move to the corresponding side of the corresponding expanded area in the vertical direction.
  • the horizontal and vertical distances can be equal or different. When the horizontal distance and the vertical distance are equal, only one of the distance parameters can be transmitted in the code stream to reduce the complexity of the code stream.
  • neither the side ac nor the side ab of the partition 16 extends in the horizontal direction. Therefore, their corresponding sides a'c 'and a'b' corresponding to the expanded area can be identified by the horizontal distance parameter WH.
  • the side bc of the partition 16 extends in the horizontal direction, so the distance between it and the corresponding side b'c 'of the expanded area can be identified by the vertical distance WV.
  • the horizontal distance parameter WH and the vertical distance parameter WV between the corresponding edges can still be used to identify the filled area.
  • the extended parameter may further include an interpolation area for identifying the pixel (such as the first area or the second area described above).
  • the parameter of the position Still taking FIG. 11 as an example, WH 1 can be used to identify the interpolation pixel area, and horizontal distance parameter WH 2 can be used to identify the filled area.
  • the codec can adjust the above implementation according to the actual situation.
  • the encoder can expand the extended area according to the default or preset expansion parameters and write the expansion parameters into the code stream.
  • the extension parameter is obtained in the stream, and the extension area is extended using the extension parameter.
  • the encoding end or the decoding end may perform the following operations to complete the expansion of the region to be expanded: determining the area ratio of each region to be expanded and the corresponding expanded region according to the area parameter; The area ratio of the expanded area to the corresponding expanded area determines the location of the corresponding expanded area; according to the location of each area to be expanded and the location of the corresponding area to be expanded, the location of each area to be expanded is determined Padding area; pixel filling is performed on the padding area of each to-be-expanded area, so that the boundary of each to-be-expanded area is extended outward by one circle.
  • the embodiments of the present application do not specifically limit the positions of the parameters listed above in the code stream. For example, they can be written in at least one of the following: a sequence header, an image header, a slice header, a video parameter set, and a sequence parameter. Set, image parameter set, additional enhancement information, extended data.
  • the method in FIG. 5 may be executed by a video encoding device or a video decoding device.
  • steps specific to each device may be added to the method of FIG. 5.
  • the method of FIG. 5 may further include storing the target image in a reference image buffer.
  • the method of FIG. 5 may further include storing the target image in a decoded picture buffer (DPB); or, the method of FIG. 5 may further include: storing the image to be expanded.
  • the output image that is, the expanded image is used as a reference frame, the image to be displayed can still be selected as the image to be expanded.
  • the method in FIG. 5 when executed by a video encoding device, the method in FIG. 5 may further include: writing the extension parameter into a code stream.
  • FIG. 14 is a schematic structural diagram of a video processing apparatus according to an embodiment of the present application.
  • the video processing apparatus 1400 of FIG. 14 includes a memory 1410 and a processor 1420.
  • the memory 1410 may be used to store instructions.
  • the processor 1420 may be configured to execute instructions stored in the memory to perform the following operations:
  • the to-be-expanded image is a flat image obtained by splicing a plurality of partitions obtained by mapping a curved surface image by a polyhedron, and a plurality of the partitions form a plurality of regions to be expanded. Including at least one of the plurality of partitions, and the image content in each of the regions to be expanded is continuous;
  • each of the regions to be expanded is composed of one of the partitions.
  • each of the expanded regions is consistent with a shape of a corresponding region to be expanded.
  • the plurality of partitions include a right triangle partition and an equilateral triangle partition, each equilateral triangle partition is enlarged at the same ratio, and each right triangle partition is enlarged at the same ratio.
  • each of the at least one region to be expanded is stitched from at least two of the partitions in which image content is continuous.
  • extending a boundary of a plurality of the regions to be expanded outward for one week includes: for a plurality of target to-be-expanded regions in the plurality of to-be-expanded regions, according to a correspondence relationship between a surface of the polyhedron and a vertex, Selecting a target partition from a plurality of the partitions, wherein the target partition and a partition forming the target to-be-expanded region have two vertices in common in the mapping region of the polyhedron; Outward expansion for one week, wherein at least a part of pixels in the target filling area between the expanded area corresponding to the target to-be-expanded area and the target to-be-expanded area are obtained based on the pixels in the target partition.
  • extending the boundary of the target area to be expanded outward for one week includes: using pixels in the target partition to fill a part of the target filling area; using default pixels to fill the target The remaining areas of the area other than the partial area are filled.
  • extending the boundary of the target to-be-expanded area outward for one week includes: using pixels in the target partition to fill a part of the target filling area; using the target filling area The peripheral pixels of the remaining area other than the partial area fill the remaining area.
  • the partial area is a common area of the target graphics area and the target filling area, which is a planar figure stitched together by the target partition and the target to-be-expanded area according to a common vertex.
  • the plurality of regions to be expanded include a first region to be expanded and a second region to be expanded, the first region to be expanded includes a first region, the second region to be expanded includes a second region, and the The first region and the second region are adjacent regions in the to-be-expanded image, and the step of extending a boundary of the plurality of to-be-expanded regions outward for one week includes: Pixels in the two regions are interpolated to obtain pixels in a filled region between the first region and the second region.
  • the shapes of the first region to be expanded and the second region to be expanded are both rectangular.
  • the video processing device is a video encoding device
  • the processor 1420 is further configured to perform the following operations: write extension parameters into a code stream, where the extension parameters are used to identify each of the regions to be extended The position of the padding area from the corresponding expanded area.
  • the video processing device is a video decoding device, and extending a boundary of a plurality of the regions to be extended outward for one week includes reading extension parameters from a code stream, and the extension parameters are used to identify each The position of the padding area between each of the to-be-expanded areas and the corresponding to-be-expanded areas; and according to the expansion parameter, the boundaries of the plurality of to-be-expanded areas are extended outward by one circle.
  • the extension parameter includes: an area parameter, where the area parameter is used to identify an area ratio of each of the to-be-expanded areas to a corresponding expanded area; and / or a distance parameter, the distance parameter is used for A parallel line distance between an edge of each of the regions to be expanded and a corresponding edge of the corresponding expanded region is identified.
  • the step of expanding the boundaries of a plurality of areas to be expanded outward for one week according to the expansion parameter includes: determining each of the area to be expanded and a corresponding expanded area according to the area parameter. Determine the location of the corresponding expanded area according to the area ratio of each of the areas to be expanded and the corresponding expanded area; according to the location of each of the areas to be expanded, and the corresponding area to be expanded The position of the expanded area determines the filling area of each of the areas to be expanded; pixel filling is performed on the filling area of each of the areas to be expanded, thereby extending the boundary of each of the areas to be expanded outward by one circle.
  • the polyhedron map is a hexahedron map, an octahedron map, or an icosahedron map.
  • At least one of the partitions is non-rectangular.
  • At least one of the partitions is triangular or diamond-shaped.
  • At least one of the partitions is a regular triangle or a right triangle.
  • the curved surface image is part or all of a frame image in a panoramic video.
  • the image to be expanded and the target image are both rectangular images.
  • the video processing device is a video encoding device
  • the processor 1420 is further configured to perform the following operation: storing the target image in a reference image buffer.
  • the video processing device is a video decoding device
  • the processor 1420 is further configured to perform the following operation: storing a target image in a decoded image buffer.
  • the video processing device 1400 is a video decoding device, and the processor 1420 is further configured to perform the following operation: using the image to be expanded as an output image.
  • the performing prediction based on the target image includes performing intra prediction using pixels of the target image as prediction pixels.
  • the performing prediction based on the target image includes performing inter prediction using the target image as a reference image.
  • the computer program product includes one or more computer instructions.
  • the computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.
  • the computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be from a website site, computer, server, or data center Transmission by wire (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) to another website site, computer, server, or data center.
  • the computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, a data center, and the like that includes one or more available medium integration.
  • the usable medium may be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a digital video disc (DVD)), or a semiconductor medium (for example, a solid state disk (SSD)), etc. .
  • a magnetic medium for example, a floppy disk, a hard disk, a magnetic tape
  • an optical medium for example, a digital video disc (DVD)
  • DVD digital video disc
  • SSD solid state disk

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

提供一种视频处理方法及相关装置。该方法包括获取待扩展图像，待扩展图像是由曲面图像经多面体映射之后得到的多个分区拼接而成的平面图像，多个分区共形成多个待扩展区域，每个待扩展区域包括多个分区中的至少一个分区，且每个待扩展区域中的图像内容连续；将多个待扩展区域的边界向外扩展一周，得到与多个待扩展区域一一对应的多个扩展后的区域，每个待扩展区域的每条边与相应的扩展后的区域的对应边相互平行，且每个待扩展区域与相应的扩展后的区域的形状相同；根据多个扩展后的区域拼接而成的目标图像进行预测。采用均匀扩展方式对待扩展区域进行统一地扩展，可以提高预测效率，简化编码实现。

Description

视频处理方法及相关装置 技术领域
本申请涉及图像处理领域，并且更具体地，涉及一种视频处理方法及相关装置。
背景技术
近年来，由于各种类型的便携式、手持式或者可穿戴设备的盛行，视频的内容量一直在增长。例如，虚拟现实或者增强现实可以集成到不同的头戴式设备(HMD)中。随着视频内容的形式变得越来越复杂，视频内容的存储和传输变得越来越具有挑战性。例如，需要减少视频存储和传输的带宽。
发明内容
本申请提供一种视频处理方法和相关装置。
第一方面，提供一种视频处理方法，包括：获取待扩展图像，所述待扩展图像是由曲面图像经多面体映射之后得到的多个分区拼接而成的平面图像，多个所述分区共形成多个待扩展区域，每个所述待扩展区域包括所述多个分区中的至少一个分区，且每个所述待扩展区域中的图像内容连续；将多个所述待扩展区域的边界向外扩展一周，得到与多个所述待扩展区域一一对应的多个扩展后的区域，其中每个待扩展区域的每条边与相应的扩展后的区域的对应边相互平行，且每个所述待扩展区域与相应的待扩展后的区域的形状相同；根据所述目标图像进行预测。
第二方面，提供一种视频处理装置，包括：存储器，用于存储指令；处理器，用于执行所述存储器中存储的指令，以执行以下操作：获取待扩展图像，所述待扩展图像是由曲面图像经多面体映射之后得到的多个分区拼接而成的平面图像，多个所述分区共形成多个待扩展区域，每个所述待扩展区域包括所述多个分区中的至少一个分区，且每个所述待扩展区域中的图像内容连续；将多个所述待扩展区域的边界向外扩展一周，得到与多个所述待扩展区域一一对应的多个扩展后的区域，其中每个待扩展区域的每条边与相应的扩展后的区域的对应边相互平行，且每个所述待扩展区域与相应的待扩展后的区域的形状相同；根据所述目标图像进行预测。
第三方面，提供一种机器可读存储介质，所述机器可读存储介质中存储有用于执行如第一方面所述的方法的指令。
第四方面，提供一种计算机程序产品，所述计算机程序产品包括用于执行如第一方面所述的方法的指令。
附图说明
图1为一个曲面视频的编码/压缩流程示意图。
图2中左图是球面图像与外切正八面体的映射示意图，右图是左图中的正八面体表面上的多个三角形合并成一个二维的矩形图像的一种实施例的示意图。
图3左图是球面图像与外切正二十面体的映射示意图，右图是将左图中的正二十面体表面上的多个三角形合并成一个二维的矩形图像的一种实施例的示意图。
图4是特定边界的一种扩展方式的示意图。
图5是本申请实施例提供的视频处理方法的示意性流程图。
图6是本申请一个实施例提供的边界扩展方式的示意图。
图7是本申请另一实施例提供的边界扩展方式的示意图。
图8是图5中的步骤520的一种实现方式的示意性流程图。
图9是本申请一个实施例提供的填充区域的填充方式的示例图。
图10是图5中的步骤520的另一具体实现方式的示意性流程图。
图11是本申请另一实施例提供的填充区域的填充方式的示例图。
图12是本申请又一实施例提供的填充区域的填充方式的示例图。
图13是本申请实施例提供的填充区域的标识方式的示例图。
图14是本申请实施例提供的视频处理装置的示意性结构图。
具体实施方式
在本文的各个实施例中，各装置或者系统能够减小存储和传输一个曲面视频所占用的带宽。在一些实施例中，该曲面视频可以是在任何平滑的表面(例如球面或者椭球面的至少部分)上投影的画面。一个曲面视频包括多帧曲面图像，其中该曲面图像中的内容包括在同一时刻不同方向上拍摄到的内容。因此，一个曲面视频能够覆盖一个较大的视场角。例如，一个球面视频 (或者360度全景视频)包括多帧可以覆盖一个360度全景的视场角的球面图像。在一些实施例中，一个球面视频可以具有360度的水平视场角，以及180度的垂直视场角，通常以三维球面形式呈现。下文中，以球面视频为例来对曲面视频进行描述。本领域技术人员可以明显确定，曲面视频并不局限于球面视频。
图1为一个曲面视频的编码/压缩流程示意图。如图1所示，曲面视频的编码/压缩流程包括以下几个步骤：映射101、预测102、变换103、量化104和熵编码105。
在映射步骤101中，编码装置将三维的曲面视频映射成二维视频，以能够利用各种编码/压缩技术。编码装置可以采用二维矩形格式来存储和传输该曲面视频。另外，编码装置可以采用二维矩形格式来支持数字图像处理以及编解码操作。
曲面视频的映射方法有多种。例如，将曲面视频映射成二维视频。以球面视频为例，在一些实施方式中，球面视频中的球面图像可以基于经纬图投影(equirectangular projection)的方法映射成二维图像。在经纬图投影的一些实施方式中，可以将球面上的各经线映射到二维平面上等间隔的多个垂直直线，并将各纬线映射到该二维平面上等间隔的多个水平直线。
在一些实施方式中，球面视频中的球面图像可以基于多面体投影的方法映射成二维图像。具体的，将球面的不同区域上的图像分别投影到该球面的外切多面体的表面的多个多边形上，再将该多个多边形上的二维图像合并成一个二维图像。在一个实施例中，投影的方法可以是从球心发出虚拟射线依次穿过球面上的一点和多面体表面上的一点，将球面上该点的像素信息投影到多面体表面的该点上。
其中，多面体可以是任意多面体。例如，多面体可以是六面体、八面体、二十面体或者其他等等。其中，该六面体可以是立方体、长方体、菱形六面体、平行六面体等等；该八面体可以是正八面体、六角柱、三角反棱柱等等；该二十面体可以是正二十面体、正九角反棱柱等等，在此不做限制。
将多面体上的多个多边形合并成一个二维图像的方法有多种。在一些实施方式中，将多面体表面上的多个多边形合并成一个二维的矩形。合并的方式也有多种，例如不同的合并方式中各多边形之间的位置关系不同；又例如，不同的合并方式中合并成的矩形的大小/形状不同。
如图2所示，图2中左图是球面图像与外切正八面体的映射示意图，右图是左图中的正八面体表面上的多个三角形合并成一个二维的矩形图像的一种实施例的示意图。图2所示的正八面体表面上的每个三角形和矩形图像中的每个三角形中都标示有对应的数字，用于指示矩形图像中的各三角形与正八面体表面上的各三角形之间的对应关系。其中，图2所示矩形图像中标号为3_1和3_2的两个直角三角形指的是正八面体中标号为3的三角形被切割成为标号为3_1和3_2的两个直角三角形；其他类似标号同理。可选的，矩形图像中部分共用一条边的两个三角形在正八面体表面上也是共用同一条边，以使得矩形图像中该两个三角形之间的图像内容存在连续，可以提高编码效率。图2中右图所示矩形只是正八面体上的多个三角形的合并方式的一个示例，还可以有其他合并成二维的矩形图像的方式，例如可以改变图2右图中各三角形之间的位置关系，或者改变矩形图像的长度和宽度，在此不做限制。
如图3所示，图3左图是球面图像与外切正二十面体的映射示意图，右图是将左图中的正二十面体表面上的多个三角形合并成一个二维的矩形图像的一种实施例的示意图。图3所示的正二十面体和矩形图像中的每个三角形都标示有对应的数字，以指示矩形图像中的各三角形与正八面体表面上的各三角形之间的对应关系。其中，图3所示矩形图像中标号为3_1和3_2的两个直角三角形指的是正二十面体中标号为3的三角形被切割成为标号为3_1和3_2的两个直角三角形；其他类似标号同理。可选的，图3左图所示矩形图像中部分共用一条边的两个三角形在右图所示的正二十面体表面上也是共用同一条边，以使得矩形图像中该两个三角形之间的图像内容存在连续，可以提高编码效率。图3右图所示矩形只是正二十面体上的多个三角形的合并方式的一个示例，还可以有其他合并成矩形的方式，例如可以改变图3右图中各三角形之间的位置关系，或者改变矩形图像的长度和宽度，在此不做限制。
除了如上所述的经纬图投影和多面体投影之外，还可以利用其他投影机制来将球面视频映射成二维视频。所映射成的二维视频可以是矩形视频，也可以是其他形状(例如圆形、椭圆形、正多边形)的视频。在一些实施方式中，曲面视频也可以不是球面视频，而是其他三维曲面视频，例如球面上的一部曲面视频(如半球面视频)、椭球面视频、椭球面上的一部分曲面(如 半椭球面视频)、其他规则的或不规则的三维曲面视频。将球面视频映射成二维视频的方法也可以适用于非球面视频的三维曲面视频映射成二维视频。基于一些常用的视频编解码器标准，例如高效率视频编码HEVC/H.265，H.264/AVC，AVS1-P2，AVS2-P2，VP8，VP9，可以对二维视频进行压缩，编码和解码。
在一些实施方式中，编码装置还可以将用于指示映射方法的映射信息写入码流中。例如，该映射信息可以写入以下至少一项中：序列头(sequence header)、图像头(picture header)、条带头(slice header)、视频参数集(video parameter set，VPS)、序列参数集(sequence parameter set，SPS)、图像参数集(picture parameter set，PPS)、附加增强信息(supplemental enhancement information，SEI)、扩展数据(extension data)。
为描述方便，下文中称映射后的二维图像中包括多个分区，其中每个分区对应于多面体表面上的一个多边形，例如，图2和图3右图中的矩形图像中每个三角形内的图像作为一个分区。需注意的是，多面体上的多个多边形合并成一个二维的矩形时，可能会对多面体上的部分多边形进行分割，将一个多边形分割成至少两个多边形。例如图2左图中的三角形7在右图的矩形中被分割成了两个三角形(三角形7_1和三角形7_2)；图3左图中的三角形4在右图的矩形中被分割成了两个三角形(三角形4_1和三角形4_2)。在这种多面体上的一个多边形对应二维图像中的至少两个多边形的情形中，二维图像中的该至少两个多边形中每个多边形内的图像分别作为一个分区。例如图2右图的矩形图形中三角形7_1为一个分区，三角形7_2作为另一个分区。
在一些实施方式中，映射后的二维图像中的至少一个分区的形状可以是规则的和/或不规则的。在一些实施方式中，二维图像中的至少一个分区的形状可以包括矩形、三角形、菱形、圆形、五边形、六边形或者其他形状。其中，该三角形可以包括正三角形、直角三角形、等腰三角形中的至少一种。
预测步骤102用于减少图像中的冗余信息。预测块指的是进行预测的基本单元，在一些标准中，该预测块也称为预测单元(Prediction Unit,PU)。在对一帧图像进行编码/压缩之前，图像被分成多个图像块，进一步的，该多个图像块中的每一个图像块可以再次被分成多个图像块，以此类推。不同的编码方法中，分割的层级数量可以不同，所承担的操作方法也不同。不同的编码标准中，对同一层级上的图像块的名称可能不同。例如，在一些视频标 准中，一帧图像第一次被分割成的多个图像块中的每个图像块称为编码树单元(coding tree unit，CTU)；每个编码树单元可以包含一个编码单元(coding unit，CU)或者再次分割成多个编码单元；一个编码单元可以根据预测方式分割成一个、两个、四个或者其他数量的预测单元。在一些视频标准中，该编码树单元也被称为最大编码单元(largest coding unit，LCU)。在现有的一些视频标准中，预测单元是图像中最小的单元，预测单元不会继续被划分成多个图像块。
预测指的是查找与该预测块相似的图像数据，也称为该预测块的参考块。通过对该预测块和该预测块的参考块之间的差异进行编码/压缩，以减少编码/压缩中的冗余信息。其中，预测块与参考块的差异可以是由该预测块与该参考块的相应像素值相减得到的残差。预测包括帧内预测和帧间预测。帧内预测指的是在预测块所在帧内查找该预测块的参考块，帧间预测指的是在除预测块所在帧以外的其他帧内查找该预测块的参考块。
变换步骤103可以进一步减少图像块中的冗余信息。在变换步骤103中，预测块与该预测块的参考块之间的残差可以使用变换矩阵去除该残差的相关性，提高编码效率。预测块的变换通常采用二维变换，即在编码装置中将预测块块的残差信息分别与一个N*M的变换矩阵及其转置矩阵相乘，相乘之后得到的是变换系数。该变换系数可以在量化步骤104中经量化得到量化后的系数。熵编码步骤105中将量化后的系数进行熵编码。最后将熵编码得到的比特流及进行编码后的编码模式信息，如帧内预测模式、运动矢量信息等，进行存储或发送到解码装置。
解码装置接收到数据后，执行和编码装置中的编码/压缩流程相反的流程，例如依次执行以下步骤：熵解码、反量化、反变换、预测、映射。其中，在预测步骤中，解码装置可以采用和上文描述的预测方法相同的方法进行预测。在映射步骤中，解码装置采用和上面描述的映射方法相反的方法进行映射，也即编码装置中是将曲面图像映射到多面体的表面上后，将多面体表面上的多个多边形合并成一个二维平面图像，而解码装置中是将二维平面图像映射到曲面图像上，在一个实施例中，解码装置可以先将二维平面图像中的多个分区拆分后拼凑成一个多面体的至少部分，再将该多面体的至少部分上的图像映射到曲面上，形成曲面图像。
在上述映射步骤101中，三维曲面视频映射成二维视频时，会出现二维 视频中两个相邻分区内容不连续的情况。例如，在图2左图中，三角形7和三角形4不相邻，三角形5和三角形6不相邻，三角形3和三角形0不相邻，三角形1和2不相邻；因此，在图2右图所示的矩形图像中，分区7_1和分区4之间、分区3_1和分区0之间、分区5和分区6之间、分区1和分区2之间的图像内容均不连续，形成了如图2右图所示的不连续图像内容之间的边界L1、L2，为了便于描述，下文称该这种类型的边界为特定边界。
又如，在图3左图中，三角形19和三角形2不相邻，三角形2和三角形18不相邻，三角形18和三角形0不相邻，三角形0和三角形16不相邻，三角形16和三角形1不相邻；因此，在图3右图所示的矩形图像中，分区19和分区2之间，分区2和分区18之间，分区18和分区0之间，分区0和分区16之间，分区16和分区1之间的图像内容均不连续，形成了特定边界L3。此外，在图3左图中，三角形13与三角形6不相邻，三角形6与三角形4不相邻，三角形4与三角形3不相邻；因此，在图3右图所示的矩形图像中，分区13_1和分区6_1之间，分区6_1和分区4_1之间，分区4_1和分区3_1之间的图像内容均不连续，形成了如图3所示的特定边界L4。
在对待编码图像中的一个待编码块进行编码/压缩时，该待编码块被分为至少一个预测块，通过分别查找该至少一个预测块的参考块，并分别对该至少一个预测块和对应的参考块之间的差异进行编码/压缩，以减少编码/压缩中的冗余信息。其中，预测块与参考块的差异可以是由该预测块与该参考块的相应像素值相减得到的残差。因此，查找到的参考块与预测块的差异越小，减少的冗余信息越多，对该预测块编码/压缩后的数据量越小。然而，传统的曲面视频编码，预测块的形状为矩形的，编解码时会出现部分预测块横跨特定边界的现象。因此该部分预测块中分别位于特定边界两侧的图像内容不连续，导致查找到的参考块与该预测块的内容差异较大，从而会影响预测效率，降低视频编解码的效率和质量。
为了解决该问题，一种可能的解决方案(下称解决方案一)是对特定边界进行扩展(或称边界像素扩展)。扩展方式通常是在特定边界外部填充一些像素，该填充像素通常与所述的图像区域的边界内容相似或在图像内容上是连续的。上述方式可用以保证预测块横跨特定边界之后，仍可能获得在图像内容上连续的像素信息，从而可以提高预测效率，进而可以提升编码的效率和质量。以特定边界为图3右图所示的特定边界为例，可以采用图4所示 的方式进行扩展。从图4可以看出，特定边界两侧的图像被相互隔开，形成需要填充像素的填充区域(或称像素扩展区域)，且填充区域的形状设计使得每一个扩展后的区域与扩展前的区域的形状保持一致(例如区域在扩展前为三角形图像，则扩展后的区域为三角形图像)，且多个扩展后的区域仍然可以拼接成一个矩形图像。
上述解决方案一的问题在于，每种映射格式形成的特定边界并不相同，因此，如果采用解决方案一，则需要针对每种映射格式设计填充区域的形状，使得扩展前后的图像的整体形状保持一致。由此可见，解决方案一实现起来比较复杂。下面结合图5，详细描述本申请实施例提供的视频处理方法，以简化图像的扩展方式，从而简化编码实现。
图5的方法可以包括步骤510至步骤530，下面对图5中的各个步骤进行详细说明。
在步骤510，获取待扩展图像。该待扩展图像是由曲面图像经多面体映射之后得到的多个分区拼接而成的平面图像。
本申请实施例对曲面图像到待扩展图像的形成方式不做具体限定，可以参照传统技术实现。例如，可以先确定曲面图像到多面体的映射方式，如四面体映射、六面体映射、八面体映射、二十面体映射等。在多面体映射方式确定之后，可以采用相应的映射方式将曲面图像映射至多面体的各个面，从而形成多个分区。此外，在编码端，可以将映射信息写入码流，以便解码端从码流获取该映射信息。该映射信息可以写入码流的序列头、图像头、条带头、视频参数集、序列参数集、图像参数集、附加增强信息、扩展数据中的至少一个中。
多面体映射之后得到的多个分区可以形成多个待扩展区域，其中每个待扩展区域可以包括该多个分区中的至少一个分区，且每个待扩展区域中的图像内容连续。
换句话说，一个所述待扩展区域可以由一个所述分区组成，也可以由待扩展图像中的相邻的至少两个分区拼接而成，只要该相邻的至少两个分区的图像内容连续即可。
在步骤520中，将多个待扩展区域的边界向外扩展一周，得到与多个待扩展区域一一对应的多个扩展后的区域。其中，每个待扩展区域的每条边与相应的扩展后的区域的对应边相互平行。进一步地，每个待扩展区域与相应 的待扩展后的区域的形状可以相同。
可选地，在某些实施例中，每个待扩展区域与相应的待扩展后的区域的形状可以相同可以替换为：多个扩展后的区域(所有扩展后的区域)无缝拼接形成的目标图像与所述待扩展图像的形状相同。
在某些实施例中，步骤520可替换为：每个待扩展区域等比例放大，且具有相同面积的多个待扩展区域分别进行等比例放大之后形成的多个扩展后区域仍具有相同面积。例如，多个分区包括至少两个直角三角形分区和至少两个等边三角形分区，其中，各个等边三角形分区以相同的比例进行等比例放大，且各个直角三角形分区以相同的比例进行等比例放大。
在某些实施例中(如每个待扩展区域仅包括一个分区的实施例)，步骤520可替换为将多个待扩展区域的边界向外扩展一周，得到多个待扩展区域一一对应的多个扩展后的区域，其中扩展后的区域与待扩展区域满足如下关系中的至少两种：每个扩展后的区域与相应待扩展区域的形状一致，不同扩展后的区域与各自对应的待扩展区域的面积比相等，每个扩展后的区域与相应待扩展区域之间的对应边的比例相等。
应理解，上述“形状一致”指的是待扩展区域和相应的扩展后的区域的形状为相似形。例如，待扩展的区域为等边三角形，则相应的扩展后的区域仍为等边三角形；又如，待扩展的区域为直角三角形，则相应的扩展后的区域仍为直角三角形。例如，扩展后的区域是由待扩展区域进行等比例放大得到的。
或者，在某些实施例中，步骤520可替换为将多个待扩展区域的边界向外扩展一周，得到多个待扩展区域一一对应的多个扩展后的区域，其中扩展后的区域与待扩展区域满足如下关系：每个待扩展区域的每条边与相应的扩展后的区域的对应边相互平行，且每个待扩展区域的每条边与相应的扩展后的区域的对应边的水平距离和/或垂直距离相等。
在步骤530中，根据目标图像进行预测。
目标图像可用于帧内预测，也可用于帧间预测，本申请实施例对此并不限定。例如，步骤530可包括：将目标图像的像素作为预测像素进行帧内预测。又如，步骤530可包括将目标图像作为参考图像进行帧间预测。具体的预测方式可以按照传统方式进行，例如可以参照前文对图1中的步骤102的描述进行。
本申请实施例将待扩展图像中的所有待扩展区域均向外扩展了一周，且各待扩展区域扩展前后的面积比保持不变，因此，扩展后的区域能够自然地拼接在一起，形成一幅与待扩展图像形状一致的目标图像，无需像上述解决方案一那样为每种映射方式设计专门的扩展区域，以确保扩展前后的图像的形状保持一致，因此，本申请实施例在提高预测效率的同时能够简化编解码实现。
上述已经指出，一个待扩展区域可以仅包括一个分区，也可以包括图像内容连续的至少两个分区。下面以如图3所示的正二十面体映射方式为例对待扩展区域的上述两种实现方式进行详细说明。
当按照如图3所示的方式进行多面体映射之后，映射后得到的每个分区可以作为一个待扩展区域。接着，可以按照如图6所示的方式对各个分区进行扩展。图6中的虚线所围成的区域表示的是待扩展区域，实线所围成的区域表示的是扩展后的区域。从图6可以看出，扩展前，各待扩展区域包括等边三角形和直角三角形两种形状，且各等边三角形的面积相等，各直角三角形的面积相等。各待扩展区域进行扩展之后，得到的目标图像仍包括等边三角形和直角三角形两种形状，且各等边三角形的面积相等，各直角三角形的面积相等。例如，分区15为等边三角形，扩展后仍为等边三角形；分区3_1为直角三角形，扩展后仍为直角三角形。
曲面图像经多面体映射之后会得到多个分区，本申请实施例对每个分区进行统一的扩展，使得扩展前后分区的面积、各边的边长比例关系保持不变，将每个分区作为一个待扩展区域进行扩展的方式无需确定多面体映射之后形成的特定边界的位置，提高了图像扩展方式的易用性，进一步简化了编解码的实现。
可选地，也可以将二十面体中的相邻的多个分区划分成一个待扩展区域，只要该相邻的多个分区的图像内容连续即可。参见图3，可以将分区6_1和分区4_1划分在一个待扩展区域。在确定每个待扩展区域所包含的分区之后，按照图5描述的方法对各待扩展区域进行统一的扩展即可。分区6_1和分区4_1的扩展结果如图7所示，与图6所示的扩展方式的区别在于分区6_1和分区4_1作为一个整体被扩展。又如，可以将分区3_2、分区19、分区5、分区11、分区7、分区13_1、分区18、分区10、分区12、分区14以及分区16划分在一个待扩展区域。总之，无论将哪些分区作为一个待扩展区域 进行整体扩展，扩展后的各个区域均可以拼接成一个矩形。
步骤520将每个待扩展区域向外扩展了一周，形成了相应的扩展后的区域。扩展后的区域与待扩展区域之间的填充区域中的像素的选取方式可以有多种，本申请实施例对此并不限定。例如，可以直接将待扩展区域的边界附近的像素作为填充区域中的像素。又如，可以从曲面图像中选取与该待扩展区域中的像素具有相邻关系的像素作为填充区域中的像素。下面结合图8和图10，给出步骤520的两种可能的实现方式。
如图8所示，步骤520可以包括步骤810和步骤820。
在步骤810，针对多个待扩展区域中的目标待扩展区域(目标待扩展区域可以为多个待扩展区域中的任意一个待扩展区域)，根据多面体的面与顶点的对应关系，从多个分区中选取目标分区。
目标分区与形成目标待扩展区域的分区在多面体的映射区域具有共同的两个顶点(或具有公共边)。
在步骤820，将目标待扩展区域的边界向外扩展一周，其中目标待扩展区域对应的扩展后的区域与目标待扩展区域之间的目标填充区域中的至少部分像素是基于目标分区中的像素得到的。
仍以图3所示的映射方式为例，球面经二十面体映射后得到二十个面(每个面为一个三角形)，每一个面与其顶点的对应关系如表1所示。
表1
二十面体的面的编号 组成面的顶点的编号
0 {V 8,V 9,V 0}
1 {V 2,V 9,V 8}
2 {V 0,V 9,V 6}
3 {V 7,V 6,V 9}
4 {V 6,V 1,V 0}
5 {V 11,V 1,V 6}
6 {V 4,V 0,V 1}
7 {V 10,V 4,V 1}
8 {V 0,V 4,V 8}
9 {V 5,V 8,V 4}
10 {V 3,V 10,V 11}
11 {V 1,V 11,V 10}
12 {V 10,V 3,V 5}
13 {V 4,V 10,V 5}
14 {V 2,V 5,V 3}
15 {V 8,V 5,V 2}
16 {V 3,V 7,V 2}
17 {V 9,V 2,V 7}
18 {V 11,V 7,V 3}
19 {V 6,V 7,V 11}
以目标待扩展区域为面16映射成的分区16为例，由于面16由顶点V 3、V 7、V 2组成。由上表可知面16与面18具有共同的两个顶点V 3、V 7(或具有公共边V 3-V 7)、面16与面17具有共同的两个顶点V 2、V 7(或具有公共边V 2-V 7)、面16与面14具有共同的两个顶点V 2、V 3(或具有公共边V 2-V 3)，这表明分区16与分区18,17,14的图像内容连续。因此，可以将分区18,17,14中的一个或多个分区作为上述目标分区，并利用该目标分区中的像素对分区16进行扩展。
可选地，可以使用目标分区的像素将目标填充区域填充完整；或者，可以使用目标分区的像素填充目标填充区域中的部分区域，剩余区域可以采用其他方式进行填充。
上述部分区域的定义方式可以有多种。例如，该部分区域可以为将目标分区与目标待扩展区域按照共同的顶点拼接成的平面图形与所述目标填充区域的公共区域。以图9为例，根据分区16的顶点V 3、V 2、V 7，通过查上述表一，可以找到与分区16具有两个共同顶点的目标分区为分区14，分区17和分区18。其中，分区14与分区16以V 2和V 3为共同的顶点；分区17与分区16以V 2和V 7为共同的顶点；分区18与分区16以V 7和V 3为共同的顶点。按照共同的顶点，可以将分区14、分区16、分区17和分区18按照如图9所示的方式拼接成平面图形。该平面图形与分区16的填充区域(虚线所围的区域为分区16的扩展后的区域，该扩展后的区域与分区16之间的区域为填充区域)的公共区域(即图9中的阴影线所标识的区域a、区域b以及区域c)即为上述部分区域，可以采用目标分区的像素进行填充。
下面结合图9对步骤820的实现方式进行描述。
可选地，作为一种实现方式，步骤820可包括：利用目标分区中的像素对目标填充区域中的部分区域进行填充；利用默认像素对目标填充区域中的除上述部分区域之外的剩余区域进行填充。以图9为例，可以分别利用目标分区18,17,14中的像素目标填充区域中的区域a，区域b和区域c，使用默认像素填充剩余的区域d，区域e，区域f。
可选地，作为另一种实现方式，步骤820可包括：利用目标分区中的像素对目标填充区域中的部分区域进行填充；利用目标填充区域中的除上述部分区域之外的剩余区域的周边像素对该剩余区域进行填充。仍以图9为例，区域d，区域e，区域f的像素可以从周围的已填充区域中选取。
图10示出的是图5中的步骤520的另一种可能的实现方式。具体地，假设待扩展区域包括第一待扩展区域和第二待扩展区域。第一待扩展区域包括第一区域，第二待扩展区域包括第二区域，第一区域和所述第二区域为所述待扩展图像中的相邻区域，如图10所示，步骤520可包括步骤1010：对第一区域和第二区域的像素进行插值，得到第一区域和第二区域之间的填充区域的像素。
第一待扩展区域和第二待扩展区域的形状例如可以是矩形。以图11为例，假设采用六面体映射方式对曲面图像进行映射，得到6个分区，第一待扩展区域和第二待扩展区域可以是该6个分区中的两个相邻的分区，如第一待扩展区域为图11中的左面，第二待扩展区域为图11中的前面。第一待扩展区域中的第一区域的宽度由如图11所示的WH 1所标识。第一待扩展区域的右侧的需要被填充的区域的宽度由如图11所示的WH 2所标识。第二待扩展区域中的第二区域和第二待扩展区域的左侧的需要被填充的区域可以采用相同的方式进行标识(图11中未示出)。
第一区域和第二区域之间的填充区域为图11中的左面的右侧虚线与前面的左侧虚线之间的区域。该区域的像素可以利用第一区域和第二区域中的像素的插值得到。
可选地，第一待扩展区域和第二待扩展区域的形状也可以为三角形。以第一待扩展区域为如图12所示的分区3_1，第二待扩展区域为如图12所示的分区4_1为例，第一区域为分区3_1中的直角梯形区域，WH 1用于标识该直角梯形区域的高；第一分区上侧的需要利用插值方式填充的区域为矩形区域，其宽度通过WH 2标识。除了插值区域之外的剩余区域可以采用上文提 及的其他方式进行填充，如采用默认像素进行填充。
采用插值的方式确定填充区域中的像素可以保证预测块跨过分区边界时的像素平滑性，从而提高编码效率。
下文结合具体的实施例，详细描述填充区域的确定和/或标识方式。
可选地，作为一种实现方式，编解码端可以采用相同的规则确定填充区域，并对确定出的填充区域进行像素填充。
可选地，作为另一种实现方式，编码端可以将扩展参数写入码流；解码端可以从码流中获取扩展参数，然后利用扩展参数对待扩展区域进行扩展。
该扩展参数可用于标识每个待扩展区域与相应扩展后的区域之间的填充区域的位置。该扩展参数例如可以包括以下参数中的一种或多种：面积参数和距离参数。其中，面积参数可用于标识每个待扩展区域与相应的扩展后的区域的面积比。距离参数可用于标识每个待扩展区域的边与相应扩展后的区域的对应边之间的平行线距离。本申请实施例对距离参数的具体实现方式不做限定，能够标识出两条边之间的平行线距离的任意参数均可用于本申请。
作为一个示例，该距离参数可以包括水平距离参数和垂直距离参数。水平距离参数可用于标识每个扩展区域的除沿水平方向延伸的边之外的剩余边沿水平方向移动至相应的扩展后的区域的对应边所需的移动距离；垂直距离参数可用于标识每个扩展区域的沿水平方向延伸的边沿垂直方向移动至相应的扩展后的区域的对应边所需的移动距离。水平距离和垂直距离可以相等也可以不等。当水平距离和垂直距离相等时，可以仅在码流中传输其中一个距离参数，以降低码流的复杂度。
如果13所示，分区16的边ac和边ab均不沿水平方向延伸，因此，它们与扩展后的区域的对应边a’c’和a’b’可以采用水平距离参数WH进行标识。分区16的边bc沿水平方向延伸，因此，它与扩展后的区域的对应边b’c’之间的距离可以采用垂直距离WV进行标识。
同理，以待扩展区域包括图6中的分区6_1和分区4_1为例，如图7所示，仍可采用对应边之间的水平距离参数WH和垂直距离参数WV来标识填充区域。
需要说明的是，如果填充区域的像素是采用如图11所示的插值方式获得的，则扩展参数还可以包括用于标识像素的插值区域(如上文描述的第一 区域或第二区域)的位置的参数。仍以图11为例，可以采用WH 1对插值像素区域进行标识，并采用水平距离参数WH 2对填充区域进行标识。
编解码端可以根据实际情况对上述实现方式进行调整，例如，编码端可以按照默认的或预先设定的扩展参数对待扩展区域进行扩展，并将扩展参数写入码流中；解码端可以从码流中获取该扩展参数，并利用该扩展参数对待扩展区域进行扩展。
在某些实施例中，编码端或解码端可以执行以下操作，以完成待扩展区域的扩展：根据面积参数，确定每个待扩展区域与相应的扩展后的区域的面积比；根据每个待扩展区域与相应的扩展后的区域的面积比，确定相应的扩展后的区域的位置；根据每个待扩展区域的位置，以及相应的待扩展后的区域的位置，确定每个待扩展区域的填充区域；对每个待扩展区域的填充区域进行像素填充，从而将每个待扩展区域的边界向外扩展一周。
本申请实施例对上文列举的参数在码流中的位置不做具体限定，例如，可以写入码流的以下至少一项中：序列头、图像头、条带头、视频参数集、序列参数集、图像参数集、附加增强信息、扩展数据。
图5的方法可以由视频编码装置执行，也可以由视频解码装置执行。当图5的方法由不同装置执行时，可以在图5方法的基础上添加各装置特有的步骤。
例如，当图5的方法由视频编码装置执行时，图5的方法还可以包括将目标图像存储于参考图像缓冲区。当图5的方法由视频解码装置执行时，图5的方法还可以包括将目标图像存储于解码图像缓冲区(decoded picture buffer，DPB)；或者，图5的方法还可包括：将待扩展图像作为输出图像，即扩展后的图像作为参考帧使用，用于显示输出的图像仍然可以选用该待扩展图像。
又如，当图5的方法由视频编码装置执行时，图5的方法还可包括：将扩展参数写入码流中。
上文结合图1至图13，详细描述了本申请的方法实施例，下面结合图13，详细描述本申请的装置实施例。应理解，方法实施例的描述与装置实施例的描述相互对应，因此，未详细描述的部分可以参见方法装置实施例。
图14是本申请实施例提供的视频处理装置的示意性结构图。图14的视频处理装置1400包括存储器1410和处理器1420。
存储器1410可用于存储指令。处理器1420可用于执行所述存储器中存储的指令，以执行以下操作：
获取待扩展图像，所述待扩展图像是由曲面图像经多面体映射之后得到的多个分区拼接而成的平面图像，多个所述分区共形成多个待扩展区域，每个所述待扩展区域包括所述多个分区中的至少一个分区，且每个所述待扩展区域中的图像内容连续；
将多个所述待扩展区域的边界向外扩展一周，得到与多个所述待扩展区域一一对应的多个扩展后的区域，其中每个待扩展区域的每条边与相应的扩展后的区域的对应边相互平行，且每个待扩展区域与相应的待扩展后的区域的形状可以相同；
根据所述目标图像进行预测。
可选地，每个所述待扩展区域由一个所述分区组成。
可选地，每个所述扩展后的区域与相应的待扩展区域的形状一致。
可选地，多个所述分区包括直角三角形分区和等边三角形分区，各个等边三角形分区以相同的比例进行等比例放大，且各个直角三角形分区以相同的比例进行等比例放大。
可选地，至少一个所述待扩展区域中的每个所述待扩展区域由图像内容连续的至少两个所述分区拼接而成。
可选地，所述将多个所述待扩展区域的边界向外扩展一周，包括：针对多个所述待扩展区域中的目标待扩展区域，根据所述多面体的面与顶点的对应关系，从多个所述分区中选取目标分区，其中所述目标分区与形成所述目标待扩展区域的分区在所述多面体的映射区域具有共同的两个顶点；将所述目标待扩展区域的边界向外扩展一周，其中所述目标待扩展区域对应的扩展后的区域与所述目标待扩展区域之间的目标填充区域中的至少部分像素是基于所述目标分区中的像素得到的。
可选地，所述将所述目标待扩展区域的边界向外扩展一周，包括：利用所述目标分区中的像素对所述目标填充区域的部分区域进行填充；利用默认像素对所述目标填充区域中的除所述部分区域之外的剩余区域进行填充。
可选地，所述将所述目标待扩展区域的边界向外扩展一周，包括：利用所述目标分区中的像素对所述目标填充区域中的部分区域进行填充；利用所述目标填充区域中的除所述部分区域之外的剩余区域的周边像素对所述剩 余区域进行填充。
可选地，所述部分区域为将所述目标分区与所述目标待扩展区域按照共同的顶点拼接成的平面图形与所述目标填充区域的公共区域。
可选地，多个所述待扩展区域包括第一待扩展区域和第二待扩展区域，所述第一待扩展区域包括第一区域，所述第二待扩展区域包括第二区域，所述第一区域和所述第二区域为所述待扩展图像中的相邻区域；所述将多个所述待扩展区域的边界向外扩展一周，包括：对所述第一区域和所述第二区域的像素进行插值，得到所述第一区域和所述第二区域之间的填充区域的像素。
可选地，所述第一待扩展区域和所述第二待扩展区域的形状均为矩形。
可选地，所述视频处理装置为视频编码装置，所述处理器1420还用于执行以下操作：将扩展参数写入码流中，其中所述扩展参数用于标识每个所述待扩展区域与相应扩展后的区域之间的填充区域的位置。
可选地，所述视频处理装置为视频解码装置，所述将多个所述待扩展区域的边界向外扩展一周，包括：从码流中读取扩展参数，所述扩展参数用于标识每个所述待扩展区域与相应扩展后的区域之间的填充区域的位置；根据所述扩展参数，将多个所述待扩展区域的边界向外扩展一周。
可选地，所述扩展参数包括：面积参数，所述面积参数用于标识每个所述待扩展区域与相应的扩展后的区域的面积比；和/或距离参数，所述距离参数用于标识每个所述待扩展区域的边与相应扩展后的区域的对应边之间的平行线距离。
可选地，所述根据所述扩展参数，将多个所述待扩展区域的边界向外扩展一周，包括：根据所述面积参数，确定每个所述待扩展区域与相应的扩展后的区域的面积比；根据每个所述待扩展区域与相应的扩展后的区域的面积比，确定所述相应的扩展后的区域的位置；根据每个所述待扩展区域的位置，以及相应的待扩展后的区域的位置，确定每个所述待扩展区域的填充区域；对每个所述待扩展区域的填充区域进行像素填充，从而将每个所述待扩展区域的边界向外扩展一周。
可选地，所述多面体映射为六面体映射、八面体映射或二十面体映射。
可选地，至少一个所述分区非矩形。
可选地，至少一个所述分区为三角形或菱形。
可选地，至少一个所述分区为正三角形或直角三角形。
可选地，所述曲面图像为全景视频中的一帧图像中的部分或全部图像。
可选地，所述待扩展图像和所述目标图像均为矩形图像。
可选地，所述视频处理装置为视频编码装置，所述处理器1420还用于执行以下操作：将所述目标图像存储于参考图像缓冲区。
可选地，所述视频处理装置为视频解码装置，所述处理器1420还用于执行以下操作：将目标图像存储于解码图像缓冲区。
可选地，所述视频处理装置1400为视频解码装置，所述处理器1420还可用于执行以下操作：将所述待扩展图像作为输出图像。
可选地，所述根据所述目标图像进行预测，包括：将所述目标图像的像素作为预测像素进行帧内预测。
可选地，所述根据所述目标图像进行预测，包括：将所述目标图像作为参考图像进行帧间预测。
在上述实施例中，可以全部或部分地通过软件、硬件、固件或者其他任意组合来实现。当使用软件实现时，可以全部或部分地以计算机程序产品的形式实现。所述计算机程序产品包括一个或多个计算机指令。在计算机上加载和执行所述计算机程序指令时，全部或部分地产生按照本申请实施例所述的流程或功能。所述计算机可以是通用计算机、专用计算机、计算机网络、或者其他可编程装置。所述计算机指令可以存储在计算机可读存储介质中，或者从一个计算机可读存储介质向另一个计算机可读存储介质传输，例如，所述计算机指令可以从一个网站站点、计算机、服务器或数据中心通过有线(例如同轴电缆、光纤、数字用户线(digital subscriber line，DSL))或无线(例如红外、无线、微波等)方式向另一个网站站点、计算机、服务器或数据中心进行传输。所述计算机可读存储介质可以是计算机能够存取的任何可用介质或者是包含一个或多个可用介质集成的服务器、数据中心等数据存储设备。所述可用介质可以是磁性介质(例如，软盘、硬盘、磁带)、光介质(例如数字视频光盘(digital video disc，DVD))、或者半导体介质(例如固态硬盘(solid state disk，SSD))等。
本领域普通技术人员可以意识到，结合本文中所公开的实施例描述的各示例的单元及算法步骤，能够以电子硬件、或者计算机软件和电子硬件的结合来实现。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特 定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本申请的范围。
以上所述，仅为本申请的具体实施方式，但本申请的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本申请揭露的技术范围内，可轻易想到变化或替换，都应涵盖在本申请的保护范围之内。因此，本申请的保护范围应以所述权利要求的保护范围为准。

Claims (53)

  1. 一种视频处理方法，其特征在于，包括：
    获取待扩展图像，所述待扩展图像是由曲面图像经多面体映射之后得到的多个分区拼接而成的平面图像，多个所述分区共形成多个待扩展区域，每个所述待扩展区域包括所述多个分区中的至少一个分区，且每个所述待扩展区域中的图像内容连续；
    将多个所述待扩展区域的边界向外扩展一周，得到与多个所述待扩展区域一一对应的多个扩展后的区域，其中每个待扩展区域的每条边与相应的扩展后的区域的对应边相互平行，且每个所述待扩展区域与相应的待扩展后的区域的形状相同；
    根据所述目标图像进行预测。
  2. 根据权利要求1所述的视频处理方法，其特征在于，每个所述待扩展区域由一个所述分区组成。
  3. 根据权利要求2所述的视频处理方法，其特征在于，每个所述扩展后的区域与相应的待扩展区域的形状一致。
  4. 根据权利要求2所述的视频处理方法，其特征在于，多个所述分区包括直角三角形分区和等边三角形分区，各个等边三角形分区以相同的比例进行等比例放大，且各个直角三角形分区以相同的比例进行等比例放大。
  5. 根据权利要求1所述的视频处理方法，其特征在于，至少一个所述待扩展区域中的每个所述待扩展区域由图像内容连续的至少两个所述分区拼接而成。
  6. 根据权利要求1所述的视频处理方法，其特征在于，所述将多个所述待扩展区域的边界向外扩展一周，包括：
    针对多个所述待扩展区域中的目标待扩展区域，根据所述多面体的面与顶点的对应关系，从多个所述分区中选取目标分区，其中所述目标分区与形成所述目标待扩展区域的分区在所述多面体的映射区域具有共同的两个顶点；
    将所述目标待扩展区域的边界向外扩展一周，其中所述目标待扩展区域对应的扩展后的区域与所述目标待扩展区域之间的目标填充区域中的至少部分像素是基于所述目标分区中的像素得到的。
  7. 根据权利要求6所述的视频处理方法，其特征在于，所述将所述目 标待扩展区域的边界向外扩展一周，包括：
    利用所述目标分区中的像素对所述目标填充区域的部分区域进行填充；
    利用默认像素对所述目标填充区域中的除所述部分区域之外的剩余区域进行填充。
  8. 根据权利要求6所述的视频处理方法，其特征在于，所述将所述目标待扩展区域的边界向外扩展一周，包括：
    利用所述目标分区中的像素对所述目标填充区域中的部分区域进行填充；
    利用所述目标填充区域中的除所述部分区域之外的剩余区域的周边像素对所述剩余区域进行填充。
  9. 根据权利要求7或8所述的视频处理方法，其特征在于，所述部分区域为将所述目标分区与所述目标待扩展区域按照共同的顶点拼接成的平面图形与所述目标填充区域的公共区域。
  10. 根据权利要求1所述的视频处理方法，其特征在于，多个所述待扩展区域包括第一待扩展区域和第二待扩展区域，所述第一待扩展区域包括第一区域，所述第二待扩展区域包括第二区域，所述第一区域和所述第二区域为所述待扩展图像中的相邻区域；
    所述将多个所述待扩展区域的边界向外扩展一周，包括：
    对所述第一区域和所述第二区域的像素进行插值，得到所述第一区域和所述第二区域之间的填充区域的像素。
  11. 根据权利要求10所述的视频处理方法，其特征在于，所述第一待扩展区域和所述第二待扩展区域的形状均为矩形。
  12. 根据权利1所述的视频处理方法，其特征在于，所述视频处理方法应用于视频编码装置，
    所述视频处理方法还包括：
    将扩展参数写入码流中，其中所述扩展参数用于标识每个所述待扩展区域与相应扩展后的区域之间的填充区域的位置。
  13. 根据权利要求1所述的视频处理方法，其特征在于，所述视频处理方法应用于视频解码装置，
    所述将多个所述待扩展区域的边界向外扩展一周，包括：
    从码流中读取扩展参数，所述扩展参数用于标识每个所述待扩展区域与 相应扩展后的区域之间的填充区域的位置；
    根据所述扩展参数，将多个所述待扩展区域的边界向外扩展一周。
  14. 根据权利要求12或13所述的视频处理方法，其特征在于，所述扩展参数包括：
    面积参数，所述面积参数用于标识每个所述待扩展区域与相应的扩展后的区域的面积比；和/或
    距离参数，所述距离参数用于标识每个所述待扩展区域的边与相应扩展后的区域的对应边之间的平行线距离。
  15. 根据权利要求13或14所述的视频处理方法，其特征在于，所述根据所述扩展参数，将多个所述待扩展区域的边界向外扩展一周，包括：
    根据所述面积参数，确定每个所述待扩展区域与相应的扩展后的区域的面积比；
    根据每个所述待扩展区域与相应的扩展后的区域的面积比，确定所述相应的扩展后的区域的位置；
    根据每个所述待扩展区域的位置，以及相应的待扩展后的区域的位置，确定每个所述待扩展区域的填充区域；
    对每个所述待扩展区域的填充区域进行像素填充，从而将每个所述待扩展区域的边界向外扩展一周。
  16. 根据权利要求1所述的视频处理方法，其特征在于，所述多面体映射为六面体映射、八面体映射或二十面体映射。
  17. 根据权利要求1所述的视频处理方法，其特征在于，至少一个所述分区非矩形。
  18. 根据权利要求1所述的视频处理方法，其特征在于，至少一个所述分区为三角形或菱形。
  19. 根据权利要求1所述的视频处理方法，其特征在于，至少一个所述分区为正三角形或直角三角形。
  20. 根据权利要求1所述的视频处理方法，其特征在于，所述曲面图像为全景视频中的一帧图像中的部分或全部图像。
  21. 根据权利要求1所述的视频处理方法，其特征在于，所述待扩展图像和所述目标图像均为矩形图像。
  22. 根据权利要求1所述的视频处理方法，其特征在于，所述视频处理 方法应用于视频编码装置，
    所述视频处理方法还包括：
    将所述目标图像存储于参考图像缓冲区。
  23. 根据权利要求1所述的视频处理方法，其特征在于，所述视频处理方法应用于视频解码装置，
    所述视频处理方法还包括：
    将目标图像存储于解码图像缓冲区。
  24. 根据权利要求1所述的视频处理方法，其特征在于，所述视频处理方法应用于视频解码装置，
    所述视频处理方法还包括：
    将所述待扩展图像作为输出图像。
  25. 根据权利要求1所述的视频处理方法，其特征在于，所述根据所述目标图像进行预测，包括：
    将所述目标图像的像素作为预测像素进行帧内预测。
  26. 根据权利要求1所述的视频处理方法，其特征在于，所述根据所述目标图像进行预测，包括：
    将所述目标图像作为参考图像进行帧间预测。
  27. 一种视频处理装置，其特征在于，包括：
    存储器，用于存储指令；
    处理器，用于执行所述存储器中存储的指令，以执行以下操作：
    获取待扩展图像，所述待扩展图像是由曲面图像经多面体映射之后得到的多个分区拼接而成的平面图像，多个所述分区共形成多个待扩展区域，每个所述待扩展区域包括所述多个分区中的至少一个分区，且每个所述待扩展区域中的图像内容连续；
    将多个所述待扩展区域的边界向外扩展一周，得到与多个所述待扩展区域一一对应的多个扩展后的区域，其中每个待扩展区域的每条边与相应的扩展后的区域的对应边相互平行，且每个所述待扩展区域与相应的待扩展后的区域的形状相同；
    根据所述目标图像进行预测。
  28. 根据权利要求27所述的视频处理装置，其特征在于，每个所述待扩展区域由一个所述分区组成。
  29. 根据权利要求28所述的视频处理装置，其特征在于，每个所述扩展后的区域与相应的待扩展区域的形状一致。
  30. 根据权利要求28所述的视频处理装置，其特征在于，多个所述分区包括直角三角形分区和等边三角形分区，各个等边三角形分区以相同的比例进行等比例放大，且各个直角三角形分区以相同的比例进行等比例放大。
  31. 根据权利要求27所述的视频处理装置，其特征在于，至少一个所述待扩展区域中的每个所述待扩展区域由图像内容连续的至少两个所述分区拼接而成。
  32. 根据权利要求27所述的视频处理装置，其特征在于，所述将多个所述待扩展区域的边界向外扩展一周，包括：
    针对多个所述待扩展区域中的目标待扩展区域，根据所述多面体的面与顶点的对应关系，从多个所述分区中选取目标分区，其中所述目标分区与形成所述目标待扩展区域的分区在所述多面体的映射区域具有共同的两个顶点；
    将所述目标待扩展区域的边界向外扩展一周，其中所述目标待扩展区域对应的扩展后的区域与所述目标待扩展区域之间的目标填充区域中的至少部分像素是基于所述目标分区中的像素得到的。
  33. 根据权利要求32所述的视频处理装置，其特征在于，所述将所述目标待扩展区域的边界向外扩展一周，包括：
    利用所述目标分区中的像素对所述目标填充区域的部分区域进行填充；
    利用默认像素对所述目标填充区域中的除所述部分区域之外的剩余区域进行填充。
  34. 根据权利要求32所述的视频处理装置，其特征在于，所述将所述目标待扩展区域的边界向外扩展一周，包括：
    利用所述目标分区中的像素对所述目标填充区域中的部分区域进行填充；
    利用所述目标填充区域中的除所述部分区域之外的剩余区域的周边像素对所述剩余区域进行填充。
  35. 根据权利要求33或34所述的视频处理装置，其特征在于，所述部分区域为将所述目标分区与所述目标待扩展区域按照共同的顶点拼接成的平面图形与所述目标填充区域的公共区域。
  36. 根据权利要求27所述的视频处理装置，其特征在于，多个所述待扩展区域包括第一待扩展区域和第二待扩展区域，所述第一待扩展区域包括第一区域，所述第二待扩展区域包括第二区域，所述第一区域和所述第二区域为所述待扩展图像中的相邻区域；
    所述将多个所述待扩展区域的边界向外扩展一周，包括：
    对所述第一区域和所述第二区域的像素进行插值，得到所述第一区域和所述第二区域之间的填充区域的像素。
  37. 根据权利要求36所述的视频处理装置，其特征在于，所述第一待扩展区域和所述第二待扩展区域的形状均为矩形。
  38. 根据权利27所述的视频处理装置，其特征在于，所述视频处理装置为视频编码装置，
    所述处理器还用于执行以下操作：
    将扩展参数写入码流中，其中所述扩展参数用于标识每个所述待扩展区域与相应扩展后的区域之间的填充区域的位置。
  39. 根据权利要求27所述的视频处理装置，其特征在于，所述视频处理装置为视频解码装置，
    所述将多个所述待扩展区域的边界向外扩展一周，包括：
    从码流中读取扩展参数，所述扩展参数用于标识每个所述待扩展区域与相应扩展后的区域之间的填充区域的位置；
    根据所述扩展参数，将多个所述待扩展区域的边界向外扩展一周。
  40. 根据权利要求38或39所述的视频处理装置，其特征在于，所述扩展参数包括：
    面积参数，所述面积参数用于标识每个所述待扩展区域与相应的扩展后的区域的面积比；和/或
    距离参数，所述距离参数用于标识每个所述待扩展区域的边与相应扩展后的区域的对应边之间的平行线距离。
  41. 根据权利要求39或40所述的视频处理装置，其特征在于，所述根据所述扩展参数，将多个所述待扩展区域的边界向外扩展一周，包括：
    根据所述面积参数，确定每个所述待扩展区域与相应的扩展后的区域的面积比；
    根据每个所述待扩展区域与相应的扩展后的区域的面积比，确定所述相 应的扩展后的区域的位置；
    根据每个所述待扩展区域的位置，以及相应的待扩展后的区域的位置，确定每个所述待扩展区域的填充区域；
    对每个所述待扩展区域的填充区域进行像素填充，从而将每个所述待扩展区域的边界向外扩展一周。
  42. 根据权利要求27所述的视频处理装置，其特征在于，所述多面体映射为六面体映射、八面体映射或二十面体映射。
  43. 根据权利要求27所述的视频处理装置，其特征在于，至少一个所述分区非矩形。
  44. 根据权利要求27所述的视频处理装置，其特征在于，至少一个所述分区为三角形或菱形。
  45. 根据权利要求27所述的视频处理装置，其特征在于，至少一个所述分区为正三角形或直角三角形。
  46. 根据权利要求27所述的视频处理装置，其特征在于，所述曲面图像为全景视频中的一帧图像中的部分或全部图像。
  47. 根据权利要求27所述的视频处理装置，其特征在于，所述待扩展图像和所述目标图像均为矩形图像。
  48. 根据权利要求27所述的视频处理装置，其特征在于，所述视频处理装置为视频编码装置，
    所述处理器还用于执行以下操作：
    将所述目标图像存储于参考图像缓冲区。
  49. 根据权利要求27所述的视频处理装置，其特征在于，所述视频处理装置为视频解码装置，
    所述处理器还用于执行以下操作：
    将目标图像存储于解码图像缓冲区。
  50. 根据权利要求27所述的视频处理装置，其特征在于，所述视频处理装置为视频解码装置，
    所述处理器还用于执行以下操作：
    将所述待扩展图像作为输出图像。
  51. 根据权利要求27所述的视频处理装置，其特征在于，所述根据所述目标图像进行预测，包括：
    将所述目标图像的像素作为预测像素进行帧内预测。
  52. 根据权利要求27所述的视频处理装置，其特征在于，所述根据所述目标图像进行预测，包括：
    将所述目标图像作为参考图像进行帧间预测。
  53. 一种机器可读存储介质，其特征在于，所述机器可读存储介质中存储有用于执行如权利要求1-26中任一项所述的方法的指令。
PCT/CN2018/103676 2018-08-31 2018-08-31 视频处理方法及相关装置 WO2020042185A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN201880036750.0A CN110741644A (zh) 2018-08-31 2018-08-31 视频处理方法及相关装置
PCT/CN2018/103676 WO2020042185A1 (zh) 2018-08-31 2018-08-31 视频处理方法及相关装置

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2018/103676 WO2020042185A1 (zh) 2018-08-31 2018-08-31 视频处理方法及相关装置

Publications (1)

Publication Number Publication Date
WO2020042185A1 true WO2020042185A1 (zh) 2020-03-05

Family

ID=69236650

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/103676 WO2020042185A1 (zh) 2018-08-31 2018-08-31 视频处理方法及相关装置

Country Status (2)

Country Link
CN (1) CN110741644A (zh)
WO (1) WO2020042185A1 (zh)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060034523A1 (en) * 2004-08-13 2006-02-16 Samsung Electronics Co., Ltd. Method and apparatus for encoding and decoding an icosahedron panorama image
CN106254874A (zh) * 2016-09-19 2016-12-21 上海国茂数字技术有限公司 一种全景视频编码方法及装置
CN108111851A (zh) * 2016-11-25 2018-06-01 华为技术有限公司 一种去块滤波方法及终端
CN108235031A (zh) * 2016-12-15 2018-06-29 华为技术有限公司 一种运动矢量解码方法及解码器

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060034523A1 (en) * 2004-08-13 2006-02-16 Samsung Electronics Co., Ltd. Method and apparatus for encoding and decoding an icosahedron panorama image
CN106254874A (zh) * 2016-09-19 2016-12-21 上海国茂数字技术有限公司 一种全景视频编码方法及装置
CN108111851A (zh) * 2016-11-25 2018-06-01 华为技术有限公司 一种去块滤波方法及终端
CN108235031A (zh) * 2016-12-15 2018-06-29 华为技术有限公司 一种运动矢量解码方法及解码器

Also Published As

Publication number Publication date
CN110741644A (zh) 2020-01-31

Similar Documents

Publication Publication Date Title
KR102273199B1 (ko) 곡선 뷰 비디오 인코딩/디코딩에서 효율성 향상을 위한 시스템 및 방법
CN107454468B (zh) 对沉浸式视频进行格式化的方法、装置和流
CN111034201B (zh) 编码和解码体积视频的方法、设备和流
US10264282B2 (en) Method and apparatus of inter coding for VR video using virtual reference frames
US11004173B2 (en) Method for processing projection-based frame that includes at least one projection face packed in 360-degree virtual reality projection layout
US20190281273A1 (en) Adaptive loop filtering method for reconstructed projection-based frame that employs projection layout of 360-degree virtual reality projection
US11138460B2 (en) Image processing method and apparatus
TW201916685A (zh) 用於處理360°vr幀序列的方法及裝置
JP7407703B2 (ja) ３ｄシーンの点を生成するための方法およびデバイス
TWI702567B (zh) 用於處理包括封裝在360度虛擬現實投影佈局中的至少一個投影面的基於投影的圖框的方法
TWI690728B (zh) 用於處理包括堆疊在具有填充的基於立方體的投影佈局中的投影面的基於投影的圖框的方法
TWI702835B (zh) Vr360 視訊編解碼的運動向量推導方法和裝置
TWI681662B (zh) 用於減少基於投影的圖框中的偽影的方法和裝置
JP2001186516A (ja) 画像データの符号化復号化方法及び装置
KR20200009119A (ko) 360도 비디오에 대한 영상 코딩 시스템에서 프로젝션 타입 기반 양자화 파라미터를 사용한 영상 디코딩 방법 및 장치
US10997795B2 (en) Method and apparatus for processing three dimensional object image using point cloud data
US20230388542A1 (en) A method and apparatus for adapting a volumetric video to client devices
CN114270863A (zh) 一种编码和解码立体视频的方法和装置
WO2019127100A1 (zh) 视频编码的方法、装置和计算机系统
KR20200062258A (ko) 360도 비디오에 대한 영상 코딩 시스템에서 회전 파라미터를 사용한 영상 디코딩 방법 및 장치
TWI756526B (zh) 用於重構的基於投影的幀的樣本適應性偏移濾波方法
WO2020042185A1 (zh) 视频处理方法及相关装置
CN109496429B (zh) 视频编码方法、视频解码方法及相关装置
CN115885513A (zh) 用于对体积视频进行编码和解码的方法和装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18931647

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18931647

Country of ref document: EP

Kind code of ref document: A1