CN109525842B - Position-based multi-Tile permutation coding method, device, equipment and decoding method - Google Patents

Abstract

The invention discloses a multi-Tile permutation coding method based on position correlation, which comprises the following steps: the method is characterized by comprising the following steps: obtaining Tile fragments to be arranged; and arranging the Tile fragments according to the position relation of the Tile fragments. The position relation of the Tile fragment is specifically as follows: and carrying out position division on the Tile slices subjected to the ERP projection according to different resolutions. According to the invention, when the Tile fragments are arranged by decoding, the arrangement is carried out according to the position correlation of the Tile fragments, the Tile fragments in the same position region have the same resolution, and the Tile fragments can be combined into one region for uniform processing during rendering after decoding, so that the processing speed is increased, seam noise possibly occurring at the adjacent positions of the two Tile fragments can be avoided, and the Tile rendering performance can be improved by adjusting the arrangement sequence.

Description

Position-based multi-Tile permutation coding method, device, equipment and decoding method

Technical Field

The invention relates to the field of video coding, in particular to a multi-Tile permutation coding method, a multi-Tile permutation coding device, multi-Tile permutation coding equipment and a multi-Tile permutation coding decoding method based on position correlation.

Background

And (3) 360-degree video: the method has the advantages that multiple cameras are used for shooting objects in the same space at multiple angles simultaneously to obtain videos shot in all directions at 360 degrees, real scenes are restored and displayed on the Internet completely by using a certain network technology, strong interactivity is achieved, and when 360-degree video playing is carried out, a user is allowed to freely adjust the watching angles of the videos in a vertical and horizontal mode without pausing the video playing.

In the prior art, for transmission coding of a video (such as a 360-degree video), Tile fragment segmentation is performed on the video, and after receiving the Tile fragments, a decoding end arranges the Tile fragments according to an increasing order of Tile fragment coding for decoding, but because the position correlation of the Tile fragments is not considered during arrangement, rendering processing efficiency is low, and there is a possibility that seam noise may occur at the adjacent positions of two Tile fragments, it is necessary to provide a Tile fragment arrangement coding method capable of improving processing efficiency and reducing seam noise between adjacent Tile fragments.

Disclosure of Invention

The present invention is directed to solving, at least to some extent, one of the technical problems in the related art. Therefore, the invention aims to provide a multi-Tile permutation coding method, a multi-Tile permutation coding device, a multi-Tile permutation coding equipment and a multi-Tile permutation decoding method based on position correlation.

The technical scheme adopted by the invention is as follows:

in a first aspect, the present invention provides a position correlation-based multi-Tile permutation coding method, including the steps of:

obtaining Tile fragments to be arranged;

and arranging the Tile fragments according to the position relation of the Tile fragments.

Further, the Tile fragment is a data fragment subjected to an ERP projection mode.

Further, the position relationship of the Tile fragment is specifically as follows: and carrying out position division on the Tile fragments subjected to the ERP projection according to different resolutions to obtain a position relation.

Further, the location division according to different resolutions specifically takes a first area resolution as a standard:

the second region is down-sampled 1/2 at horizontal resolution;

the third region is down-sampled 1/4 at horizontal resolution.

Further, the position relationship of the Tile fragment is specifically as follows: and arranging the Tile fragments located in the same region adjacently, wherein the Tile fragments located in the same region have the same resolution.

Further, the arranging also includes arranging Tile fragments located in the same region according to a position-adjacent order.

In a second aspect, the present invention provides a position correlation-based multi-Tile permutation decoding method, including the video data obtained by any one of the position correlation-based multi-Tile permutation coding methods described in the first aspect, and during rendering, Tile slices in the same region are merged into one region for processing.

Further, when the number of the obtained Tile fragments to be arranged is less than the minimum number of the Tile fragments required by decoding, filling the rest positions with the content of the Tile fragments positioned at the tail end of the arrangement sequence.

In a third aspect, the present invention provides a position correlation-based multi-Tile permutation coding apparatus, including:

a Tile fragment acquisition module to be arranged: the method comprises the steps of obtaining Tile fragments to be arranged;

an arrangement module: and the method is used for arranging the Tile fragments according to the position relation of the Tile fragments.

In a fourth aspect, the present invention provides a control device for multi-Tile permutation coding based on position correlation, including:

at least one processor; and a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of the first aspect.

The invention has the beneficial effects that:

according to the invention, when the Tile fragments are arranged in a coding mode, the Tile fragments are arranged according to the position relevance of the Tile fragments, the Tile fragments in the same position area have the same resolution ratio, and can be combined into one area for unified processing when being rendered after decoding, so that the processing speed is increased, seam noise possibly occurring at the adjacent positions of the two Tile fragments can be avoided, and the Tile rendering performance can be improved by adjusting the arrangement sequence.

Drawings

FIG. 1a is a schematic view of a spherical panoramic video;

FIG. 1b is an ERP projection diagram;

FIG. 2 is a flow chart of a multi-Tile permutation coding method based on location correlation according to an embodiment of the present invention;

FIG. 3 is a schematic diagram of Tile location partitioning via ERP mapping in an embodiment of the present invention;

FIG. 4 is a schematic diagram of different resolution areas in one embodiment of the present invention;

FIG. 5 is a diagram illustrating a conventional ascending order according to Tile slice coding;

FIG. 6 is an actual ERP arrangement position of received Tile patches in an embodiment of the present invention;

FIG. 7 is a schematic diagram of a Tile slice arrangement based on location correlation according to an embodiment of the present invention;

fig. 8 is a block diagram of a multi-Tile permutation and coding apparatus based on position correlation according to an embodiment of the present invention.

Detailed Description

It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict.

In the prior art, most of the collected 360-degree spherical panoramic videos are processed by projecting the spherical panoramic videos onto a 2D screen to form an ERP-format video, and most of shooting sequences are stored in the ERP format. Equirectangular projection (ERP) is a simple mapping method, in which the longitude lines are mapped to vertical lines with constant spacing, and the latitude lines are mapped to horizontal lines with constant spacing. The projection mode is simple in mapping relation, but is neither equal in area nor conformal, and considerable distortion is introduced.

As shown in fig. 1a, it is a schematic view of a spherical panoramic video, and the video is seen to surround the sphere in 360 degrees.

As shown in fig. 1b, which is a schematic view of ERP projection, it is equivalent to stretch the spherical video of fig. 1a into a plane, and 3D-2D conversion is implemented, where ERP is an image plane of 360 degrees, it can be seen that north and south poles at high latitude are all stretched, and left and right boundaries of the image plane can be folded together.

The first embodiment is as follows:

as shown in fig. 2, a flowchart of a multi-Tile permutation coding method based on position correlation in this embodiment includes the steps of: s1: obtaining Tile fragments to be arranged; s2: and arranging the Tile fragments according to the position relation of the Tile fragments.

Wherein the Tile is a data Tile through ERP projection mode, and the position relation of the Tile Tile is as follows: and carrying out position division on the Tile fragments subjected to the ERP projection according to different resolutions to obtain a position relation.

As shown in fig. 3, which is a schematic diagram of Tile position division through ERP mapping, it can be seen that the spherical panoramic video shown in fig. 1a is divided into 42 Tile slices based on shooting positions after ERP mapping, and a horizontal axis of a shooting range is [0,2 pi ]]The longitudinal axis is

As shown in fig. 4, which is a schematic diagram of different resolution areas for easy display, the Tile position division schematic diagram of fig. 3 is rearranged, and it can be seen that:

the first region includes: tile fragment {9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19,20, 21,22, 23, 24, 25, 26, 27, 28, 29, 30, 31,32 };

the second region includes: tile fragment {3, 4, 5, 6, 7, 8,33, 34, 35, 36, 37, 38 };

the third region includes: tile slicing {0, 1,2, 39, 40, 41 };

as can be seen from comparison with fig. 3, with the origin of the longitudinal axis as the center, the farther from the origin, the closer to the north and south poles at the high latitude, the content contained in the Tile video can be considered to be relatively less, so the Tile slicing positions are classified into the above three regions according to the distance from the origin of the longitudinal axis, that is, the third region is farthest from the north and south poles, the second region is farther between the first region and the third region, and the first region is located on both sides of the origin of the longitudinal axis.

Because the Tile slices after ERP projection are subjected to position division according to different resolutions, the different resolutions are set as follows, with the resolution of the first area as a standard, 1/2 downsampling is performed on the second area in the horizontal resolution, and 1/4 downsampling is performed on the third area in the horizontal resolution. Since the third region is relatively farther away than the second region, it is considered to contain less active content, and therefore 1/4 downsampling is applied to it, Tile slices located in the same region have the same resolution.

Example two:

according to the first embodiment, in decoding, Tile fragments of the same region are combined into one region for processing.

In this embodiment, it is assumed that a plurality of Tile fragments received when the user decodes are {3, 8, 9,10, 19,20, 21,22, 31,32, 33, 38}, and thus 12 Tile fragments are obtained.

As shown in fig. 5, the schematic diagram of the conventional ascending sequence arrangement according to Tile fragment coding is shown, wherein when a Tile fragment is decoded, the arrangement sequence of the Tile fragment is arranged according to Tile _ id increment, and because in the current Tile-based coding application, at most M × N Tile fragments (for example, fig. 5, M ═ 6, N ═ 3, and 18 Tile fragments in total) are downloaded during decoding, and the at most M × N Tile fragments are merged into one Tile coding region and sent to a hardware decoder for decoding, and the M × N fragments form an MxN Tile list.

When the number of the downloaded Tile fragments is less than the minimum number (M × N) required for decoding, the content of the Tile fragment at the end of the arrangement sequence needs to be used to fill the rest positions.

As shown in fig. 6, in the actual ERP arrangement position of the Tile fragment received in this embodiment, it can be seen that Tile fragment 8 and Tile fragment 3 are adjacent to each other, Tile fragment 19 and Tile fragment 20 are adjacent to each other, Tile fragment 20 is adjacent to Tile fragment 9, Tile fragment 9 is adjacent to Tile fragment 10, Tile fragment 31 and Tile fragment 32 are adjacent to each other, Tile fragment 32 is adjacent to Tile fragment 21, Tile fragment 21 is adjacent to Tile fragment 22, and Tile fragment 38 is adjacent to Tile fragment 33, so that according to the adjacent relationship, the actual arrangement position is not the same as the arrangement position shown in fig. 5 according to the incremental relationship.

Therefore, a schematic diagram of Tile slice arrangement based on the position correlation shown in fig. 7 can be obtained, where [8, 3], [19,20,9,10], [31,32,21,22], [38,33], and Tile slice 38 is used to fill the remaining positions, and only this arrangement enables all tiles adjacent to each other in the ERP image to be arranged together, and Tile slices in the same area can be easily merged into one area for processing.

In this embodiment, Tile fragment 8 and Tile fragment 3 are adjacent to each other in the left-right direction and have the same horizontal scaling, so that they can be merged into one region for uniform processing when rendering is performed after decoding. Therefore, the processing speed is accelerated, seam noise possibly occurring at the adjacent positions of the two Tile fragments can be avoided, and the Tile rendering performance is improved by adjusting the arrangement sequence.

Example three:

as shown in fig. 8, a block diagram of a multi-Tile permutation and coding apparatus based on position correlation in this embodiment includes:

a Tile fragment acquisition module to be arranged: the method comprises the steps of obtaining Tile fragments to be arranged; an arrangement module: and the method is used for arranging the Tile fragments according to the position relation of the Tile fragments.

In another aspect, the present invention provides a control apparatus for multi-Tile permutation coding based on position correlation, including:

at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executable by the at least one processor to enable the at least one processor to perform the method according to embodiment one.

According to the invention, when the Tile fragments are arranged in a coding mode, the Tile fragments are arranged according to the position relevance of the Tile fragments, the Tile fragments in the same position area have the same resolution ratio, and can be combined into one area for unified processing when being rendered after decoding, so that the processing speed is accelerated, seam noise possibly occurring at the adjacent positions of the two Tile fragments can be avoided, and the Tile rendering performance is improved by adjusting the arrangement sequence.

While the preferred embodiments of the present invention have been illustrated and described, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims.

Claims (4)

1. A multi-Tile permutation coding method based on position correlation is characterized by comprising the following steps:

obtaining Tile fragments to be arranged, wherein the Tile fragments are data fragments formed in an ERP projection mode, and before the Tile fragments to be arranged are obtained, the Tile fragments are processed in the following mode:

determining a longitudinal axis and an origin of the Tile slice formed in an ERP projection mode, wherein the origin is located at the center of the longitudinal axis;

dividing the Tile formed in an ERP projection mode into a first area, a second area and a third area according to the distance from the origin of the longitudinal axis, wherein the third area is farthest away from the origin of the longitudinal axis, the second area is farther away from the origin of the longitudinal axis and is positioned between the first area and the third area, and the first area is positioned on two sides of the origin of the longitudinal axis;

taking the resolution of the Tile fragment as a standard, performing 1/2 downsampling on the second area in the horizontal resolution, performing 1/4 downsampling on the third area in the horizontal resolution, wherein the Tile fragments in the same area have the same resolution;

carrying out position division on the Tile fragments to be arranged according to different resolutions to obtain the position relation of the Tile fragments;

carrying out Tile arrangement according to the position relation of the Tile, wherein the Tile arrangement comprises arranging the Tile in the same region according to the adjacent sequence of the positions;

and merging the arranged Tile fragments into a Tile coding region, and sending the Tile coding region to a hardware decoder for decoding.

2. A multi-Tile permutation decoding method based on position correlation, which is characterized by decoding the video data obtained by the multi-Tile permutation coding method based on position correlation according to claim 1, and merging Tile fragments of the same region into one region for processing during rendering.

3. The multi-Tile permutation decoding method based on the position correlation according to claim 2, wherein when the number of the obtained Tile fragments to be permuted is less than the minimum number of Tile fragments required for decoding, the content of the Tile fragment at the end of the permutation order is used to fill the rest of the positions.

4. A control device for multi-Tile permutation coding based on position correlation is characterized by comprising:

at least one processor; and a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of claim 1.

Images