CN110942428A - Panoramic image filtering method and device - Google Patents
Panoramic image filtering method and device Download PDFInfo
- Publication number
- CN110942428A CN110942428A CN201811154939.8A CN201811154939A CN110942428A CN 110942428 A CN110942428 A CN 110942428A CN 201811154939 A CN201811154939 A CN 201811154939A CN 110942428 A CN110942428 A CN 110942428A
- Authority
- CN
- China
- Prior art keywords
- dimension
- image
- panoramic
- pair
- boundary region
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001914 filtration Methods 0.000 title claims abstract description 199
- 238000000034 method Methods 0.000 title claims abstract description 83
- 238000012545 processing Methods 0.000 claims abstract description 136
- 230000008569 process Effects 0.000 claims description 44
- 238000010586 diagram Methods 0.000 description 4
- GHOKWGTUZJEAQD-ZETCQYMHSA-N (D)-(+)-Pantothenic acid Chemical compound OCC(C)(C)[C@@H](O)C(=O)NCCC(O)=O GHOKWGTUZJEAQD-ZETCQYMHSA-N 0.000 description 2
- 238000012805 post-processing Methods 0.000 description 2
- 238000013139 quantization Methods 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/13—Edge detection
-
- G06T5/70—
Abstract
The invention discloses a panoramic image filtering method and device. Determining the position of a surface boundary area pair which is adjacent on a panoramic spherical surface and is not adjacent on a projection expansion image in the panoramic projection expansion image according to format information of the panoramic projection expansion image, determining the direction of first-dimension and second-dimension filtering processing in the surface boundary area pair according to the direction of a specific surface boundary area in the surface boundary area pair on the panoramic projection expansion image, and sequentially performing first-dimension and second-dimension filtering processing on the surface boundary area pair, wherein the method for determining the direction of the first-dimension and the second-dimension filtering processing in the surface boundary area pair is consistent with the direction for determining the first-dimension and the second-dimension filtering processing in the non-surface boundary area on the panoramic projection expansion image. The invention can reduce the coding artifact and the discontinuity of the surface boundary of the coded image, improve the subjective quality of the coded image and ensure the matching of the coding and decoding ends.
Description
Technical Field
The invention relates to the field of panoramic image processing, in particular to a panoramic image pixel block filtering method and a panoramic image pixel block filtering device.
Background
The panoramic image refers to all scenes around an observation point in space, and is composed of all light rays received by the observation point, and the spherical surface can describe all the scenes around the observation point. Since the spherical image is difficult to store and encode, the spherical surface of the panoramic image is expanded into a planar image by defining a specific projection expansion method, which is generally determined by certain panoramic image expansion format information.
A cube is one of the most common panoramic image projection expansion formats at present, as shown in fig. 1, a spherical surface is projected onto six corresponding surfaces of a circumscribed cube by using a central projection method, eight vertices of the cube are A, B, C, D, E, F, G, H respectively, and then the cube is expanded, wherein a left side surface, a front surface and a right side surface are horizontally arranged, and an upper surface, a rear surface and a lower surface are vertically arranged. Since the left, front and right sides show the outer faces of the cubes and the vertical arrangements of the upper, rear and lower faces in fig. 1 show the inner faces of the cubes, it is necessary to turn the vertical arrangements of the upper, rear and lower faces left and right to represent the outer faces of the cubes, as shown in fig. 2, and turn the upper, rear and lower faces left and right and then rotate them clockwise by 90 ° to compactly arrange 6 faces of the cubes into a rectangular image of 3 × 2, fig. 2 shows only one arrangement of 6 faces of the cubes, and the 6 faces of the cubes may not be arranged in this manner.
Besides the cubic format, there are other formats improved relative to the cubic format, and the panoramic image projection expansion formats with higher expression efficiency at present include EAC, PAU, HEC [1-3], and the like.
After the spherical projection is expanded into the panoramic projection expanded image, the surface boundary regions originally adjacent on the spherical surface may not be adjacent on the panoramic projection expanded image, as shown in fig. 3, the surface boundary regions 1 and 1 ' have a common boundary AE on the cubic surface, while the boundary regions 1 and 1 ' are not adjacent on the panoramic projection expanded image, and the gray regions represented by the numbers a and a ' in fig. 3 represent pairs of surface boundary regions adjacent on the spherical surface but not adjacent on the panoramic projection expanded image, where a may be 1, 2, 3, 4, 5, 6, 7, and 8. The white areas in fig. 3 represent non-planar bounding areas on the panoramic projected expanded image, while the black areas represent planar bounding areas that are adjacent on a spherical surface and still adjacent on the panoramic projected expanded image.
In a conventional image or video encoding and decoding method, an entire image is divided into pixel blocks for processing, and since there is inconsistency between the pixel blocks in the processes of prediction, transformation, quantization, etc., and there is discontinuity at the pixel block boundary, a deblocking filtering process is performed on the pixel block boundary. In the conventional deblocking filtering method, the direction of the deblocking filtering process is uniform over the entire image, and the deblocking filtering process is in units of pixel blocks.
The conventional deblocking filtering method is two-dimensional separation filtering, and in order to match at the encoding and decoding ends, it is necessary to specify the directions of the first and second-dimensional filtering processes, and sequentially perform the first and second-dimensional filtering processes on pixel blocks. For example, first dimension filtering processing is performed on a boundary direction (such as a vertical direction) of a pixel block on an image, and then second dimension filtering processing is performed on another boundary direction (such as a horizontal direction) of the pixel block on the image, where the boundary direction and the filtering processing direction of the pixel block are perpendicular to each other, that is, filtering processing is performed on a vertical boundary in the horizontal direction, and the first dimension filtering processing direction and the second dimension filtering processing direction are also perpendicular to each other. As shown in fig. 4, the boundary direction between the pixel blocks a and B is the vertical direction, so that the vertical boundary between the pixel blocks A, B is first subjected to the first-dimension filtering process, i.e., the filtering process is first performed in the horizontal direction, and the pixel blocks a and C are horizontal boundaries, and then the result after the first-dimension filtering process is subjected to the second-dimension filtering process, i.e., the filtering process is performed in the vertical direction. In hevc (high Efficiency Video coding) and h.264, when performing deblocking filtering operation on a pixel block boundary, horizontal filtering is performed on a vertical boundary first, and then vertical filtering is performed on a horizontal boundary, that is, horizontal filtering is performed first and then vertical filtering is performed.
In the conventional encoding and decoding method, the encoding or decoding order is from top to bottom and from left to right, as shown in fig. 5, for the pixel blocks A, B, C and D, the pixel block a is encoded or decoded first, then the pixel block B is encoded or decoded, then the pixel block C is encoded or decoded, and finally the pixel block D is encoded or decoded. Thus, in the pair of plane boundary regions a and a 'adjacent on the spherical surface but not adjacent on the panorama projection expanded image in fig. 3, a' is a plane boundary region of the later encoding or decoding process, where a takes any one of 1, 2, 3, 4, 5, 6, 7, and 8.
Reference documents:
[1]M.Zhou,“AHG8:A study on Equi-Angular Cubemap projection(EAC),”JVET-G0056,July 2017
[2]Y.Sun,X.Huangfu,R.Zheng,B.Wang,L.Yu,“Description of 360°videocoding technology proposal by Zhejiang University，”JVET-J0033,March 2018
[3]Y.-H.Lee,J.-L.Lin,S.-K.Chang,C.-C.Ju,“CE13:Modified CubemapProjection in JVET-J0019(Test 5),”JVET-K0131,July 2018
disclosure of Invention
It is an object of the present invention to address discontinuities at the junction of pairs of face boundary regions that are adjacent on a panoramic spherical surface but not adjacent on a projected unfolded image. The direction of the first dimension and the second dimension of the surface boundary region pair is determined by appointing a specific surface boundary in the surface boundary region pair, so that matching can be realized when the encoding and decoding end carries out filtering processing on the surface boundary region pair.
A first object of the present invention is to provide a panoramic image filtering method, including:
determining the positions of surface boundary area pairs which are adjacent on a panoramic spherical surface and are not adjacent on the projection expansion image in the panoramic projection expansion image according to the format information of the panoramic projection expansion image;
determining the first dimension and the second dimension filtering processing direction in the surface boundary region pair according to the direction of a specific surface boundary region in the surface boundary region pair on the panoramic projection expansion image, wherein the first dimension and the second dimension filtering processing direction are determined by the surface boundary region pair and the non-surface boundary region on the panoramic projection expansion image;
and sequentially carrying out first-dimension and second-dimension filtering processing on the surface boundary region pair.
Further, the specific surface boundary region is a surface boundary region of the pair of surface boundary regions, which is subjected to later encoding or decoding processing on the panorama projection expanded image.
Further, for a common sub-region belonging to both surface boundary regions, the pixels filtered by the common sub-region are determined by the pixels filtered by the sub-region corresponding to the surface boundary region which is subjected to later encoding or decoding processing on the panorama extension projection image in the both surface boundary regions.
A second object of the present invention is to provide a panoramic image filtering apparatus, including the following modules:
a module for deriving the boundary region pair position of the surface to be filtered: the input of the module is format information of a panoramic projection expansion image, the position of a surface boundary area pair which is adjacent on a panoramic spherical surface and is not adjacent on the projection expansion image in the panoramic projection expansion image is determined according to the format information, and the output of the module is the position of the surface boundary area pair to be filtered in the panoramic projection expansion image;
a filtering processing direction determining module: the module inputs a pair of boundary areas of a surface to be filtered and the position of the boundary area pair in a panoramic projection expansion image, determines the direction of first-dimension and second-dimension filtering processing in the pair of boundary areas according to the direction of a specific boundary area in the pair of boundary areas on the panoramic projection expansion image, and the method for determining the direction of the first-dimension and the second-dimension filtering processing in the pair of boundary areas is consistent with the method for determining the direction of the first-dimension and the second-dimension filtering processing in the non-surface boundary area on the panoramic projection expansion image; the module outputs the first dimension and the second dimension of the boundary region pair of the surface to be filtered;
a filtering processing module: inputting the directions of the first dimension and the second dimension filtering processing of the surface boundary region pair to be filtered, and sequentially carrying out the first dimension and the second dimension filtering processing on the surface boundary region pair.
Further, a specific surface boundary region in the filtering processing direction determining module is a surface boundary region of the pair of surface boundary regions, which is subjected to later encoding or decoding processing on the panoramic projection expansion image.
Further, in the filtering processing module, for a common sub-region belonging to both surface boundary regions, the pixels filtered by the common sub-region are determined by the pixels filtered by the sub-region corresponding to the surface boundary region that is subjected to later encoding or decoding processing on the panorama expansion projection image in the both surface boundary regions.
The method has the advantages of reducing coding artifacts and discontinuity of the surface boundary of the panoramic image, improving the subjective quality of the panoramic image after coding, and ensuring that the matching of the coding and decoding ends can be realized when the filtering processing is carried out on the surface boundary area after the directions of the first dimension filtering processing and the second dimension filtering processing are specified.
Drawings
The principles of the present invention can be explained from the embodiments cited below with reference to the drawings.
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the invention and together with the description serve to explain the nature of the invention and not to limit the scope of the invention. In the drawings:
FIG. 1 is a perspective view of a spherical surface projected onto a circumscribed cube of a sphere, and six faces of the circumscribed cube are unfolded;
FIG. 2 shows six faces of a cube after being unfolded and compactly arranged into a 3x2 format by rotation;
FIG. 3 shows the distribution of boundary regions above a panoramic projection expansion image and non-planar boundary regions;
fig. 4 shows the directions and the processing sequences of the first-dimension filtering processing and the second-dimension filtering processing;
FIG. 5 illustrates an encoding or decoding order of pixel blocks within an image according to a conventional encoding/decoding technique;
FIG. 6 is a pair of surface bounding regions that are adjacent on a spherical surface but not on a panoramic projection expansion image in one embodiment of the present invention;
FIG. 7 is a block of pixels to be filtered in a pair of surface bounding regions that are adjacent on a spherical surface but not adjacent on a panoramic projection unfolded image in an embodiment of the present invention;
FIG. 8 is a drawing of the filtering directions and order of the pixel blocks to be filtered in pairs 2 and 2' of surface bounding regions that are adjacent on a spherical surface but not adjacent on a panoramic projection unfolded image in one embodiment of the present invention;
FIG. 9 is a diagram illustrating the alignment of pixel blocks to be filtered in pairs 1 and 1' of surface boundary regions that are adjacent on a spherical surface but not adjacent on a panoramic projection expanded image, in accordance with an embodiment of the present invention;
FIG. 10 is a diagram of the alignment of pixel blocks to be filtered in pairs 3 and 3' of surface bounding regions that are adjacent on a spherical surface but not adjacent on a panoramic projection expansion image, in accordance with an embodiment of the present invention;
FIG. 11 is a block of pixels to be filtered in pairs of surface bounding regions 4 and 4' that are adjacent on a spherical surface but not on a panoramic projection expansion image in an embodiment of the present invention;
FIG. 12 is a drawing of the filtering directions and order of the pixel blocks to be filtered in pairs 4 and 4' of surface bounding regions that are adjacent on a spherical surface but not adjacent on a panoramic projection unfolded image in one embodiment of the present invention;
FIG. 13 is a block of pixels to be filtered in pairs of surface bounding regions 5 and 5' that are adjacent on a spherical surface but not on a panoramic projection expansion image in an embodiment of the present invention;
FIG. 14 is a drawing of the filtering direction and order of the pixel blocks to be filtered in pairs 5 and 5' of surface bounding regions that are adjacent on a spherical surface but not adjacent on a panoramic projection unfolded image in one embodiment of the present invention;
FIG. 15 is a block of pixels to be filtered in a pair of face bounding regions 7 and 7' that are adjacent on a spherical surface but not on a panoramic projection expansion image in an embodiment of the present invention;
FIG. 16 is a drawing of the filtering direction and order of the pixel blocks to be filtered in pairs 7 and 7' of surface bounding regions that are adjacent on a spherical surface but not adjacent on a panoramic projection unfolded image in one embodiment of the present invention;
FIG. 17 is a diagram of an apparatus in accordance with an embodiment of the present invention;
FIG. 18 is a schematic view of a common sub-region belonging to both face boundary regions in accordance with an embodiment of the present invention;
fig. 19 is a flow of the filtering process for the common subregion.
Detailed Description
Example 1
The embodiment of the invention provides a panoramic image filtering method.
The panoramic image filtering method provided by the embodiment comprises the following steps:
determining the positions of surface boundary area pairs which are adjacent on a panoramic spherical surface and are not adjacent on the projection expansion image in the panoramic projection expansion image according to the format information of the panoramic projection expansion image;
determining the first dimension and the second dimension filtering processing direction in the surface boundary region pair according to the direction of a specific surface boundary region in the surface boundary region pair on the panoramic projection expansion image, wherein the first dimension and the second dimension filtering processing direction are determined by the surface boundary region pair and the non-surface boundary region on the panoramic projection expansion image;
and sequentially carrying out first-dimension and second-dimension filtering processing on the surface boundary region pair.
In this embodiment, the specific surface boundary region is a surface boundary region that is subjected to post-encoding or decoding processing on the panoramic projection expanded image in the surface boundary region pair;
the panned projected expanded image in the cubic format shown in fig. 6 is arranged as a rectangular image of 3 × 2, and therefore, it can be easily determined that the surface boundary areas a and a' on the panned projected expanded image shown in fig. 6 are pairs of surface boundary areas adjacent on the panoramic spherical surface in the panned image but not adjacent on the panned projected expanded image, where a is any one of 1, 2, 3, 4, 5, 6, 7 and 8. Since the encoding and decoding of prediction, transformation, quantization, etc. are performed on the panoramic projection expanded image, a special discontinuity occurs between pixel blocks adjacent to each other on the panoramic spherical surface but not adjacent to each other on the panoramic projection expanded image, and therefore, it is necessary to deblock filter these pixel blocks to smooth the discontinuity. As shown in fig. 6, the pixel block b in the surface boundary region 2 and the pixel block b ' in the surface boundary region 2 ' are pixel blocks adjacent to each other on the panoramic spherical surface but not adjacent to each other on the panoramic projection expansion image, and therefore, it is necessary to perform deblocking filtering on the pixel blocks b and b ' to smooth discontinuity between the pixel blocks.
In the pair of plane boundary regions 2 and 2 ' of fig. 6, the plane boundary region 2 ' is a region to be subjected to later encoding or decoding processing, and the direction of the plane boundary region 2 ' on the panorama projection expanded image is a vertical direction, and in this embodiment, the direction of the first-dimension filtering processing of the non-plane boundary region is set to be a horizontal direction and the direction of the second-dimension filtering processing is set to be a vertical direction. When the direction of the first-dimension filter processing of the non-surface boundary region is the horizontal direction, the first-dimension filter processing direction of the pair of surface boundary regions 2 and 2' is also the horizontal direction, that is, the first-dimension filter processing direction of the pair of surface boundary regions coincides with the first-dimension filter processing direction of the non-surface boundary region. And sequentially performing first-dimension and second-dimension filtering processing on the pixel blocks in the 2 and 2' in the opposite boundary area, namely performing first-dimension filtering processing according to a first-dimension filtering processing direction and then performing second-dimension filtering processing according to a second-dimension filtering processing direction.
As shown in fig. 7, when filtering the pixel blocks d, e, d ', and e', the pixel blocks d, e, d ', and e' are rotated 90 ° counterclockwise and then aligned with the pixel blocks d ', and e', and then the first-dimension and the second-dimension filtering are performed on the pixel blocks sequentially. As shown in fig. 8, first, the deblocking filtering process is performed on the pixel blocks d ' and d ', e ' in the horizontal direction to obtain the result after the filtering process, and then the deblocking filtering process is performed on the pixel blocks d ' and e ', d, and e in the vertical direction. According to the deblocking filtering process employed in the present embodiment, the order of the first and second-dimensional filtering processes of the pixel blocks in the face boundary region 2 is different from the order of the first and second-dimensional filtering processes of the pixel blocks in the non-face boundary region.
For the other numbered face boundary region pairs in fig. 7, the filter direction is also determined according to the method described in the present embodiment to filter the boundaries of the face boundary region pairs. For example, in the plane boundary regions 1 and 1 ', the direction of the plane boundary region 1' on the panoramic projection expansion image is the horizontal direction, then the plane boundary region performs filtering on the pixel blocks in 1 and 1 'in the order of filtering the vertical boundary first and then performing filtering on the horizontal boundary, the pixel blocks in the plane boundary region 1 do not need to be rotated or transposed to perform deblocking filtering processing directly with the pixel blocks in the plane boundary region 1', and as shown in fig. 9, the plane boundary region does not change the order of the first and second-dimensional filtering processing of the pixels in 1; in the plane boundary regions 3 and 3 ', the direction of 3' on the panoramic projection expansion image is the horizontal direction. The filtering order of the pixel blocks in the face boundary region pair 3 and 3 'is to perform vertical boundary filtering first and then horizontal boundary filtering, and it is necessary to perform deblocking filtering processing with the face boundary region 3' after rotating the pixel blocks in the face boundary region 3 by 180 °, as shown in fig. 10, the order of the first-dimensional filtering processing and the second-dimensional filtering processing of the pixels in the face boundary region pair 3 is not changed.
The pixel blocks in the other pairs of surface boundary regions in fig. 6 that are adjacent on the spherical surface but not on the panoramic projection expansion image are also processed according to the method described in the present embodiment.
Example 2
The embodiment of the invention provides a panoramic image filtering method.
The panoramic image filtering method provided by the embodiment comprises the following steps:
determining the positions of surface boundary area pairs which are adjacent on a panoramic spherical surface and are not adjacent on the projection expansion image in the panoramic projection expansion image according to the format information of the panoramic projection expansion image;
determining the first dimension and the second dimension filtering processing direction in the surface boundary region pair according to the direction of a specific surface boundary region in the surface boundary region pair on the panoramic projection expansion image, wherein the first dimension and the second dimension filtering processing direction are determined by the surface boundary region pair and the non-surface boundary region on the panoramic projection expansion image;
and sequentially carrying out first-dimension and second-dimension filtering processing on the surface boundary region pair.
In this embodiment, the specific surface boundary region is a surface boundary region of the pair of surface boundary regions, which is encoded or decoded first on the panoramic projection expansion image;
in the pair of plane boundary regions 4 and 4' of fig. 6, the plane boundary region 4 is a region to be encoded or decoded first, and the direction of the plane boundary region 4 on the panorama projection expanded image is a vertical direction, and in this embodiment, the direction of the first-dimension filter processing of the non-plane boundary region is set to be a horizontal direction, and the direction of the second-dimension filter processing is set to be a vertical direction. When the direction of the first-dimension filter processing of the non-surface boundary region is the horizontal direction, the first-dimension filter processing direction of the pair of surface boundary regions 4 and 4' is also the horizontal direction, that is, the first-dimension filter processing direction of the pair of surface boundary regions coincides with the first-dimension filter processing direction of the non-surface boundary region. And sequentially performing first-dimension and second-dimension filtering processing on the pixel blocks in the 4 and 4' in the opposite boundary area, namely performing first-dimension filtering processing according to a first-dimension filtering processing direction and then performing second-dimension filtering processing according to a second-dimension filtering processing direction.
As shown in fig. 11, when filtering the pixel blocks b, c, b ', and c ', the pixel blocks b ', c, b ', and c ' are rotated 90 ° counterclockwise and aligned with the pixel blocks b and c, and then the first-dimension and the second-dimension filtering are performed on the pixel blocks sequentially. As shown in fig. 12, the horizontal deblocking filtering process is first performed on the pixel blocks b 'and b, and c' to obtain the result after the filtering process, and then the vertical deblocking filtering process is performed on the pixel blocks b 'and c', and b and c. According to the deblocking filtering process employed in the present embodiment, the order of the first and second-dimensional filtering processes of the pixel blocks in the face boundary region 4 is different from the order of the first and second-dimensional filtering processes of the pixel blocks in the non-face boundary region.
The pixel blocks in the other pairs of surface boundary regions in fig. 6 that are adjacent on the spherical surface but not on the panoramic projection expansion image are also processed according to the method described in the present embodiment.
Example 3
The embodiment of the invention provides a panoramic image filtering method.
The panoramic image filtering method provided by the embodiment comprises the following steps:
determining the positions of surface boundary area pairs which are adjacent on a panoramic spherical surface and are not adjacent on the projection expansion image in the panoramic projection expansion image according to the format information of the panoramic projection expansion image;
determining the first dimension and the second dimension filtering processing direction in the surface boundary region pair according to the direction of a specific surface boundary region in the surface boundary region pair on the panoramic projection expansion image, wherein the first dimension and the second dimension filtering processing direction are determined by the surface boundary region pair and the non-surface boundary region on the panoramic projection expansion image;
and sequentially carrying out first-dimension and second-dimension filtering processing on the surface boundary region pair.
In this embodiment, if the directions of the two surface boundary regions in the surface boundary region pair are different, the specific surface boundary region is a surface boundary region in the horizontal direction in the surface boundary region pair.
In the pair of plane boundary regions 5 and 5 'of fig. 6, the direction of the plane boundary region 5 and the plane boundary region 5' on the panoramic projection expansion image is different. The surface boundary region 5' is a surface boundary region in the horizontal direction, and in this embodiment, the direction of the first-dimension filtering process of the non-surface boundary region is set to be the vertical direction, and the direction of the second-dimension filtering process is set to be the horizontal direction. When the first-dimension filtering processing direction of the non-surface boundary region is the vertical direction, the first-dimension filtering processing direction of the surface boundary region pair 5 and 5' is also the vertical direction, that is, the first-dimension filtering processing direction of the surface boundary region pair is consistent with the first-dimension filtering processing direction of the non-surface boundary region. And sequentially performing first-dimension and second-dimension filtering processing on the pixel blocks in the 5 and 5' in the opposite boundary area, namely performing first-dimension filtering processing according to a first-dimension filtering processing direction and then performing second-dimension filtering processing according to a second-dimension filtering processing direction.
As shown in fig. 13, when filtering the pixel blocks b, c, b ', and c', the pixel blocks b, c, b ', and c' are rotated 90 ° clockwise and then aligned with the pixel blocks b ', and c', and then the first-dimensional filtering and the second-dimensional filtering are performed on the pixel blocks sequentially. As shown in fig. 14, the deblocking filtering process is performed on the pixel blocks b 'and b, c' and c in the vertical direction to obtain the result after the filtering process, and then the deblocking filtering process is performed on the pixel blocks b and c, b 'and c' in the horizontal direction. According to the deblocking filtering process employed in the present embodiment, the first and second-dimensional filtering processing orders of the pixel blocks in the face boundary region 5 are the same as those of the pixel blocks in the non-face boundary region.
The pixel blocks in the other pairs of surface boundary regions in fig. 6 that are adjacent on the spherical surface but not on the panoramic projection expansion image are also processed according to the method described in the present embodiment.
Example 4
The embodiment of the invention provides a panoramic image filtering method.
The panoramic image filtering method provided by the embodiment comprises the following steps:
determining the positions of surface boundary area pairs which are adjacent on a panoramic spherical surface and are not adjacent on the projection expansion image in the panoramic projection expansion image according to the format information of the panoramic projection expansion image;
determining the first dimension and the second dimension filtering processing direction in the surface boundary region pair according to the direction of a specific surface boundary region in the surface boundary region pair on the panoramic projection expansion image, wherein the first dimension and the second dimension filtering processing direction are determined by the surface boundary region pair and the non-surface boundary region on the panoramic projection expansion image;
and sequentially carrying out first-dimension and second-dimension filtering processing on the surface boundary region pair.
In this embodiment, if the directions of the two surface boundary regions in the surface boundary region pair are different, the specific surface boundary region is a surface boundary region in the vertical direction in the surface boundary region pair.
In the pair of plane boundary regions 7 and 7 'of fig. 6, the direction of the plane boundary region 7 and the plane boundary region 7' on the panoramic projection expansion image is different. The plane boundary region 7' is a plane boundary region in the vertical direction, and in this embodiment, the direction of the first-dimension filtering process of the non-plane boundary region is set to be the horizontal direction, and the direction of the second-dimension filtering process is set to be the vertical direction. When the direction of the first-dimension filter processing of the non-surface boundary region is the horizontal direction, the first-dimension filter processing direction of the pair of surface boundary regions 7 and 5' is also the horizontal direction, that is, the first-dimension filter processing direction of the pair of surface boundary regions coincides with the first-dimension filter processing direction of the non-surface boundary region. And sequentially performing first-dimension and second-dimension filtering processing on the pixel blocks in the 7 and 7' in the opposite boundary area, namely performing first-dimension filtering processing according to a first-dimension filtering processing direction and then performing second-dimension filtering processing according to a second-dimension filtering processing direction.
As shown in fig. 15, when filtering the pixel blocks b, c, b ', and c', the pixel blocks b, c, b ', and c' are rotated 90 ° clockwise and then aligned with the pixel blocks b ', and c', and then the first-dimensional filtering and the second-dimensional filtering are performed on the pixel blocks sequentially. As shown in fig. 16, the horizontal deblocking filtering process is first performed on the pixel blocks b 'and b, c' and c to obtain the result after the filtering process, and then the vertical deblocking filtering process is performed on the pixel blocks b and c, b 'and c'. According to the deblocking filtering process employed in the present embodiment, the order of the first and second-dimensional filtering processes of the pixel blocks in the face boundary region 7 is different from the order of the first and second-dimensional filtering processes of the pixel blocks in the non-face boundary region.
The pixel blocks in the other pairs of surface boundary regions in fig. 6 that are adjacent on the spherical surface but not on the panoramic projection expansion image are also processed according to the method described in the present embodiment.
Example 5
The embodiment of the invention provides a panoramic image filtering device.
The panoramic image filtering apparatus provided in this embodiment includes:
a module for deriving the boundary region pair position of the surface to be filtered: the input of the module is format information of a panoramic projection expansion image, the position of a surface boundary area pair which is adjacent on a panoramic spherical surface and is not adjacent on the projection expansion image in the panoramic projection expansion image is determined according to the format information, and the output of the module is the position of the surface boundary area pair to be filtered in the panoramic projection expansion image;
a filtering processing direction determining module: the module inputs a pair of boundary areas of a surface to be filtered and the position of the boundary area pair in a panoramic projection expansion image, determines the direction of first-dimension and second-dimension filtering processing in the pair of boundary areas according to the direction of a specific boundary area in the pair of boundary areas on the panoramic projection expansion image, and the method for determining the direction of the first-dimension and the second-dimension filtering processing in the pair of boundary areas is consistent with the method for determining the direction of the first-dimension and the second-dimension filtering processing in the non-surface boundary area on the panoramic projection expansion image; the module outputs the first dimension and the second dimension of the boundary region pair of the surface to be filtered;
a filtering processing module: inputting the directions of the first dimension and the second dimension filtering processing of the surface boundary region pair to be filtered, and sequentially carrying out the first dimension and the second dimension filtering processing on the surface boundary region pair.
Fig. 17 is a schematic diagram of a panoramic image filtering apparatus according to the present embodiment. In the module for deriving the boundary area pair position of the to-be-filtered surface, the format information of the input panoramic projection expansion image determines the expansion mode of the panoramic projection expansion image, such as a cube format, EAC, PAU, and the like, in this embodiment, the cube format is taken as an example, and as shown in fig. 2, the panoramic projection expansion image is projected onto six faces of a cube, and 6 faces are arranged into a rectangular image of 3 × 2. The regions corresponding to the same numerals shown in fig. 6 are pairs of boundary regions of the faces to be filtered.
In the filter processing direction determining module, one of a pair of surface boundary regions needs to be specified, and the first and second filter processing directions of the pair of surface boundary regions are specified according to the direction of the surface boundary region in the panoramic projection expansion image. The designated face boundary region is one of the following 4 cases:
a) the designated surface boundary area is a surface boundary area which is centered on the surface boundary area and is subjected to encoding or decoding processing on the panoramic projection expansion image;
b) the designated surface boundary area is a surface boundary area which is centered on the panoramic projection expansion image and then is subjected to encoding or decoding processing;
c) if the directions of the two surface boundary areas in the surface boundary area pair are different, the specified surface boundary area is a surface boundary area in the horizontal direction in the surface boundary area pair;
d) if the directions of the two surface boundary areas in the surface boundary area pair are different, the specified surface boundary area is a surface boundary area in the vertical direction in the surface boundary area pair;
the filter processing direction determining module then specifies first and second-dimensional filter processing directions of the pair of face boundary regions according to the specified direction of the face boundary region.
And the filtering processing module sequentially performs first-dimension and second-dimension filtering processing on the surface boundary region pair. In the filtering processing module, for a common sub-region belonging to both surface boundary regions, the pixels filtered by the common sub-region are determined by the pixels filtered by the sub-region corresponding to the surface boundary region which is subjected to later encoding or decoding processing on the panorama expansion projection image in the both surface boundary regions.
The pixel blocks in the other pairs of surface boundary regions in fig. 6 that are adjacent on the spherical surface but not on the panoramic projection expansion image are also processed in accordance with the apparatus described in the present embodiment.
Example 6
The embodiment of the invention provides a panoramic image filtering method.
The panoramic image filtering method provided by the embodiment comprises the following steps:
and the pixels subjected to common subregion filtering are determined by subregions filtered pixels corresponding to surface boundary regions subjected to later encoding or decoding processing on the panoramic expansion projection image in the two surface boundary regions.
In this embodiment, as shown in fig. 18, the pixel blocks b belong to the surface boundary region 8 and the surface boundary region 5, and the pixel blocks b are common subregions in the panoramic image filtering method. The surface boundary area 8 'adjacent on the spherical surface but not adjacent on the panoramic projection expansion image is 8'; the area of the surface boundary 5 adjacent on the spherical surface but not on the panoramic projection expansion image is 5'. The spherical neighboring pixel block of the pixel block b in the surface boundary region 8' is b1', the spherical neighboring pixel block of the pixel block b in the surface boundary region 5' is b2'. At the pixel blocks b and b1' and b2'when performing the filtering process, the pixel block b in the plane boundary region 5 is placed in the pixel block b in the plane boundary region 5' after rotating the pixel block b by 90 ° clockwise2Upper part of the' as shown in fig. 19.
In the present embodiment, it is specified that the directions of the first and second dimensional filter processing of the pair of plane boundary regions are determined in accordance with the directions of the plane boundary regions post-encoded or decoded in the pair of plane boundary regions on the panorama projection expanded image. In this embodiment, if the first-dimension filtering processing direction of the non-planar boundary region is set to be the vertical direction, the pixel blocks b and b are processed1' and b2' performing a filtering processFirst, pixel blocks b and b are processed1' conducting deblocking Filter processing in vertical direction for pixel blocks b and b2' a vertical-direction deblocking filtering process is performed, and then horizontal-direction filtering is performed on the pixel blocks b in the face boundary region 8 and the pixel blocks b in the face boundary region 5. The result of filtering the block of pixels b on the panoramic projection expansion image is determined by the block of pixels b in the area of the surface boundary 5. Since the face boundary region 5 is post-processed in codec order with respect to the face boundary region 8.
If the pixel blocks are subjected to the deblocking filtering processing in serial order, the pixel blocks on the lower side and the right side of the panoramic image are subjected to the post processing, that is, the surface boundary region 5 is subjected to the post deblocking filtering processing relative to the surface boundary region 8 in fig. 19, and if the pixel blocks are subjected to the parallel deblocking filtering processing, the pixels subjected to the filtering by the common sub-region are still determined by the surface boundary region subjected to the post processing according to the serial processing, so that the consistency of the serial processing and the parallel processing is ensured.
The invention ensures that the coding and decoding ends can be matched when the opposite boundary region pair is subjected to filtering processing by determining the first dimension and the second dimension of the surface boundary region pair and the processing sequence.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present invention.
Claims (6)
1. A panoramic image filtering method, comprising:
determining the positions of surface boundary area pairs which are adjacent on a panoramic spherical surface and are not adjacent on the projection expansion image in the panoramic projection expansion image according to the format information of the panoramic projection expansion image;
determining the first dimension and the second dimension filtering processing direction in the surface boundary region pair according to the direction of a specific surface boundary region in the surface boundary region pair on the panoramic projection expansion image, wherein the first dimension and the second dimension filtering processing direction are determined by the surface boundary region pair and the non-surface boundary region on the panoramic projection expansion image;
and sequentially carrying out first-dimension and second-dimension filtering processing on the surface boundary region pair.
2. The panorama image pixel filtering method of claim 1, further characterized in that the specific plane boundary region is a plane boundary region of the pair of plane boundary regions that is subjected to a later encoding or decoding process on the panorama projection expanded image.
3. The method of filtering panoramic image pixels according to claim 1, further characterized in that, for a common subregion belonging to both face boundary regions, the common subregion filtered pixels are determined by subregions filtered pixels corresponding to face boundary regions of said both face boundary regions that are later encoded or decoded on the panoramic expansion projection image.
4. A panoramic image filtering device is characterized by comprising the following modules:
a module for deriving the boundary region pair position of the surface to be filtered: the input of the module is format information of a panoramic projection expansion image, the position of a surface boundary area pair which is adjacent on a panoramic spherical surface and is not adjacent on the projection expansion image in the panoramic projection expansion image is determined according to the format information, and the output of the module is the position of the surface boundary area pair to be filtered in the panoramic projection expansion image;
a filtering processing direction determining module: the module inputs a pair of boundary areas of a surface to be filtered and the position of the boundary area pair in a panoramic projection expansion image, determines the direction of first-dimension and second-dimension filtering processing in the pair of boundary areas according to the direction of a specific boundary area in the pair of boundary areas on the panoramic projection expansion image, and the method for determining the direction of the first-dimension and the second-dimension filtering processing in the pair of boundary areas is consistent with the method for determining the direction of the first-dimension and the second-dimension filtering processing in the non-surface boundary area on the panoramic projection expansion image; the module outputs the first dimension and the second dimension of the boundary region pair of the surface to be filtered;
a filtering processing module: inputting the directions of the first dimension and the second dimension filtering processing of the surface boundary region pair to be filtered, and sequentially carrying out the first dimension and the second dimension filtering processing on the surface boundary region pair.
5. The panorama image filtering apparatus of claim 4, further characterized in that a specific one of the plane boundary regions in the filtering process direction determining module is a plane boundary region of the pair of plane boundary regions that is subjected to a later encoding or decoding process on the panorama projection expanded image.
6. The panorama image filtering apparatus according to claim 4, wherein, in the filtering processing module, for a common subregion belonging to both face boundary regions, the common subregion-filtered pixel is determined by a subregion-filtered pixel corresponding to a face boundary region that is subjected to later encoding or decoding processing on the panorama expanded projection image in the both face boundary regions.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811111831 | 2018-09-24 | ||
| CN2018111118310 | 2018-09-24 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110942428A true CN110942428A (en) | 2020-03-31 |
Family
ID=69905213
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811154939.8A Pending CN110942428A (en) | 2018-09-24 | 2018-09-30 | Panoramic image filtering method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110942428A (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| TW201813380A (en) * | 2016-07-15 | 2018-04-01 | 聯發科技股份有限公司 | Method and apparatus for filtering 360-degree video boundaries |
| CN107948547A (en) * | 2017-12-29 | 2018-04-20 | 北京奇艺世纪科技有限公司 | Processing method, device and the electronic equipment of panoramic video splicing |
| CN108111851A (en) * | 2016-11-25 | 2018-06-01 | 华为技术有限公司 | A kind of block-removal filtering method and terminal |
-
2018
- 2018-09-30 CN CN201811154939.8A patent/CN110942428A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| TW201813380A (en) * | 2016-07-15 | 2018-04-01 | 聯發科技股份有限公司 | Method and apparatus for filtering 360-degree video boundaries |
| CN108111851A (en) * | 2016-11-25 | 2018-06-01 | 华为技术有限公司 | A kind of block-removal filtering method and terminal |
| CN107948547A (en) * | 2017-12-29 | 2018-04-20 | 北京奇艺世纪科技有限公司 | Processing method, device and the electronic equipment of panoramic video splicing |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109716766B (en) | Method and device for filtering 360-degree video boundary | |
| US11677926B1 (en) | Image data encoding/decoding method and apparatus | |
| CN111010568B (en) | Training method and device of interpolation filter, video image coding and decoding method and coder-decoder | |
| US20180242016A1 (en) | Deblock filtering for 360 video | |
| CN110892719A (en) | Image encoding/decoding method and apparatus | |
| CN109891895A (en) | Image coding/decoding method and its device | |
| CN111801947A (en) | Apparatus and method for image coding under selective loop filtering | |
| BR112020009019A2 (en) | Interpretation apparatus, method and computer program product for video encoding | |
| US11696035B2 (en) | Image data encoding/decoding method and apparatus | |
| US11212527B2 (en) | Entropy-inspired directional filtering for image coding | |
| CN107948547B (en) | Processing method and device for panoramic video stitching and electronic equipment | |
| JP2023040178A (en) | Image data encoding/decoding method and device | |
| CN116097649A (en) | Parallel encoding of video frames without filtering dependencies | |
| CN107959855B (en) | Motion compensated prediction method and apparatus | |
| CN110942428A (en) | Panoramic image filtering method and device | |
| WO2021162724A1 (en) | Intra prediction for image and video compression | |
| CN106303535B (en) | Image compression method and device with reference pixels taken from different-degree reconstruction pixels | |
| CN110349226B (en) | Panoramic image processing method and device | |
| US20210014511A1 (en) | Image processing apparatus and method | |
| CN110675401B (en) | Panoramic image pixel block filtering method and device | |
| WO2020007094A1 (en) | Panoramic image filtering method and device | |
| TW202034699A (en) | Image coding device, image decoding device, image coding method, image decoding method, and program | |
| CN117979031A (en) | Image filtering method, storage medium, electronic device and product | |
| Nozue et al. | Efficient Coding for Video Including Text Using Image Generation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |