CN108769680B - Slope-based segmented sampling method and device for panoramic video - Google Patents
Slope-based segmented sampling method and device for panoramic video Download PDFInfo
- Publication number
- CN108769680B CN108769680B CN201810544637.5A CN201810544637A CN108769680B CN 108769680 B CN108769680 B CN 108769680B CN 201810544637 A CN201810544637 A CN 201810544637A CN 108769680 B CN108769680 B CN 108769680B
- Authority
- CN
- China
- Prior art keywords
- slope
- sampling
- spherical surface
- region
- edge
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/132—Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/85—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Image Processing (AREA)
- Studio Circuits (AREA)
Abstract
The invention provides a slope-based segmented sampling method and device for a panoramic video. The invention is generated by spreading the spherical surface along the latitude lines and taking the center of the spherical surface after spreading as the originA two-dimensional rectangular coordinate system is adopted, and the slope k of the edge curve L after the spherical surface is unfolded is obtained; then using the weft W1、W2、W3、W4Dividing the expanded spherical surface into 5 regions, which are O in sequence from top to bottomiI =0,1,2,3, 4; and then, according to the value range of the slope k of the edge curve of each divided region, setting a fixed slope for each region. Then obtaining the shape of each area after sampling according to the slope; according to the spatial definition requirement, for a layer comprising OiEach region where i is 0,1,2,3,4 is sampled; and finally arranging the sampled images into a rectangle. The sampling structure redundancy is reduced by the sampling rectangle, so that the sampling point number is small, the data volume after sampling is small, and the compression efficiency is high under the condition of reaching the same spatial definition.
Description
Technical Field
The invention relates to a video coding technology, in particular to a slope-based segmented sampling method and device for panoramic video.
Background
The panoramic video refers to all scenes around an observation point in space and is composed of all light rays received by the observation point. Panoramic video can be abstracted as a sphere centered at the viewpoint.
When processing panoramic video with a computer, discretized spatial sampling of the panoramic video is inevitable. When the discretization space sampling is carried out on the panoramic video, certain space sampling density needs to be ensured so as to achieve the required definition. Meanwhile, considering that the memory of the computer is not suitable for storing the data of the spherical structure, the sampling points need to be arranged on the plane in some way.
The existing spherical sampling method generally divides the spherical sampling process into three steps: firstly, mapping, namely mapping a spherical surface into an area on a plane; secondly, planar sampling, wherein the distribution mode of sampling points is designed on the basis of the mapped planar area; and thirdly, arranging, namely arranging the sampling points into a rectangle.
In the first mapping process, the ratio of the areas of different areas of the spherical surface before and after mapping is different. In the second step of planar sampling, the sampling points are often designed to be uniformly distributed in the mapped planar area. The combination of the first step and the second step can cause uneven distribution of the sampling points on the spherical surface. Some of the region sampling points are dense, and some of the region sampling points are sparse. The space definition achieved by final sampling is based on the region with the sparsest sampling points on the spherical surface. Therefore, under the condition of given spatial definition, the uneven distribution of the sampling points on the spherical surface can bring redundancy. In the third step, the position distribution of the sampling points is changed in order to arrange the sampling points into a rectangle, so that the adjacent relation of the sampling points on the spherical surface is lost, and the subsequent predictive coding is not facilitated.
Currently, there are three common spherical sampling methods: longitude and latitude map sampling, hexahedron sampling and pyramid sampling.
Longitude and latitude map sampling As shown in FIG. 2, the longitude theta and latitude where any point on the sphere can be used
Describing that theta epsilon [0,2 pi),
the sphere can then be mapped to
The next rectangle in the coordinate system has an aspect ratio of 2: 1. This rectangle is sampled uniformly. The longitude and latitude map sampling method has high sampling density near two poles of a spherical surface, and generates great redundancy.
As shown in fig. 3, the hexahedron sampling is performed by firstly mapping a spherical surface to six surfaces of a circumscribed regular hexahedron thereof to obtain six plane squares, then uniformly sampling the six plane squares, and finally splicing the six squares into a rectangle in a certain manner. The distribution of hexahedron sampling points on the spherical surface is also uneven, and still larger redundancy exists.
Pyramid sampling as shown in fig. 4, a spherical surface is first mapped into a circumscribed regular rectangular pyramid, and each side surface (isosceles triangle) of the regular rectangular pyramid is compressed in the direction of the perpendicular line of the base until the vertex angle becomes a right angle. Thus, the bottom surface and the four secondarily mapped side surfaces can be just spliced into a square. Finally, this square is sampled uniformly. The distribution of sampling points of pyramid sampling on the spherical surface is also uneven, the uneven degree of the sampling points is between the longitude and latitude map sampling and the hexahedron sampling, and the redundancy degree is also between the longitude and latitude map sampling and the hexahedron sampling.
In summary, the main disadvantage of the existing panoramic video sampling method is that there is a large sampling structure redundancy. The sampling points are unevenly distributed on the spherical surface, so that not only can sampling redundancy be brought, but also inconvenience is brought to video quality evaluation, because the spherical areas corresponding to the final sampling points are different, the influence of the distortion of each sampling point on the video quality is different, and when the quality loss of the panoramic video after certain processing, such as compression and decompression, is calculated, the importance of each sampling point must be considered to be different.
Disclosure of Invention
In order to overcome the defects of the prior art, the invention aims to provide a slope-based segmented sampling method and a slope-based segmented sampling device for panoramic video, which can ensure that the distribution of sampling points on a spherical surface meets the condition that the number of sampling points near the equator is more and the number of sampling points near the two poles is less, thereby avoiding the redundancy of a sampling structure, reducing the number of sampling points under the condition of reaching the same spatial definition, and the spliced rectangle has higher content adjacency and can improve the coding compression efficiency.
In order to achieve the above and other objects, the present invention provides a method for sampling a panoramic video based on slope segments, comprising the following steps:
step one, as shown in fig. 5, the ball is put inThe surface is spread along the latitude lines, and the spread widths at different latitudes on the spherical surface can be seen to change along with the change of the latitudes, the equator is widest, and the two poles are narrowest. Generating a two-dimensional rectangular coordinate system by taking the center of the spherical surface after expansion as an origin; taking the connecting line from the leftmost point to the rightmost point after the spherical surface expansion as the abscissa axis, the value range from left to right is [ -pi, pi [ -pi [ ]]Corresponding to longitude on a sphere; the connecting line from the lowest point to the highest point after the spherical surface is unfolded is taken as the ordinate axis, and the value range from bottom to top is
Corresponding to the latitude on the spherical surface; meanwhile, the slope k of the edge curve L after the spherical surface is expanded can be obtained.
Step two, as shown in FIG. 6, with the weft W1、W2、W3、W4Dividing the expanded spherical surface into 5 regions, which are O in sequence from top to bottomi，i＝0,1,2,3,4。
And step three, setting a fixed slope for each region according to the value range of the edge curve slope k of each segmented region. Wherein the region O0Is set to
The slope of the right edge is
O1The slope of the left edge is set to be 1, and the slope of the right edge is-1; o is2The slope of the left edge is set to be infinite, and the slope of the right edge is infinite; o is3The slope of the left edge is set to be-1, and the slope of the right edge is set to be 1; o is4Left edge slope is set to
The slope of the right edge is
Then, the sampled shape of each region is obtained according to the slope, as shown in fig. 7.
Step four, according to the blankInter-resolution requirement for including OiEach region where i is 0,1,2,3,4 is sampled, and the shape after sampling is as shown in step three.
And step five, arranging the sampled images into a rectangle, wherein the arrangement method is shown in fig. 8.
Further, in the above method, in the first step, the slope of the portion of the edge curve L in the second quadrant and the third quadrant
Wherein y is latitude; and the slope of the part of the edge curve L in the first and fourth quadrants
Where y is the latitude.
Further, in the above method, in the second step, the weft W1,W2At a latitude greater than zero, W1At W2Above; weft W3,W4The latitude is less than zero, W3At W4Above. And the weft W1,W2,W3,W4At latitude Yw1，Yw2，Yw3，Yw4The following conditions are satisfied:
further, in the above method, in step three, after each region is sampled, O0Lower boundary of (1) and (O)1Do not have to be the same length, O1Lower boundary of (1) and (O)2Must be of the same length, O2Lower boundary of (1) and (O)3Must be of the same length, O3Lower boundary of (1) and (O)4Are not necessarily the same length.
Further, in the above method, in the fourth step, the region O in the vicinity of the equator of the spherical surface2The sum of the transverse sampling points is M; o is1And O2、O3Total 3 areasThe sum of the vertical sampling points is N.
Further, in the above method, in the fifth step, in the spliced rectangle, the original region O0The pixel point at the edge of the rectangle is not adjacent to the surrounding area; original region O4The pixel point at the edge of the rectangle is not adjacent to the surrounding area; original region O1In the upper left position of the rectangle; original region O3In the lower left position of the rectangle; original region O2In the center of the rectangle. The width of the spliced rectangle is M, and the height of the spliced rectangle is N.
Further, in the above method, the method further comprises: according to the requirement of spatial definition, when storing, firstly recording N and M, M is the area O2The sum of the horizontal sampling points; n is O1And O2、O3The sum of longitudinal sampling points of 3 regions in total; the data obtained for all the sampling points are then arranged in a column in the following order: (0,0), (0,1),., (0, M-1), (1,0), (1,1),., (1, M-1),.., (N-1,0), (N-1,1),. and (N-1, M-1).
Further, in the above method, the method further comprises: when the data is displayed on a flat panel display, the obtained sampling point data is arranged in a rectangular area with N rows and M columns, the data with the serial number (i,0) of each row is aligned, and other data are sequentially arranged.
In order to achieve the above object, the present invention further provides a slope-based segmented sampling apparatus for panoramic video, which is applied to the above method operations, and includes:
the spherical surface unfolding unit, as shown in fig. 5, spreads the spherical surface along the latitude lines, and it can be seen that the unfolding widths at different latitudes on the spherical surface vary with the latitude, with the widest at the equator and the narrowest at the two poles. Generating a two-dimensional rectangular coordinate system by taking the center of the spherical surface after expansion as an origin; taking the connecting line from the leftmost point to the rightmost point after the spherical surface expansion as the abscissa axis, the value range from left to right is [ -pi, pi [ -pi [ ]]Corresponding to longitude on a sphere; the connecting line from the lowest point to the highest point after the spherical surface is unfolded is taken as the ordinate axis, and the value range from bottom to top is
Corresponding to the latitude on the spherical surface; meanwhile, the slope k of the edge curve L after the spherical surface is unfolded can be obtained;
spherical surface dividing means, as shown in FIG. 6, using a weft W1、W2、W3、W4Dividing the expanded spherical surface into 5 regions, which are O in sequence from top to bottomi，i＝0,1,2,3,4；
And the slope distribution unit sets a fixed slope for each region according to the value range of the edge curve slope k of each segmented region. Wherein the region O0Is set to
The slope of the right edge is
O1The slope of the left edge is set to be 1, and the slope of the right edge is-1; o is2The slope of the left edge is set to be infinite, and the slope of the right edge is infinite; o is3The slope of the left edge is set to be-1, and the slope of the right edge is set to be 1; o is4Left edge slope is set to
The slope of the right edge is
Then, obtaining the shape of each region after sampling according to the slope, as shown in fig. 7;
the sampling value calculation unit samples each area including Oi, i ═ 0,1,2,3 and 4 according to the requirement of spatial definition, and the shape after sampling is shown in step three;
the rearranging unit arranges the sampled images in a rectangular shape, and the arrangement method is as shown in fig. 8.
Further, in the above device, the slope of the portion of the edge curve L in the second and third quadrants of the sphere expansion unit
Wherein y is latitude; and the slope of the part of the edge curve L in the first and fourth quadrants
Wherein y is latitude;
further, in the above device, the spherical surface dividing unit, the weft W1,W2At a latitude greater than zero, W1At W2Above; weft W3,W4The latitude is less than zero, W3At W4Above. And the weft W1,W2,W3,W4At latitude Yw1，Yw2，Yw3，Yw4The following conditions are satisfied:
further, in the above apparatus, the slope assigning unit may sample each region, and then0Lower boundary of (1) and (O)1Do not have to be the same length, O1Lower boundary of (1) and (O)2Must be of the same length, O2Lower boundary of (1) and (O)3Must be of the same length, O3Lower boundary of (1) and (O)4The length of the upper boundary of (a) is not necessarily the same;
further, in the above apparatus, the sampling value calculation unit may be located in a region O near the equator of the spherical surface2The sum of the transverse sampling points is M; o is1And O2、O3The sum of longitudinal sampling points of 3 regions is N;
further, in the above apparatus, the rearranging unit may rearrange the original region O in the rectangular region after the splicing0The pixel point at the edge of the rectangle is not adjacent to the surrounding area; original region O4The pixel point at the edge of the rectangle is not adjacent to the surrounding area; original region O1In the upper left position of the rectangle; original region O3At the position ofThe lower left position of the rectangle; original region O2In the center of the rectangle. The width of the spliced rectangle is M, and the height of the spliced rectangle is N;
further, in the above apparatus, a storage module is further included, configured to record N and M as an area O first during storage according to a spatial definition requirement, where M is an area2The sum of the horizontal sampling points; n is O1And O2、O3The sum of longitudinal sampling points of 3 regions in total; the data obtained for all the sampling points are then arranged in a column in the following order: (0,0), (0,1),., (0, M-1), (1,0), (1,1),., (1, M-1),.., (N-1,0), (N-1,1),. and (N-1, M-1).
Further, in the above apparatus, the apparatus further includes a display module, configured to arrange the obtained sample point data in a rectangular area with N rows and M columns, align data with each row numbered as (i,0), and arrange other data sequentially when displaying on the flat panel display.
Compared with the prior art, the invention has obvious prominent substantive characteristics and remarkable technical progress: the slope-based segmented sampling method and device for the panoramic video reduce the redundancy of a sampling structure, so that the sampling points are few, the data volume after sampling is small and the compression efficiency is high under the condition of reaching the same spatial definition.
Drawings
Fig. 1 is a block diagram of a sampling method process according to the present invention.
Fig. 2 is a schematic diagram of longitude and latitude map sampling in a spherical sampling method in the prior art.
Fig. 3 is a schematic diagram of hexahedral sampling in a spherical sampling method of the prior art.
Fig. 4 is an experimental diagram of pyramid sampling in a prior art spherical sampling method.
Fig. 5 is a schematic diagram of spherical expansion of a panoramic video based on a slope piecewise sampling method.
Fig. 6 is a schematic diagram of a panoramic video divided by wefts after spherical expansion based on a slope segmented sampling method.
Fig. 7 is a schematic diagram of determining slopes and shapes of the regions after spherical segmentation of the panoramic video based on the slope segmentation sampling method.
Fig. 8 is a schematic diagram of image rearrangement after sampling of a panoramic video based on a slope piecewise sampling method according to the present invention.
Detailed Description
In order to make the aforementioned objects, features and advantages of the present invention comprehensible, preferred embodiments accompanied with figures are described below.
The first embodiment is as follows: referring to fig. 5 to 8, the slope-based segmented sampling method for panoramic video is characterized by comprising the following operation steps:
step one, spreading the spherical surface along the latitude lines, and observing that the spreading widths at different latitudes on the spherical surface change along with the change of the latitude, wherein the equator is widest, and the two poles are narrowest. Generating a two-dimensional rectangular coordinate system by taking the center of the spherical surface after expansion as an origin; taking a connecting line from the leftmost point to the rightmost point after the spherical surface is unfolded as an abscissa axis, wherein the range from left to right is [ -pi, pi ], and the range corresponds to longitude on the spherical surface; taking a connecting line from the lowest point to the highest point after the spherical surface is unfolded as an ordinate axis, wherein the value range from bottom to top is [ -pi/2, pi/2 ], and the value range corresponds to the latitude on the spherical surface; meanwhile, the slope k of the edge curve L after the spherical surface is unfolded can be obtained;
step two, using the weft W1、W2、W3、W4Dividing the expanded spherical surface into 5 areas, wherein the areas are sequentially Oi, i is 0,1,2,3 and 4 from top to bottom;
step three, setting a fixed slope for each region according to the value range of the edge curve slope k of each segmented region; wherein the region O0The slope of the left edge of (a) is set to 1/4, and the slope of the right edge is-1/4; o is1The slope of the left edge is set to be 1, and the slope of the right edge is-1; o is2The slope of the left edge is set to be infinite, and the slope of the right edge is infinite; o is3The slope of the left edge is set to be-1, and the slope of the right edge is set to be 1; o is4The slope of the left edge is set to-1/4 and the slope of the right edge is 1/4; and then obtaining the shape of each region after sampling according to the slope.
Step four, according to the space definition requirement, theComprising OiEach region where i is 0,1,2,3,4 is sampled, and the shape after sampling is as shown in step three;
and step five, arranging the sampled images into a rectangle.
Example two: the slope-based segmented sampling device for the panoramic video is applied to the operation of the method and is characterized by comprising the following steps of:
a spherical expansion unit: the spherical surface is spread along the latitude lines, so that the spreading widths at different latitudes on the spherical surface are changed along with the change of the latitude, the equator is widest, and the two poles are narrowest. Generating a two-dimensional rectangular coordinate system by taking the center of the spherical surface after expansion as an origin; taking a connecting line from the leftmost point to the rightmost point after the spherical surface is unfolded as an abscissa axis, wherein the range from left to right is [ -pi, pi ], and the range corresponds to longitude on the spherical surface; taking a connecting line from the lowest point to the highest point after the spherical surface is unfolded as an ordinate axis, wherein the value range from bottom to top is [ -pi/2, pi/2 ], and the value range corresponds to the latitude on the spherical surface; meanwhile, the slope k of the edge curve L after the spherical surface is unfolded can be obtained;
a spherical surface dividing unit: with weft threads W1、W2、W3、W4Dividing the expanded spherical surface into 5 regions, which are O in sequence from top to bottomi，i＝0,1,2,3,4；
A slope allocation unit: and setting a fixed slope for each region according to the value range of the slope k of the edge curve of each segmented region. Wherein the region O0The slope of the left edge of (a) is set to 1/4, and the slope of the right edge is-1/4; o is1The slope of the left edge is set to be 1, and the slope of the right edge is-1; o is2The slope of the left edge is set to be infinite, and the slope of the right edge is infinite; o is3The slope of the left edge is set to be-1, and the slope of the right edge is set to be 1; o is4The left edge slope is set to-1/4 and the right edge slope is 1/4. Then obtaining the shape of each area after sampling according to the slope;
a sampling value calculation unit: according to the spatial definition requirement, for a layer comprising OiEach region where i is 0,1,2,3,4 is sampled, and the shape after sampling is as shown in step three;
a rearranging unit: the sampled images are arranged in a rectangle.
Example three: the slope-based segmented sampling method for the panoramic video comprises the following steps:
one of the images of the panoramic video, which in this particular embodiment of the invention is a color video, has three components, represented by a warp and weft map with a resolution of 4096 x 2048. Assuming that the color components used are RGB, the color of each sample point is still represented by RGB after sampling. Assuming that the requirement of spatial resolution is the same for three components, N1368 and M3456, the quantization accuracy requirement is the same for each component, all quantized to 256 levels. Then for each component, repeating steps one through five as follows:
step one, as shown in fig. 5, the spherical surface is spread along the latitude lines, and it can be seen that the spread widths at different latitudes on the spherical surface vary with the latitude, with the widest at the equator and the narrowest at the two poles. Generating a two-dimensional rectangular coordinate system by taking the center of the spherical surface after expansion as an origin; taking the connecting line from the leftmost point to the rightmost point after the spherical surface expansion as the abscissa axis, the value range from left to right is [ -pi, pi [ -pi [ ]]Corresponding to longitude on a sphere; the connecting line from the lowest point to the highest point after the spherical surface is unfolded is taken as the ordinate axis, and the value range from bottom to top is [ -pi/2, pi/2]Corresponding to the latitude on the sphere; meanwhile, the slope of the part of the edge curve L in the second quadrant and the third quadrant after the spherical expansion can be obtained
Wherein y is latitude; and the slope of the part of the edge curve L in the first and fourth quadrants
Where y is the latitude.
Step two, as shown in FIG. 6, the latitude is sequentially
And
weft ofWire W1、W2、W3、W4Dividing the expanded spherical surface into 5 regions, which are O in sequence from top to bottomi，i＝0,1,2,3,4。
And step three, setting a fixed slope for each region according to the value range of the edge curve slope k of each segmented region. Wherein the region O0The slope of the left edge of (a) is set to 1/4, and the slope of the right edge is-1/4; o is1The slope of the left edge is set to be 1, and the slope of the right edge is-1; o is2The slope of the left edge is set to be infinite, and the slope of the right edge is infinite; o is3The slope of the left edge is set to be-1, and the slope of the right edge is set to be 1; o is4The left edge slope is set to-1/4 and the right edge slope is 1/4. Then, the sampled shape of each region is obtained according to the slope, as shown in fig. 7.
Step four, according to the space definition requirement, for the product containing OiEach region where i is 0,1,2,3,4 is sampled, and the shape after sampling is shown in fig. 7 in step three. Wherein a region O in the vicinity of the equator of the spherical surface2The sum of the transverse sampling points is 3456; o is1And O2、O3The sum of the 3 region longitudinal sample points is 1368.
And step five, arranging the sampled images into a rectangle, wherein the arrangement method is shown in fig. 8. In the spliced rectangle, the original region O0The pixel point at the edge of the rectangle is not adjacent to the surrounding area; original region O4The pixel point at the edge of the rectangle is not adjacent to the surrounding area; original region O1In the upper left position of the rectangle; original region O3In the lower left position of the rectangle; original region O2In the center of the rectangle. The width of the spliced rectangle is 3456 and the height is 1368.
During storage, firstly, the line number 1368 of each component sample and the number 3456 of sampling points of each line are recorded; the data obtained for all the sampling points are then arranged in a column in the following order: (0,0),(0,1),...,(0,3455),(1,0),(1,1),...,(1,3455),...,(1367,0),(1367,1),...,(1367,3455). The three components of the same sample point are arranged in the order of B, G, R.
When the data is displayed on a flat panel display, the obtained sampling point data is arranged in a rectangular area with 1368 rows and 3456 columns, the data with the number (i,0) of each row is aligned, and other data are arranged in sequence.
Example four: the slope-based segmented sampling method for the panoramic video comprises the following steps:
one of the images of the panoramic video, which in this particular embodiment of the invention is a color video, has three components, represented by a warp and weft map with a resolution of 4096 x 2048. Assuming that the color component used is YcbCr, the sampling ratio is 4:4:4, and the color of each sample point is still represented by YcbCr after sampling. Assuming that the requirement of spatial sharpness is N1368 for the Y component, M3456 for the Cb and Cr components N684, and M1728, the quantization accuracy requirement is the same for each component, all quantized to 256 levels.
Then for the Y component, the following steps are performed:
step one, as shown in fig. 5, the spherical surface is spread along the latitude lines, and it can be seen that the spread widths at different latitudes on the spherical surface vary with the latitude, with the widest at the equator and the narrowest at the two poles. Generating a two-dimensional rectangular coordinate system by taking the center of the spherical surface after expansion as an origin; taking the connecting line from the leftmost point to the rightmost point after the spherical surface expansion as the abscissa axis, the value range from left to right is [ -pi, pi [ -pi [ ]]Corresponding to longitude on a sphere; the connecting line from the lowest point to the highest point after the spherical surface is unfolded is taken as the ordinate axis, and the value range from bottom to top is [ -pi/2, pi/2]Corresponding to the latitude on the sphere; meanwhile, the slope of the part of the edge curve L in the second quadrant and the third quadrant after the spherical expansion can be obtained
Wherein y is latitude; and the slope of the part of the edge curve L in the first and fourth quadrants
Where y is the latitude.
Step two, as shown in FIG. 6, the latitude is sequentially
And
the weft W of1、W2、W3、W4Dividing the expanded spherical surface into 5 regions, which are O in sequence from top to bottomi，i＝0,1,2,3,4。
And step three, setting a fixed slope for each region according to the value range of the edge curve slope k of each segmented region. Wherein the region O0The slope of the left edge of (a) is set to 1/4, and the slope of the right edge is-1/4; o is1The slope of the left edge is set to be 1, and the slope of the right edge is-1; o is2The slope of the left edge is set to be infinite, and the slope of the right edge is infinite; o is3The slope of the left edge is set to be-1, and the slope of the right edge is set to be 1; o is4The left edge slope is set to-1/4 and the right edge slope is 1/4. Then, the sampled shape of each region is obtained according to the slope, as shown in fig. 7.
Step four, according to the space definition requirement, for the product containing OiEach region where i is 0,1,2,3,4 is sampled, and the shape after sampling is shown in fig. 7 in step three. Wherein a region O in the vicinity of the equator of the spherical surface2The sum of the transverse sampling points is 3456; o is1And O2、O3The sum of the 3 region longitudinal sample points is 1368.
And step five, arranging the sampled images into a rectangle, wherein the arrangement method is shown in fig. 8. In the spliced rectangle, the original region O0The pixel point at the edge of the rectangle is not adjacent to the surrounding area; original region O4The pixel point at the edge of the rectangle is not adjacent to the surrounding area; original region O1In the upper left position of the rectangle; original region O3In the lower left position of the rectangle; original region O2In the center of the rectangle. The width of the spliced rectangle is 3456 and the height is 1368.
Then, for the Cb and Cr components, the following steps are performed, respectively:
step one, as shown in fig. 5, the spherical surface is put alongThe wefts are spread horizontally, so that the spread widths at different latitudes on the spherical surface can be seen to change along with the change of the latitudes, the equator is widest, and the two poles are narrowest. Generating a two-dimensional rectangular coordinate system by taking the center of the spherical surface after expansion as an origin; taking the connecting line from the leftmost point to the rightmost point after the spherical surface expansion as the abscissa axis, the value range from left to right is [ -pi, pi [ -pi [ ]]Corresponding to longitude on a sphere; the connecting line from the lowest point to the highest point after the spherical surface is unfolded is taken as the ordinate axis, and the value range from bottom to top is [ -pi/2, pi/2]Corresponding to the latitude on the sphere; meanwhile, the slope of the part of the edge curve L in the second quadrant and the third quadrant after the spherical expansion can be obtained
Wherein y is latitude; and the slope of the part of the edge curve L in the first and fourth quadrants
Where y is the latitude.
Step two, as shown in FIG. 6, the latitude is sequentially
And
the weft W of1、W2、W3、W4Dividing the expanded spherical surface into 5 regions, which are O in sequence from top to bottomi，i＝0,1,2,3,4。
And step three, setting a fixed slope for each region according to the value range of the edge curve slope k of each segmented region. Wherein the region O0The slope of the left edge of (a) is set to 1/4, and the slope of the right edge is-1/4; o is1The slope of the left edge is set to be 1, and the slope of the right edge is-1; o is2The slope of the left edge is set to be infinite, and the slope of the right edge is infinite; o is3The slope of the left edge is set to be-1, and the slope of the right edge is set to be 1; o is4The left edge slope is set to-1/4 and the right edge slope is 1/4. Then, the sampled shape of each region is obtained according to the slope, as shown in FIG. 7Shown in the figure.
Step four, according to the space definition requirement, for the product containing OiEach region where i is 0,1,2,3,4 is sampled, and the shape after sampling is shown in fig. 7 in step three. Wherein a region O in the vicinity of the equator of the spherical surface2The sum of the lateral sampling points is 1728; o is1And O2、O3The sum of the 3 region longitudinal sampling points is 684.
And step five, arranging the sampled images into a rectangle, wherein the arrangement method is shown in fig. 8. In the spliced rectangle, the original region O0The pixel point at the edge of the rectangle is not adjacent to the surrounding area; original region O4The pixel point at the edge of the rectangle is not adjacent to the surrounding area; original region O1In the upper left position of the rectangle; original region O3In the lower left position of the rectangle; original region O2In the center of the rectangle. The width of the rectangle after splicing is 1728, and the height is 684.
In storage, first, the number of lines 1368 and 3456 per line of Y component samples, the number of lines 684 and 1728 per line of Cb component samples, the number of lines 684 and the number 1728 per line of Cr component samples are recorded. Then arranging the data of the obtained Y component sampling points into a column according to the following sequence: (0,0),(0,1),...,(0,3455),(1,0),(1,1),...,(1,3455),...,(1367,0),(1367,1),...,(1367,3455). The data of the resulting Cb component sample points are arranged in a column in the following order after the data of the Y component sample points: (0,0),(0,1),...,(0,1727),(1,0),(1,1),...,(1,1727),...,(683,0),(683,1),...,(683,1727). The data of the resulting Cr component sample points are arranged in a column following the data of the Cb component sample points in the following order: (0,0),(0,1),...,(0,1727),(1,0),(1,1),...,(1,1727),...,(683,0),(683,1),...,(683,1727).
In summary, the slope-based segmented sampling method and device for the panoramic video, provided by the invention, reduce the redundancy of the sampling structure, and maintain the adjacent relation of the sampling points on the spherical surface, so that under the condition of reaching the same spatial definition, the number of the sampling points is small, the data volume after sampling is small, and the compression efficiency is high.
The embodiments in the present description are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other.
Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative components and steps have been described above generally in terms of their functionality in order to clearly illustrate this interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
It will be apparent to those skilled in the art that various changes and modifications may be made in the invention without departing from the spirit and scope of the invention. Thus, if such modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include such modifications and variations.
Claims (12)
1. A slope-based segmented sampling method for panoramic video is characterized by comprising the following operation steps:
step one, spreading a spherical surface along a weft, and observing that the spreading width of different latitudes on the spherical surface changes along with the change of the latitude, wherein the equator is widest and the two poles are narrowest; generating a two-dimensional rectangular coordinate system by taking the center of the spherical surface after expansion as an origin; taking the connecting line from the leftmost point to the rightmost point after the spherical surface expansion as the abscissa axis, the value range from left to right is [ -pi, pi [ -pi [ ]]Corresponding to longitude on a sphere; the connecting line from the lowest point to the highest point after the spherical surface is unfolded is taken as the ordinate axis, and the value range from bottom to top is
Corresponding to the latitude on the spherical surface;meanwhile, obtaining the slope k of the edge curve L after the spherical surface is unfolded;
step two, using the weft W1、W2、W3、W4Dividing the expanded spherical surface into 5 regions, which are O in sequence from top to bottomiI is 0,1,2,3, 4; weft W1,W2At a latitude greater than zero, W1At W2Above; weft W3,W4The latitude is less than zero, W3At W4Above; and the weft W1,W2,W3,W4At the latitude
The following conditions are satisfied:
step three, setting a fixed slope for each region according to the value range of the edge curve slope k of each segmented region; wherein the region O0Is set to
The slope of the right edge is
O1The slope of the left edge is set to be 1, and the slope of the right edge is-1; o is2The slope of the left edge is set to be infinite, and the slope of the right edge is infinite; o is3The slope of the left edge is set to be-1, and the slope of the right edge is set to be 1; o is4Left edge slope is set to
The slope of the right edge is
Then obtaining the shape of each area after sampling according to the slope; after each region has been sampled, O1Lower boundary of (1) and (O)2Length of upper boundary ofMust be identical, O2Lower boundary of (1) and (O)3Must be of the same length, O3Lower boundary of (1) and (O)4The length of the upper boundary of (a) is not necessarily the same;
step four, according to the space definition requirement, for the product containing OiEach region where i is 0,1,2,3,4 is sampled, and the shape after sampling is as shown in step three;
step five, arranging the sampled images into a rectangle:
in the spliced rectangle, the original region O0The pixel point at the edge of the rectangle is not adjacent to the surrounding area; original region O4The pixel point at the edge of the rectangle is not adjacent to the surrounding area; original region O1In the upper left position of the rectangle; original region O3In the lower left position of the rectangle; original region O2In the center of the rectangle.
2. The slope-based piecewise sampling method of claim 1, wherein: in the first step, the slope of the part of the edge curve L in the second quadrant and the third quadrant
Wherein y is latitude; and the slope of the part of the edge curve L in the first and fourth quadrants
Where y is the latitude.
3. The slope-based piecewise sampling method of claim 2, wherein: in the third step, after each area is sampled, O0Lower boundary of (1) and (O)1Are not necessarily the same length.
4. The slope-based piecewise sampling method of claim 3, wherein: in the fourth step, on the equator of the sphereNear region O2The sum of the transverse sampling points is M; o is1And O2、O3The sum of the longitudinal sampling points of the 3 regions is N.
5. The method of slope-based piecewise sampling of panoramic video of claim 4, wherein: in the fifth step, the width of the spliced rectangle is M, and the height of the spliced rectangle is N.
6. The slope-based piecewise sampling method of claim 5, wherein: according to the requirement of spatial definition, when storing, firstly recording N and M, M is the area O2The sum of the horizontal sampling points; n is O1And O2、O3The sum of longitudinal sampling points of 3 regions in total; the data obtained for all the sampling points are then arranged in a column in the following order: (0,0), (0,1),., (0, M-1), (1,0), (1,1),., (1, M-1),.., (N-1,0), (N-1,1),. and (N-1, M-1).
7. The method of slope-based piecewise sampling of panoramic video of claim 6, wherein: when the data is displayed on a flat panel display, the obtained sampling point data is arranged in a rectangular area with N rows and M columns, the data with the serial number (i,0) of each row is aligned, and other data are sequentially arranged.
8. A slope-based piecewise sampling apparatus for panoramic video, which implements the slope-based piecewise sampling method of claim 1, wherein the slope-based piecewise sampling apparatus for panoramic video comprises:
a spherical expansion unit: spreading the spherical surface along the latitude lines, and observing that the spreading widths at different latitudes on the spherical surface change along with the change of the latitudes, wherein the equator is widest, and the two poles are narrowest; generating a two-dimensional rectangular coordinate system by taking the center of the spherical surface after expansion as an origin; taking the connecting line from the leftmost point to the rightmost point after the spherical surface expansion as the abscissa axis, the value range from left to right is [ -pi, pi [ -pi [ ]]Corresponding to longitude on a sphere; the connecting line from the lowest point to the highest point after the spherical surface is unfolded is taken as the ordinateAxis, from bottom to top, having a value range of
Corresponding to the latitude on the spherical surface; meanwhile, obtaining the slope k of the edge curve L after the spherical surface is unfolded;
a spherical surface dividing unit: with weft threads W1、W2、W3、W4Dividing the expanded spherical surface into 5 regions, which are O in sequence from top to bottomiI is 0,1,2,3, 4; weft W1,W2At a latitude greater than zero, W1At W2Above; weft W3,W4The latitude is less than zero, W3At W4Above; and the weft W1,W2,W3,W4At the latitude
The following conditions are satisfied:
a slope allocation unit: setting a fixed slope for each region according to the value range of the slope k of the edge curve of each segmented region; wherein the region O0Is set to
The slope of the right edge is
O1The slope of the left edge is set to be 1, and the slope of the right edge is-1; o is2The slope of the left edge is set to be infinite, and the slope of the right edge is infinite; o is3The slope of the left edge is set to be-1, and the slope of the right edge is set to be 1; o is4Left edge slope is set to
The slope of the right edge is
Then obtaining the shape of each area after sampling according to the slope; after each region has been sampled, O1Lower boundary of (1) and (O)2Must be of the same length, O2Lower boundary of (1) and (O)3Must be of the same length, O3Lower boundary of (1) and (O)4The length of the upper boundary of (a) is not necessarily the same;
a sampling value calculation unit: sampling each region including Oi, i ═ 0,1,2,3,4 according to the spatial definition requirement, the shape after sampling being as shown in step three;
a rearranging unit: arranging the sampled images into a rectangle:
in the spliced rectangle, the original region O0The pixel point at the edge of the rectangle is not adjacent to the surrounding area; original region O4The pixel point at the edge of the rectangle is not adjacent to the surrounding area; original region O1In the upper left position of the rectangle; original region O3In the lower left position of the rectangle; original region O2In the center of the rectangle.
9. The panoramic video slope-based piecewise sampling apparatus of claim 8, wherein the sphere expansion unit: slope of the part of the edge curve L in the second and third quadrants
Wherein y is latitude; and the slope of the part of the edge curve L in the first and fourth quadrants
Where y is the latitude.
10. The panoramic video slope-based segmented sampling apparatus of claim 9, wherein the sample value calculation unit: on the spherical surface equatorNear region O2The sum of the transverse sampling points is M; o is1And O2、O3The sum of the longitudinal sampling points of the 3 regions is N.
11. The apparatus for sampling panorama video based on slope segments according to claim 10, further comprising a storage module: for recording N and M as region O first during storage according to space definition requirement2The sum of the horizontal sampling points; n is O1And O2、O3The sum of longitudinal sampling points of 3 regions in total; the data obtained for all the sampling points are then arranged in a column in the following order: (0,0), (0,1),., (0, M-1), (1,0), (1,1),., (1, M-1),.., (N-1,0), (N-1,1),. and (N-1, M-1).
12. The apparatus for slope-based piecewise sampling of panoramic video according to claim 11, further comprising a display module for: when the data is displayed on a flat panel display, the obtained sampling point data are arranged in a rectangular area with N rows and M columns, the data with the serial number (i,0) of each row are aligned, and other data are sequentially arranged.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810544637.5A CN108769680B (en) | 2018-05-31 | 2018-05-31 | Slope-based segmented sampling method and device for panoramic video |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810544637.5A CN108769680B (en) | 2018-05-31 | 2018-05-31 | Slope-based segmented sampling method and device for panoramic video |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108769680A CN108769680A (en) | 2018-11-06 |
| CN108769680B true CN108769680B (en) | 2021-02-23 |
Family
ID=64000855
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810544637.5A Active CN108769680B (en) | 2018-05-31 | 2018-05-31 | Slope-based segmented sampling method and device for panoramic video |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108769680B (en) |
Family Cites Families (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| GB2524249B (en) * | 2014-03-17 | 2021-01-20 | Sony Interactive Entertainment Inc | Image Processing |
| CN105335748B (en) * | 2014-08-07 | 2018-10-12 | 株式会社理光 | Image characteristic extracting method and system |
| CN104217459B (en) * | 2014-09-04 | 2017-03-29 | 天津大学 | A kind of spheroid character extracting method |
| CN106210716B (en) * | 2016-08-01 | 2019-08-23 | 上海国茂数字技术有限公司 | A kind of panoramic video isodensity method of sampling and device |
| CN107230180B (en) * | 2016-10-10 | 2019-04-12 | 华为技术有限公司 | A kind of mapping method of panoramic picture, device and equipment |
| CN106488233B (en) * | 2016-10-31 | 2019-08-23 | 上海国茂数字技术有限公司 | A kind of panoramic video diamond shape method of sampling and device |
| CN106658009A (en) * | 2016-12-29 | 2017-05-10 | 上海国茂数字技术有限公司 | Improved double-ring sampling method and device for panoramic video |
| CN106791840A (en) * | 2016-12-29 | 2017-05-31 | 上海国茂数字技术有限公司 | A kind of bicyclic band method of sampling and device of panoramic video |
| CN106899840B (en) * | 2017-03-01 | 2018-06-05 | 北京大学深圳研究生院 | Panoramic picture mapping method |
| CN106991642A (en) * | 2017-03-23 | 2017-07-28 | 上海国茂数字技术有限公司 | A kind of panoramic video octagonal method of sampling and device |
| CN107147910B (en) * | 2017-05-31 | 2019-11-22 | 上海国茂数字技术有限公司 | A kind of panoramic video hexagon method of sampling and device |
-
2018
- 2018-05-31 CN CN201810544637.5A patent/CN108769680B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN108769680A (en) | 2018-11-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106488233B (en) | A kind of panoramic video diamond shape method of sampling and device | |
| CN106375760B (en) | A kind of panoramic video polygon method of sampling and device | |
| CN106210716B (en) | A kind of panoramic video isodensity method of sampling and device | |
| US6504535B1 (en) | Display techniques for three-dimensional virtual reality | |
| CN114554202B (en) | Image data encoding/decoding method, medium and method of transmitting bit stream | |
| WO2018040463A1 (en) | Data compression and decompression methods for demura table, and mura compensation method | |
| CN110462678B (en) | Image processing apparatus and image processing method | |
| KR20160019387A (en) | Data processing systems | |
| JP2011505631A (en) | Content-aware resizing for images and videos | |
| CN111415300A (en) | Splicing method and system for panoramic image | |
| CN107871304B (en) | Panoramic video frame rectangular mapping method based on spherical surface circular cutting and affine transformation | |
| CN114782648A (en) | Image processing method, image processing device, electronic equipment and storage medium | |
| CN108769680B (en) | Slope-based segmented sampling method and device for panoramic video | |
| US20070065043A1 (en) | Image processing method, image processing device and program | |
| CN114897676A (en) | Unmanned aerial vehicle remote sensing multispectral image splicing method, device and medium | |
| CN111091491A (en) | Equal-distance cylindrical projection panoramic video pixel redistribution method and system | |
| CN113658144B (en) | Method, device, equipment and medium for determining geometric information of pavement diseases | |
| DE60004323T2 (en) | METHOD AND DEVICE FOR STRETCH BLITTING WITH A 3D PIPELINE PROCESSOR | |
| CN114596399A (en) | Image processing method and device and electronic equipment | |
| US6351262B1 (en) | Display techniques for three-dimensional virtual reality | |
| CN106991640B (en) | Panoramic image unfolding method and system | |
| US20210152737A1 (en) | Apparatus and method for stitching together multiple images | |
| CN106991642A (en) | A kind of panoramic video octagonal method of sampling and device | |
| US10026013B2 (en) | Clustering method with a two-stage local binary pattern and an iterative image testing system thereof | |
| WO2019037558A1 (en) | Image processing method and apparatus |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |