CN111899152A - Point cloud data compression method and system based on projection and video stitching - Google Patents
Point cloud data compression method and system based on projection and video stitching Download PDFInfo
- Publication number
- CN111899152A CN111899152A CN201910372758.0A CN201910372758A CN111899152A CN 111899152 A CN111899152 A CN 111899152A CN 201910372758 A CN201910372758 A CN 201910372758A CN 111899152 A CN111899152 A CN 111899152A
- Authority
- CN
- China
- Prior art keywords
- projection
- point cloud
- cloud data
- video
- optimal
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 115
- 238000013144 data compression Methods 0.000 title claims abstract description 38
- 238000007906 compression Methods 0.000 claims abstract description 37
- 230000006835 compression Effects 0.000 claims abstract description 37
- 239000013598 vector Substances 0.000 claims description 12
- 238000009826 distribution Methods 0.000 claims description 11
- 238000004458 analytical method Methods 0.000 claims description 9
- 230000008569 process Effects 0.000 claims description 9
- 239000011159 matrix material Substances 0.000 claims description 7
- 101000969688 Homo sapiens Macrophage-expressed gene 1 protein Proteins 0.000 claims description 5
- 102100021285 Macrophage-expressed gene 1 protein Human genes 0.000 claims description 5
- 238000005457 optimization Methods 0.000 claims description 4
- 230000009467 reduction Effects 0.000 claims description 4
- 230000009466 transformation Effects 0.000 claims description 4
- 239000000126 substance Substances 0.000 claims description 3
- 238000006243 chemical reaction Methods 0.000 abstract description 3
- 238000012545 processing Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 4
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000000513 principal component analysis Methods 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000007405 data analysis Methods 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 1
- 238000000354 decomposition reaction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 239000006185 dispersion Substances 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000014759 maintenance of location Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000012805 post-processing Methods 0.000 description 1
- 238000012847 principal component analysis method Methods 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000003860 storage Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/06—Topological mapping of higher dimensional structures onto lower dimensional surfaces
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/85—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/222—Studio circuitry; Studio devices; Studio equipment
- H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
- H04N5/265—Mixing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Image Processing (AREA)
Abstract
The invention provides a point cloud data compression method and system based on projection and video splicing, which comprises the following steps: analyzing the geometric attribute characteristics of the point cloud data to determine a corresponding projection strategy and determine an optimal projection angle; and projecting the projection area of the same projection angle for multiple times to obtain a group of two-dimensional pictures with spatial correlation at the same angle. The invention firstly carries out coordinate conversion on the point cloud data to adapt to a corresponding projection mode, and solves the problem of shielding of dense point cloud in a mode of multiple projections at the same angle. The invention also converts the projection picture splicing into a video form, converts the single space correlation among the pictures into the time correlation among the video frames, reduces the dimension of the point cloud data of the three-dimensional space to a two-dimensional plane through a projection mode, splices the projected pictures into a video format, and further fully utilizes the existing coding tool to carry out compression coding, thereby not only improving the compression performance, but also greatly improving the efficiency by adopting the existing coding tool.
Description
Technical Field
The invention relates to the field of point cloud data compression based on projection and video splicing, in particular to a point cloud data compression method and system based on projection and video splicing.
Background
In recent decades, three-dimensional scanning technology and systems have become mature, the manufacturing cost of 3D scanners is reduced, the precision is higher and higher, the applications are wider and wider, and three-dimensional coordinate information of the surfaces of actual objects can be rapidly and accurately acquired and stored, so that point cloud data can be widely applied to the related fields of image processing in scientific research and industry.
The point cloud data is data information of three-dimensional coordinates of the object after three-dimensional scanning, and information such as RGB (red, green, blue), depth and the like can be recorded. With the improvement of the precision and the speed of a three-dimensional scanning system, the amount of scanned point cloud data reaches the order of magnitude of millions or even more, and at present, the massive point cloud data increases heavy burden for computer storage, processing and transmission.
Point cloud compression algorithms have been studied more systematically. Most of the static point cloud compression methods are realized based on octree space decomposition. And decomposing the three-dimensional space in which the point clouds are positioned by using an octree structure, wherein each node represents a cube of a specific area in the space. The approximated point space coordinates can be calculated from the octree structure and the corresponding node information. Therefore, static point cloud compression can be realized by carrying out serialization coding on the octree structure. While a typical compression scheme for dynamic point clouds is to convert 3D point cloud data into 2D pictures for processing by projection mapping. This arrangement allows the point cloud to be compressed and transmitted by conventional processing means. However, when performing projection, due to the irregularity and density of the point cloud, there usually occurs a problem that projected points are overlapped on a two-dimensional plane to cause data loss, that is, a so-called occlusion problem. How to design reasonable projection and post-processing modes aiming at the problem of cloud occlusion, and retain original cloud point information as much as possible so as to realize efficient compression is a key problem to be solved urgently.
Disclosure of Invention
Aiming at the defects in the prior art, the invention aims to provide a point cloud data compression method and system based on projection and video splicing.
The invention provides a point cloud data compression method based on projection and video splicing, which comprises the following steps:
analyzing the geometric attribute characteristics of the point cloud data to determine a corresponding projection strategy and determine an optimal projection angle;
projecting the projection area of the same projection angle for multiple times to obtain a group of two-dimensional pictures with spatial correlation at the same angle;
and splicing a group of two-dimensional pictures into a video file.
Preferably, the method further comprises the following steps:
and analyzing the point cloud data to obtain geometric and attribute characteristics so as to determine a specific projection strategy.
Preferably, the projection strategy comprises:
the optimal projection axis and the optimal projection surface, and different projection modes are selected and set.
Preferably, the projection means comprises any one or more of:
based on the polyhedral projection method, the spherical projection method, the cylindrical projection method, and the conical projection method.
Preferably, the projection method adopts a cylindrical projection scheme:
analyzing the point cloud data, and judging a projection axis which furthest retains the original distribution characteristics; assume a sample point of xiAnd assuming that the new coordinate system after projective transformation is W ═ W1,w2,w3In which w1,w2,w3For the basis vector of the new coordinate system, x can be obtainediThe projection in the hyperplane is WTxiWith the optimization objective of
s.tWTW＝I
Using the lagrange multiplier method for the above equation, one can obtain:
XXTW＝λW
wherein the content of the first and second substances,
i represents a unit array;
lambda represents a matrix of undetermined eigenvalues;
X＝(x1,x2,…,xn) The method comprises the steps of (1) a point cloud data point set, wherein n is the total number of points contained in the point cloud data, and i is 1, 2, …, n;
for covariance matrix XXTDecomposing the eigenvalues to obtain the largest eigenvectors of the eigenvalues, which are the normal orthogonal basis vectors of the space where the sample is located after dimensionality reduction,
with the corresponding feature vector w1,w2,w3Establishing a new space rectangular coordinate system for the X, Y, Z axes, and determining the optimal projection axis.
Preferably, the method further comprises the following steps:
analyzing the point cloud data, and judging a projection axis which furthest retains the original distribution characteristics to obtain a determined optimal projection axis;
selecting a projection radius, and projecting and expanding points on the original point cloud data to a plane according to the principle of projection at the same angle;
determining an optimal projection area;
each point in the optimal projection area can be projected only once;
repeating the above process can obtain a group of two-dimensional pictures projected at the same angle, and splicing the two-dimensional pictures into a video sequence.
Preferably, the method further comprises the following steps:
determining the number of multiple projections by:
and setting a projection acquisition threshold value of the acquired projection points accounting for the total points of the original point cloud data, and accumulating to acquire a plurality of two-dimensional pictures without continuing projection when the projected points reach the projection acquisition threshold value after multiple projections.
Preferably, the method further comprises the following steps:
after the projection angle is determined, only one point is preferentially selected from coincident points of the same pixel in the two-dimensional image for each projection among multiple projections according to the geometric attribute characteristics for reservation, so that each point can be projected only once, and the optimal point is selected from a series of points with the shielding relation according to the geometric and attribute characteristics for projection.
Preferably, the method further comprises the following steps:
and determining an optimal projection area according to the distance and the geometric attributes in a series of points with the occlusion relation.
Preferably, the method further comprises the following steps:
and carrying out compression coding on the spliced video file.
Preferably, the method further comprises the following steps:
the video splicing is in any one or more of the following formats: YUV, MP4, RMVB, MKV and AVI formats, and the compression mode adopts HEVC/MPEG1/MPEG2/MPEG4/H264 mode.
The invention provides a point cloud data compression system based on projection and video splicing, which comprises:
an analysis module: analyzing the geometric attribute characteristics of the point cloud data to determine a corresponding projection strategy and determine an optimal projection angle;
a projection module: projecting the projection area of the same projection angle for multiple times to obtain a group of two-dimensional pictures with spatial correlation at the same angle;
splicing modules: and splicing a group of two-dimensional pictures into a video file.
Preferably, the method further comprises the following steps:
the analysis module is used for analyzing the point cloud data to obtain geometric and attribute characteristics and determining a specific projection strategy.
Preferably, the method further comprises the following steps:
the projection strategy comprises: the optimal projection axis and the optimal projection surface, and different projection modes are selected and set.
Preferably, the method further comprises the following steps:
the projection mode comprises any one or more of the following modes: based on a polyhedral projection method, a spherical projection method, a cylindrical projection method, or a conical projection method.
Preferably, the method further comprises the following steps:
analyzing the point cloud data, and judging a projection axis which furthest retains the original distribution characteristics to obtain a determined optimal projection axis;
selecting a projection radius, and projecting and expanding points on the original point cloud data to a plane according to the principle of projection at the same angle;
determining an optimal projection area;
each point in the optimal projection area can be projected only once;
repeating the above process can obtain a group of two-dimensional pictures projected at the same angle, and splicing the two-dimensional pictures into a video sequence.
Preferably, the method further comprises the following steps:
determining the number of multiple projections by:
and setting a projection acquisition threshold value of the acquired projection points accounting for the total points of the original point cloud data, and accumulating to acquire a plurality of two-dimensional pictures without continuing projection when the projected points reach the projection acquisition threshold value after multiple projections.
Preferably, the method further comprises the following steps:
after the projection angle is determined, only one point is preferentially selected from coincident points of the same pixel in the two-dimensional image for each projection among multiple projections according to the geometric attribute characteristics for reservation, so that each point can be projected only once, and the optimal point is selected from a series of points with the shielding relation according to the geometric and attribute characteristics for projection.
Preferably, the method further comprises the following steps:
and determining an optimal projection area according to the distance and the geometric attributes in a series of points with the occlusion relation.
Preferably, the method further comprises the following steps:
and the compression coding module is used for carrying out compression coding on the spliced video file.
Compared with the prior art, the invention has the following beneficial effects:
the invention firstly carries out coordinate conversion on the point cloud data to adapt to a corresponding projection mode, and solves the problem of shielding of dense point cloud in a mode of multiple projections at the same angle. In addition, projection pictures are spliced and converted into a video form, single space correlation among the pictures is converted into time correlation among video frames, point cloud data of a three-dimensional space is reduced to a two-dimensional plane in a projection mode, the projected pictures are spliced into a video format, then the existing coding tools are fully utilized for compression coding, not only is the compression performance improved, but also the efficiency can be greatly improved by adopting the existing coding tools.
Drawings
Other features, objects and advantages of the invention will become more apparent upon reading of the detailed description of non-limiting embodiments with reference to the following drawings:
FIG. 1 is a schematic diagram of original point cloud data under cylindrical projection and a cylindrical projection plane thereof according to an embodiment of the present invention;
FIG. 2 is a schematic spatial diagram of a point cloud projected onto a cylindrical surface under cylindrical projection according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of a two-dimensional image of a cylindrical projection surface after expansion under cylindrical projection according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of a group of pictures, i.e., a group of video frames after stitching, obtained by multiple projections at the same angle in the embodiment of the present invention;
fig. 5 is a schematic flowchart of a point cloud data compression method based on projection and video stitching according to an embodiment of the present invention.
Detailed Description
The present invention will be described in detail with reference to specific examples. The following examples will assist those skilled in the art in further understanding the invention, but are not intended to limit the invention in any way. It should be noted that it would be obvious to those skilled in the art that various changes and modifications can be made without departing from the spirit of the invention. All falling within the scope of the present invention.
The invention provides a point cloud data compression method based on projection and video splicing, which comprises the following steps:
analyzing the geometric attribute characteristics of the point cloud data to determine a corresponding projection strategy and determine an optimal projection angle;
projecting the projection area of the same projection angle for multiple times to obtain a group of two-dimensional pictures with spatial correlation at the same angle;
and splicing a group of two-dimensional pictures into a video file.
Preferably, the method further comprises the following steps:
and analyzing the point cloud data to obtain geometric and attribute characteristics so as to determine a specific projection strategy.
Preferably, the projection strategy comprises:
the optimal projection axis and the optimal projection surface, and different projection modes are selected and set.
Preferably, the projection means comprises any one or more of:
based on the polyhedral projection method, the spherical projection method, the cylindrical projection method, and the conical projection method.
Preferably, the projection method adopts a cylindrical projection scheme:
analyzing the point cloud data, and judging a projection axis which furthest retains the original distribution characteristics; assume a sample point of xiAnd assuming that the new coordinate system after projective transformation is W ═ W1,w2,w3In which w1,w2,w3For the basis vector of the new coordinate system, x can be obtainediThe projection in the hyperplane is WTxiWith the optimization objective of
s.tWTW＝I
Using the lagrange multiplier method for the above equation, one can obtain:
XXTW＝λW
wherein the content of the first and second substances,
i represents a unit array;
lambda represents a matrix of undetermined eigenvalues;
X＝(x1,x2,…,xn) The method comprises the steps of (1) a point cloud data point set, wherein n is the total number of points contained in the point cloud data, and i is 1, 2, …, n;
coordinating partyDifference matrix XXTDecomposing the eigenvalues to obtain the largest eigenvectors of the eigenvalues, which are the normal orthogonal basis vectors of the space where the sample is located after dimensionality reduction,
with the corresponding feature vector w1,w2,w3Establishing a new space rectangular coordinate system for the X, Y, Z axes, and determining the optimal projection axis.
Preferably, the method further comprises the following steps:
analyzing the point cloud data, and judging a projection axis which furthest retains the original distribution characteristics to obtain a determined optimal projection axis;
selecting a projection radius, and projecting and expanding points on the original point cloud data to a plane according to the principle of projection at the same angle;
determining an optimal projection area;
each point in the optimal projection area can be projected only once;
repeating the above process can obtain a group of two-dimensional pictures projected at the same angle, and splicing the two-dimensional pictures into a video sequence.
Preferably, the method further comprises the following steps:
determining the number of multiple projections by:
and setting a projection acquisition threshold value of the acquired projection points accounting for the total points of the original point cloud data, and accumulating to acquire a plurality of two-dimensional pictures without continuing projection when the projected points reach the projection acquisition threshold value after multiple projections.
Preferably, the method further comprises the following steps:
after the projection angle is determined, only one point is preferentially selected from coincident points of the same pixel in the two-dimensional image for each projection among multiple projections according to the geometric attribute characteristics for reservation, so that each point can be projected only once, and the optimal point is selected from a series of points with the shielding relation according to the geometric and attribute characteristics for projection.
Preferably, the method further comprises the following steps:
and determining an optimal projection area according to the distance and the geometric attributes in a series of points with the occlusion relation.
Preferably, the method further comprises the following steps:
and carrying out compression coding on the spliced video file.
Preferably, the method further comprises the following steps:
the video splicing is in any one or more of the following formats: YUV, MP4, RMVB, MKV and AVI formats, and the compression mode adopts HEVC/MPEG1/MPEG2/MPEG4/H264 mode.
The invention provides a point cloud data compression system based on projection and video splicing, which comprises:
an analysis module: analyzing the geometric attribute characteristics of the point cloud data to determine a corresponding projection strategy and determine an optimal projection angle;
a projection module: projecting the projection area of the same projection angle for multiple times to obtain a group of two-dimensional pictures with spatial correlation at the same angle;
splicing modules: and splicing a group of two-dimensional pictures into a video file.
Preferably, the method further comprises the following steps:
the analysis module is used for analyzing the point cloud data to obtain geometric and attribute characteristics and determining a specific projection strategy.
Preferably, the method further comprises the following steps:
the projection strategy comprises: the optimal projection axis and the optimal projection surface, and different projection modes are selected and set.
Preferably, the method further comprises the following steps:
the projection mode comprises any one or more of the following modes: based on a polyhedral projection method, a spherical projection method, a cylindrical projection method, or a conical projection method.
Preferably, the method further comprises the following steps:
analyzing the point cloud data, and judging a projection axis which furthest retains the original distribution characteristics to obtain a determined optimal projection axis;
selecting a projection radius, and projecting and expanding points on the original point cloud data to a plane according to the principle of projection at the same angle;
determining an optimal projection area;
each point in the optimal projection area can be projected only once;
repeating the above process can obtain a group of two-dimensional pictures projected at the same angle, and splicing the two-dimensional pictures into a video sequence.
Preferably, the method further comprises the following steps:
determining the number of multiple projections by:
and setting a projection acquisition threshold value of the acquired projection points accounting for the total points of the original point cloud data, and accumulating to acquire a plurality of two-dimensional pictures without continuing projection when the projected points reach the projection acquisition threshold value after multiple projections.
Preferably, the method further comprises the following steps:
after the projection angle is determined, only one point is preferentially selected from coincident points of the same pixel in the two-dimensional image for each projection among multiple projections according to the geometric attribute characteristics for reservation, so that each point can be projected only once, and the optimal point is selected from a series of points with the shielding relation according to the geometric and attribute characteristics for projection.
Preferably, the method further comprises the following steps:
and determining an optimal projection area according to the distance and the geometric attributes in a series of points with the occlusion relation.
Preferably, the method further comprises the following steps:
and the compression coding module is used for carrying out compression coding on the spliced video file.
The present invention will be described more specifically below with reference to preferred examples.
Preferred example 1:
the mainstream processing method of the point cloud is to convert the point cloud into a two-dimensional plane for processing, but the precision, irregularity and density of the point cloud collection cause the problem of point superposition, namely the problem of occlusion during projection. The common processing mode is multiple projections at different angles, but a large number of black dispersion points and even stripes appear on a two-dimensional picture obtained after multiple projections, which is extremely not beneficial to compression coding. People are further upgrading the consumption of media nowadays, and point clouds are indispensable as an important representation form of immersive media. In the fields of automatic driving, security monitoring and the like, efficient point cloud compression is also needed to optimize a solution. In order to realize higher performance and efficiency of point cloud coding based on two-dimensional plane compression, the invention provides a point cloud data compression method based on projection and video splicing, wherein point clouds are subjected to multiple projections at the same angle and are spliced to form a video file, and the time-space domain correlation is fully utilized to realize high-efficiency compression coding performance.
The method of projecting a projection line through a point or other object onto a selected projection surface and obtaining a pattern on that surface is called projection. Including but not limited to projections based on polyhedrons, spheres, cylinders, cones, etc.
In particular, when a cylindrical projection scheme is applied, the invention is implemented as follows:
firstly, analyzing original point cloud data, and judging a projection axis which furthest retains original distribution characteristics by adopting a principal component analysis method; assume a sample point of xiAnd assuming that the new coordinate system after projective transformation is W ═ W1,w2,w3In which w1,w2,w3For the basis vector of the new coordinate system, x can be obtainediThe projection in the hyperplane is WTxi. The optimization objective is
s.tWTW＝I
Using Lagrange multiplier method for the above equation
XXTW＝λW
Then, for the covariance matrix XXTAnd (4) decomposing the eigenvalues, wherein the obtained maximum eigenvectors of a plurality of (the first 3 adopted in the embodiment) eigenvalues are the orthonormal basis vectors of the space where the sample is located after dimensionality reduction. With the corresponding feature vector w1,w2,w3And establishing a new space rectangular coordinate system for the X, Y, Z axis, thereby realizing the analysis of the shape of the point cloud data and determining a projection axis, a projection plane and a most appropriate projection mode.
Secondly, selecting a projection radius, and projecting points on the original point cloud shown in the attached figure 1 onto a space cylinder according to the principle of equal-angle projection. The projected effect is shown in fig. 2, and the spatial cylinder is unfolded to a two-dimensional plane, as shown in fig. 3. Generally, the point cloud data has a serious occlusion in a region with a large density, so that when the point cloud data is projected to a specific direction, a plurality of points correspond to the same pixel position on a projection plane. In order to solve the problem, each point is specified to be projected only once, and the optimal point is selected from a series of points with occlusion relation according to geometric and attribute characteristics for projection. In addition, a percentage threshold value of the number of projection points in the total number of point cloud data (i.e. a projection acquisition threshold value) is set, a group of pictures projected at the same angle can be obtained by repeating the above process (as shown in fig. 4), and when the number of projection points reaches the threshold value after multiple projections, the process is not continued.
And splicing the projected two-dimensional pictures as a group of video frames into a yuv video sequence, and then performing coding compression by using an HEVC coding mode. The spliced video sequence can be in MP4/RMVB/MKV/AVI format, and the compression modes such as MPEG1/MPEG2/MPEG4/H264 are also optional. In the above implementation in the cylindrical projection manner, the geometric features of the point cloud data are more emphasized. It is noted that other attribute features, such as color, are also important references in the projection process and are included within the scope of the present invention.
As shown in fig. 5, a detailed flow chart of the method is briefly described as follows: after the original point cloud data is obtained, the geometric and attribute characteristics of the original point cloud data are analyzed to confirm the optimal projection mode and projection plane/axis. In order to solve the shielding problem, after the projection angle is determined, only one point is preferentially selected from coincident points of the same pixel in the two-dimensional image for retention in each projection according to geometric and attribute characteristics, and a plurality of two-dimensional images at the same angle are obtained through multiple projections. When the percentage of the projection points reaches a set threshold (i.e., a projection acquisition threshold), the projection is not continued, for example, if the projection acquisition threshold in this embodiment in fig. 5 is 99%, which means that the acquired projection points account for 99% of the total points of the original point cloud data, then the projection is not continued for 6 times in fig. 5, two-dimensional pictures of 6 frames (e.g., Frame1 to Frame6 in fig. 5) are obtained, the obtained group of two-dimensional pictures are spliced into a video sequence, and a video compression tool is used to perform coding compression. The above description is only an embodiment, and the description of the projection acquisition threshold and the number of projections is not a number limitation of the present invention.
Omitted from the drawings in the specification, the embodiment also provides a point cloud data compression system based on projection and video stitching, including: the device comprises an analysis module, a projection module, a splicing module and a compression coding module.
And the analysis module is used for analyzing the geometric attribute characteristics of the point cloud data to determine a corresponding projection strategy and determine an optimal projection angle.
And the projection module is used for projecting the projection area with the same projection angle for multiple times to obtain a group of two-dimensional pictures with spatial correlation at the same angle.
And the splicing module is used for splicing the group of two-dimensional pictures into a video file.
Optionally, the projection and video stitching based point cloud data compression system may further include: and the compression coding module is used for carrying out compression coding on the spliced video file.
In summary, the point cloud data compression method and system based on projection and video stitching can analyze according to the geometrical attribute characteristics of the point cloud data, optimize the projection axis and the projection plane, and select and set different projection modes (such as a plurality of modes including a polyhedron, a spherical surface, a cylinder, a conical surface and the like), thereby improving the projection efficiency. Particularly, after the projection angle is determined for the occlusion problem, only one point is preferentially selected from coincident points of the same pixel in the two-dimensional image for each projection according to geometric and attribute characteristics to be reserved, and a plurality of two-dimensional images at the same angle are obtained through multiple projections. The method comprises the steps of setting a percentage threshold value of projection points in the total points of point cloud data, splicing a plurality of projected two-dimensional pictures into a video format when the number of projected points reaches the threshold value after multiple times of projection, converting the spatial correlation among the pictures into the time correlation among video frames, and realizing high-efficiency compression.
In the invention, the method for analyzing the point cloud data, the projection method, the splicing mode and the compression tool are various, and preferably, the method can be realized by taking the following group of alternatives as an example:
the data analysis method comprises the following steps: principal Component Analysis (PCA) can be used to determine the optimal projection axis and the optimal projection plane. PCA is characterized in that a certain hyperplane is searched, so that all sample points are spread in the hyperplane as much as possible, namely, the maximum variance of a discrete point set is calculated, and the original distribution characteristics of the discrete point set are kept to the maximum extent.
The projection method comprises the following steps: projection methods from a three-dimensional space to a two-dimensional plane are many and have advantages and disadvantages, and projection methods based on polyhedrons, spherical surfaces, cylinders, conical surfaces and the like are taken as examples.
Splicing mode and compression tool: in order to improve the intra-frame/inter-frame coding efficiency to the maximum extent, the video is proposed to be spliced into a YUV format, and an HEVC coding compression tool is adopted. The modes of MPEG1/MPEG2/MPEG4/H264 and the like are also optional. Compression may be performed using conventional video coding tools.
It should be noted that, in the invention, the projection and splicing scheme is described only by taking the above method as an example, and is not limited to the above data analysis method, projection method, video splicing and encoding method.
In the point cloud data compression system based on projection and video stitching provided in this embodiment, each functional module corresponds to the point cloud data compression method based on projection and video stitching in the above embodiment, so that the structure and technical elements in the device can be formed by corresponding conversion of the generation method, and description is omitted here and will not be repeated.
Although the present invention has been described with reference to the preferred embodiments, it is not intended to limit the present invention, and those skilled in the art can make variations and modifications of the present invention without departing from the spirit and scope of the present invention by using the methods and technical contents disclosed above.
In the description of the present application, it is to be understood that the terms "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", and the like indicate orientations or positional relationships based on those shown in the drawings, and are only for convenience in describing the present application and simplifying the description, but do not indicate or imply that the referred device or element must have a specific orientation, be constructed in a specific orientation, and be operated, and thus, should not be construed as limiting the present application.
Those skilled in the art will appreciate that, in addition to implementing the systems, apparatus, and various modules thereof provided by the present invention in purely computer readable program code, the same procedures can be implemented entirely by logically programming method steps such that the systems, apparatus, and various modules thereof are provided in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers, embedded microcontrollers and the like. Therefore, the system, the device and the modules thereof provided by the present invention can be considered as a hardware component, and the modules included in the system, the device and the modules thereof for implementing various programs can also be considered as structures in the hardware component; modules for performing various functions may also be considered to be both software programs for performing the methods and structures within hardware components.
The foregoing description of specific embodiments of the present invention has been presented. It is to be understood that the present invention is not limited to the specific embodiments described above, and that various changes or modifications may be made by one skilled in the art within the scope of the appended claims without departing from the spirit of the invention. The embodiments and features of the embodiments of the present application may be combined with each other arbitrarily without conflict.
Claims (20)
1. A point cloud data compression method based on projection and video splicing is characterized by comprising the following steps:
analyzing the geometric attribute characteristics of the point cloud data to determine a corresponding projection strategy and determine an optimal projection angle;
projecting the projection area of the same projection angle for multiple times to obtain a group of two-dimensional pictures with spatial correlation at the same angle;
and splicing a group of two-dimensional pictures into a video file.
2. The projection and video stitching-based point cloud data compression method as claimed in claim 1, further comprising:
and analyzing the point cloud data to obtain geometric and attribute characteristics so as to determine a specific projection strategy.
3. The projection and video stitching-based point cloud data compression method as claimed in claim 1, wherein the projection strategy comprises:
the optimal projection axis and the optimal projection surface, and different projection modes are selected and set.
4. The projection and video stitching-based point cloud data compression method according to claim 3, wherein the projection mode comprises any one or more of the following modes:
based on the polyhedral projection method, the spherical projection method, the cylindrical projection method, and the conical projection method.
5. The projection and video stitching-based point cloud data compression method according to claim 4, wherein the point cloud data compression method is characterized in that a cylindrical projection scheme is adopted:
analyzing the point cloud data, and judging a projection axis which furthest retains the original distribution characteristics; assume a sample point of xiAnd assuming that the new coordinate system after projective transformation is W ═ W1,w2,w3In which w1,w2,w3For the basis vector of the new coordinate system, x can be obtainediThe projection in the hyperplane is WTxiWith the optimization objective of
s.tWTW＝I
Using the lagrange multiplier method for the above equation, one can obtain:
XXTW＝λW
wherein the content of the first and second substances,
i represents a unit array;
lambda represents a matrix of undetermined eigenvalues;
X＝(x1,x2,…,xn) The method comprises the steps of (1) a point cloud data point set, wherein n is the total number of points contained in the point cloud data, and i is 1, 2, …, n;
for covariance matrix XXTDecomposing the eigenvalues to obtain the largest eigenvectors of the eigenvalues, which are the normal orthogonal basis vectors of the space where the sample is located after dimensionality reduction,
with the corresponding feature vector w1,w2,w3Establishing a new space rectangular coordinate system for the X, Y, Z axes, and determining the optimal projection axis.
6. The projection and video stitching-based point cloud data compression method as claimed in claim 1, further comprising:
analyzing the point cloud data, and judging a projection axis which furthest retains the original distribution characteristics to obtain a determined optimal projection axis;
selecting a projection radius, and projecting and expanding points on the original point cloud data to a plane according to the principle of projection at the same angle;
determining an optimal projection area;
each point in the optimal projection area can be projected only once;
repeating the above process can obtain a group of two-dimensional pictures projected at the same angle, and splicing the two-dimensional pictures into a video sequence.
7. The projection and video stitching-based point cloud data compression method as claimed in claim 1, further comprising:
determining the number of multiple projections by:
and setting a projection acquisition threshold value of the acquired projection points accounting for the total points of the original point cloud data, and accumulating to acquire a plurality of two-dimensional pictures without continuing projection when the projected points reach the projection acquisition threshold value after multiple projections.
8. The projection and video stitching-based point cloud data compression method as claimed in claim 6, further comprising:
after the projection angle is determined, only one point is preferentially selected from coincident points of the same pixel in the two-dimensional image for each projection among multiple projections according to the geometric attribute characteristics for reservation, so that each point can be projected only once, and the optimal point is selected from a series of points with the shielding relation according to the geometric and attribute characteristics for projection.
9. The projection and video stitching-based point cloud data compression method as claimed in claim 1, further comprising:
and determining an optimal projection area according to the distance and the geometric attributes in a series of points with the occlusion relation.
10. The projection and video stitching-based point cloud data compression method as claimed in claim 1, further comprising:
and carrying out compression coding on the spliced video file.
11. The projection and video stitching-based point cloud data compression method as claimed in claim 7, further comprising:
the video splicing is in any one or more of the following formats: YUV, MP4, RMVB, MKV and AVI formats, and the compression mode adopts HEVC/MPEG1/MPEG2/MPEG4/H264 mode.
12. A point cloud data compression system based on projection and video stitching is characterized by comprising:
an analysis module: analyzing the geometric attribute characteristics of the point cloud data to determine a corresponding projection strategy and determine an optimal projection angle;
a projection module: projecting the projection area of the same projection angle for multiple times to obtain a group of two-dimensional pictures with spatial correlation at the same angle;
splicing modules: and splicing a group of two-dimensional pictures into a video file.
13. The projection and video stitching-based point cloud data compression system of claim 12, further comprising:
the analysis module is used for analyzing the point cloud data to obtain geometric and attribute characteristics and determining a specific projection strategy.
14. The projection and video stitching-based point cloud data compression system of claim 12, further comprising:
the projection strategy comprises: the optimal projection axis and the optimal projection surface, and different projection modes are selected and set.
15. The projection and video stitching-based point cloud data compression system of claim 14, further comprising:
the projection mode comprises any one or more of the following modes: based on a polyhedral projection method, a spherical projection method, a cylindrical projection method, or a conical projection method.
16. The projection and video stitching-based point cloud data compression system of claim 12, further comprising:
analyzing the point cloud data, and judging a projection axis which furthest retains the original distribution characteristics to obtain a determined optimal projection axis;
selecting a projection radius, and projecting and expanding points on the original point cloud data to a plane according to the principle of projection at the same angle;
determining an optimal projection area;
each point in the optimal projection area can be projected only once;
repeating the above process can obtain a group of two-dimensional pictures projected at the same angle, and splicing the two-dimensional pictures into a video sequence.
17. The projection and video stitching-based point cloud data compression system of claim 12, further comprising:
determining the number of multiple projections by:
and setting a projection acquisition threshold value of the acquired projection points accounting for the total points of the original point cloud data, and accumulating to acquire a plurality of two-dimensional pictures without continuing projection when the projected points reach the projection acquisition threshold value after multiple projections.
18. The projection and video stitching-based point cloud data compression system of claim 17, further comprising:
after the projection angle is determined, only one point is preferentially selected from coincident points of the same pixel in the two-dimensional image for each projection among multiple projections according to the geometric attribute characteristics for reservation, so that each point can be projected only once, and the optimal point is selected from a series of points with the shielding relation according to the geometric and attribute characteristics for projection.
19. The projection and video stitching-based point cloud data compression system of claim 12, further comprising:
and determining an optimal projection area according to the distance and the geometric attributes in a series of points with the occlusion relation.
20. The projection and video stitching-based point cloud data compression system of claim 12, further comprising:
and the compression coding module is used for carrying out compression coding on the spliced video file.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910372758.0A CN111899152A (en) | 2019-05-06 | 2019-05-06 | Point cloud data compression method and system based on projection and video stitching |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910372758.0A CN111899152A (en) | 2019-05-06 | 2019-05-06 | Point cloud data compression method and system based on projection and video stitching |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111899152A true CN111899152A (en) | 2020-11-06 |
Family
ID=73169494
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910372758.0A Pending CN111899152A (en) | 2019-05-06 | 2019-05-06 | Point cloud data compression method and system based on projection and video stitching |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111899152A (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20170336705A1 (en) * | 2016-05-19 | 2017-11-23 | Avago Technologies General Ip (Singapore) Pte. Ltd. | 360 degree video capture and playback |
| CN107798696A (en) * | 2017-10-19 | 2018-03-13 | 北京联合大学 | A kind of three-dimensional point cloud method for registering based on guarantor office PCA |
| CN108256529A (en) * | 2017-11-29 | 2018-07-06 | 深圳慎始科技有限公司 | Global point cloud based on Dian Yun projected outlines signature and distribution matrix describes method |
| CN109214982A (en) * | 2018-09-11 | 2019-01-15 | 大连理工大学 | A kind of three-dimensional point cloud imaging method based on bicylindrical projection model |
-
2019
- 2019-05-06 CN CN201910372758.0A patent/CN111899152A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20170336705A1 (en) * | 2016-05-19 | 2017-11-23 | Avago Technologies General Ip (Singapore) Pte. Ltd. | 360 degree video capture and playback |
| CN107798696A (en) * | 2017-10-19 | 2018-03-13 | 北京联合大学 | A kind of three-dimensional point cloud method for registering based on guarantor office PCA |
| CN108256529A (en) * | 2017-11-29 | 2018-07-06 | 深圳慎始科技有限公司 | Global point cloud based on Dian Yun projected outlines signature and distribution matrix describes method |
| CN109214982A (en) * | 2018-09-11 | 2019-01-15 | 大连理工大学 | A kind of three-dimensional point cloud imaging method based on bicylindrical projection model |
Non-Patent Citations (3)
| Title |
|---|
| EUEE S. JANG ET AL: "Video-Based Point-Cloud-Compression Standard in MPEG: From Evidence Collection to Committee Draft" * |
| LANYI HE, ET: "Best-effort projection based attribute compression for 3D point cloud" * |
| 马润杰;李宝胜;阚洪涛;: "基于流的LDDoS攻击特征研究" * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10547812B2 (en) | Video capture device and method | |
| TWI691197B (en) | Preprocessor for full parallax light field compression | |
| US8233661B2 (en) | Object tracking apparatus and object tracking method | |
| DE102020100684A1 (en) | IDENTIFICATION OF GRAPHIC REFERENCE MARKERS | |
| KR20190128686A (en) | Method and apparatus, equipment, and storage medium for determining the pose of an object in an image | |
| CN103345736A (en) | Virtual viewpoint rendering method | |
| US20120001902A1 (en) | Apparatus and method for bidirectionally inpainting occlusion area based on predicted volume | |
| US10535147B2 (en) | Electronic apparatus and method for processing image thereof | |
| CN108009985B (en) | Video splicing method based on graph cut | |
| US20230394833A1 (en) | Method, system and computer readable media for object detection coverage estimation | |
| Boliek et al. | Next generation image compression and manipulation using CREW | |
| Svoboda et al. | Matching in catadioptric images with appropriate windows, and outliers removal | |
| TWI716874B (en) | Image processing apparatus, image processing method, and image processing program | |
| CN111899162A (en) | Point cloud data processing method and system based on segmentation | |
| US20130128992A1 (en) | Methods and Apparatus for Preparation of Casual Stereoscopic Video | |
| CN111899152A (en) | Point cloud data compression method and system based on projection and video stitching | |
| CN112002007A (en) | Model obtaining method and device based on air-ground image, equipment and storage medium | |
| CN109215054A (en) | Face tracking method and system | |
| EP4202858A1 (en) | Method and device for cargo counting, computer equipment, and storage medium | |
| KR101482419B1 (en) | Method and apparatus for generating motion data | |
| Wang et al. | Accelerating image-domain-warping virtual view synthesis on GPGPU | |
| CN107451540B (en) | Compressible 3D identification method | |
| Jin et al. | Meshreduce: Scalable and bandwidth efficient 3d scene capture | |
| US20210217229A1 (en) | Method and apparatus for plenoptic point clouds generation | |
| Yu et al. | NeRF-Enhanced Outpainting for Faithful Field-of-View Extrapolation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |