CN114549605A - Image optimization method, device and equipment based on point cloud matching and storage medium - Google Patents
Image optimization method, device and equipment based on point cloud matching and storage medium Download PDFInfo
- Publication number
- CN114549605A CN114549605A CN202111666665.2A CN202111666665A CN114549605A CN 114549605 A CN114549605 A CN 114549605A CN 202111666665 A CN202111666665 A CN 202111666665A CN 114549605 A CN114549605 A CN 114549605A
- Authority
- CN
- China
- Prior art keywords
- point cloud
- image
- transformation
- optimized
- cloud data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
- G06T7/35—Determination of transform parameters for the alignment of images, i.e. image registration using statistical methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/77—Determining position or orientation of objects or cameras using statistical methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Abstract
The invention relates to the field of automatic driving, and discloses an image optimization method, device, equipment and storage medium based on point cloud matching. The method comprises the following steps: acquiring point cloud data corresponding to an image to be optimized, and identifying spatial transformation information corresponding to attitude transformation in the image according to the point cloud data; calculating an uncertainty value corresponding to the attitude transformation in the image by adopting a preset probability distribution optimization algorithm according to the spatial transformation information; and optimizing the point cloud data according to the uncertainty value, and generating an optimized image according to the optimized point cloud data. The method and the device improve the accuracy of image optimization based on point cloud matching, reduce the optimization calculation amount and improve the optimization efficiency.
Description
Technical Field
The invention relates to the field of automatic driving, in particular to an image optimization method, device, equipment and storage medium based on point cloud matching.
Background
During route planning of automatic driving, object information in a driving scene needs to be acquired through a sensor such as a radar, and positioning and obstacle tracking of an automatic driving vehicle are facilitated. In the acquisition mode of the driving scene, a sensor generally scans according to a certain path, and a plurality of scanned images are mashup, so that the current driving scene image can be obtained. Due to the fact that intervals exist in the scanning time of each time, the scene coordinates scanned each time are changed continuously, the accumulated error of the finally scanned images is large, and after the driving scene information is estimated to be obtained, image optimization based on point cloud matching is generally needed to be carried out, so that the accuracy of the final driving scene image is improved.
The existing image optimization method based on point cloud matching mainly comprises SLAM (simultaneous localization and mapping) algorithms such as a vector SLAM, a Gmapping algorithm, an Lsd-Slam, an SVO (singular localization and mapping), an Orb-Slam algorithm and the like and various filtering optimization algorithms, wherein the actual position coordinates of each point are adjusted by utilizing the offset of each scanning point so as to construct a driving scene image, and the offset of each point is solved by particularly representing a constraint relation by a connecting edge between each point. However, when the image optimization based on point cloud matching is performed by this type of method, a large amount of calculation is often required while the accuracy is ensured.
Disclosure of Invention
The invention mainly aims to solve the technical problem that when the accuracy is ensured, a large amount of calculation is often needed when the point cloud matching-based image optimization is carried out on the existing driving scene aiming at automatic driving.
The invention provides an image optimization method based on point cloud matching, which comprises the following steps: acquiring point cloud data corresponding to an image to be optimized, and identifying spatial transformation information corresponding to posture transformation in the image according to the point cloud data; calculating an uncertainty value corresponding to the attitude transformation in the image by adopting a preset probability distribution optimization algorithm according to the spatial transformation information; and optimizing the point cloud data according to the uncertainty value, and generating an optimized image according to the optimized point cloud data.
Optionally, in a first implementation manner of the first aspect of the present invention, the calculating, according to the spatial transformation information and by using a preset probability distribution optimization algorithm, an uncertainty value corresponding to the posture transformation in the image includes: calculating lossless scores corresponding to the posture transformations in the images by adopting a Gaussian distribution algorithm according to the spatial transformation information; calculating an n-step vector of the lossless score relative to a pose transformation in the image, wherein n is an integer greater than 1; and determining an uncertainty value corresponding to the attitude transformation in the image according to the n-step degree vector.
Optionally, in a second implementation manner of the first aspect of the present invention, the optimizing the point cloud data according to the uncertainty value includes: calculating a first-order gradient vector of the lossless score relative to the posture transformation in the image, and constructing an optimal posture transformation equation by adopting the first-order gradient vector and the uncertainty value; and solving the optimal attitude transformation equation to obtain optimal attitude transformation, and optimizing the point cloud data according to the optimal attitude transformation.
Optionally, in a third implementation manner of the first aspect of the present invention, the optimizing the point cloud data according to the optimal pose transformation includes: identifying each node to be optimized corresponding to the point cloud data, and identifying point cloud attributes corresponding to each node to be optimized; and optimizing the point cloud attributes in each node to be optimized based on the optimal attitude transformation and a preset image optimization algorithm for point cloud matching.
Optionally, in a fourth implementation manner of the first aspect of the present invention, the acquiring point cloud data corresponding to an image to be optimized includes: dividing an image to be optimized into a plurality of unit structures, and extracting each point cloud structure in the image to be optimized; identifying point cloud structures contained in the unit structures, and combining the point cloud structures contained in the unit structures respectively to obtain a plurality of point cloud sets; and respectively acquiring point cloud semantic information corresponding to each point cloud set, and combining the point cloud semantic information to obtain point cloud data corresponding to the image.
Optionally, in a fifth implementation manner of the first aspect of the present invention, the identifying, according to the point cloud data, spatial transformation information corresponding to pose transformation in the image includes: constructing a conversion relation between corresponding attitude transformation and corresponding point cloud transformation in each unit structure in the image; and calculating the space displacement of the attitude transformation corresponding to each unit structure in the image by adopting the point cloud data according to the conversion relation to obtain the space transformation information corresponding to the image.
The invention provides an image optimization device based on point cloud matching, which comprises: the identification module is used for acquiring point cloud data corresponding to an image to be optimized and identifying spatial transformation information corresponding to posture transformation in the image according to the point cloud data; the calculation module is used for calculating an uncertainty value corresponding to the attitude transformation in the image by adopting a preset probability distribution optimization algorithm according to the spatial transformation information; and the optimization module is used for optimizing the point cloud data according to the uncertainty value and generating an optimized image according to the optimized point cloud data.
Optionally, in a first implementation manner of the second aspect of the present invention, the calculation module includes: the score calculating unit is used for calculating a lossless score corresponding to the posture transformation in the image by adopting a Gaussian distribution algorithm according to the space transformation information; a vector calculation unit for calculating an n-step degree vector of the lossless score relative to the posture change in the image, wherein n is an integer greater than 1; and the determining unit is used for determining an uncertainty value corresponding to the posture transformation in the image according to the n-step degree vector.
Optionally, in a second implementation manner of the second aspect of the present invention, the optimization module is configured to: the function construction unit is used for calculating a first-order gradient vector of the lossless score relative to the posture transformation in the image, and constructing an optimal posture transformation equation by adopting the first-order gradient vector and the uncertainty value; and the optimization unit is used for solving the optimal attitude transformation equation to obtain optimal attitude transformation and optimizing the point cloud data according to the optimal attitude transformation.
Optionally, in a third implementation manner of the second aspect of the present invention, the optimizing unit is further configured to: identifying each node to be optimized corresponding to the point cloud data, and identifying point cloud attributes corresponding to each node to be optimized; and optimizing the point cloud attributes in each node to be optimized based on the optimal attitude transformation and a preset image optimization algorithm for point cloud matching.
Optionally, in a fourth implementation manner of the second aspect of the present invention, the identification module includes: the device comprises a dividing unit, a calculating unit and a processing unit, wherein the dividing unit is used for dividing an image to be optimized into a plurality of unit structures and extracting each point cloud structure in the image to be optimized; the structure combination unit is used for identifying the point cloud structures contained in each unit structure and respectively combining the point cloud structures contained in each unit structure to obtain a plurality of point cloud sets; and the information combination unit is used for respectively acquiring point cloud semantic information corresponding to each point cloud set and combining the point cloud semantic information to obtain point cloud data corresponding to the image.
Optionally, in a fifth implementation manner of the second aspect of the present invention, the identification module further includes: the relation construction unit is used for constructing a conversion relation between corresponding attitude transformation and corresponding point cloud transformation in each unit structure in the image; and the displacement calculation unit is used for calculating the space displacement of the attitude transformation corresponding to each unit structure in the image by adopting the point cloud data according to the conversion relation to obtain the space transformation information corresponding to the image.
The third aspect of the present invention provides an image optimization apparatus based on point cloud matching, including: a memory and at least one processor, the memory having instructions stored therein; the at least one processor invokes the instructions in the memory to cause the point cloud matching-based image optimization apparatus to perform the point cloud matching-based image optimization method described above.
A fourth aspect of the present invention provides a computer-readable storage medium having stored therein instructions, which, when run on a computer, cause the computer to execute the above-mentioned image optimization method based on point cloud matching.
According to the technical scheme, the point cloud data of the image to be optimized is utilized, and the uncertainty of point cloud matching is combined to participate in the image optimization based on the point cloud matching.
Drawings
FIG. 1 is a schematic diagram of a first embodiment of an image optimization method based on point cloud matching according to the present invention;
FIG. 2 is a schematic diagram of a second embodiment of the image optimization method based on point cloud matching according to the present invention;
FIG. 3 is a schematic diagram of a third embodiment of the image optimization method based on point cloud matching according to the present invention;
FIG. 4 is a schematic diagram of an embodiment of an image optimization apparatus based on point cloud matching according to the present invention;
FIG. 5 is a schematic diagram of another embodiment of the image optimization apparatus based on point cloud matching according to the present invention;
fig. 6 is a schematic diagram of an embodiment of the image optimization apparatus based on point cloud matching according to the present invention.
Detailed Description
The embodiment of the invention provides an image optimization method, device, equipment and storage medium based on point cloud matching, which are used for collecting point cloud data corresponding to an image to be optimized and identifying spatial transformation information corresponding to attitude transformation in the image according to the point cloud data; calculating an uncertainty value corresponding to the attitude transformation in the image by adopting a preset probability distribution optimization algorithm according to the spatial transformation information; and optimizing the point cloud data according to the uncertainty value, and generating an optimized image according to the optimized point cloud data. According to the method and the device, the accuracy of image optimization based on point cloud matching is improved, the optimization calculation amount is reduced, and the optimization efficiency is improved.
The terms "first," "second," "third," "fourth," and the like in the description and in the claims, as well as in the drawings, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It will be appreciated that the data so used may be interchanged under appropriate circumstances such that the embodiments described herein may be practiced otherwise than as specifically illustrated or described herein. Furthermore, the terms "comprises," "comprising," or "having," and any variations thereof, are intended to cover non-exclusive inclusions, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
For convenience of understanding, a specific process of the embodiment of the present invention is described below, and referring to fig. 1, a first embodiment of the image optimization method based on point cloud matching in the embodiment of the present invention includes:
101. acquiring point cloud data corresponding to an image to be optimized, and identifying spatial transformation information corresponding to posture transformation in the image according to the point cloud data;
it is to be understood that the executing subject of the present invention may be an image optimization device based on point cloud matching, and may also be a terminal or a server, which is not limited herein. The embodiment of the present invention is described by taking a server as an execution subject. It should be noted that the field to which the present invention is applied may be an automatic driving field, and may also be other image processing fields, such as a virtual reality field, an unmanned aerial vehicle field, an intelligent robot field, and the like, and the present invention is not particularly limited herein, and the following description will take automatic driving as an example.
In this embodiment, an image corresponding to a driving scene is scanned by sensors such as a radar, an infrared sensor, and a camera mounted on an autonomous vehicle, where point cloud data corresponding to an image to be optimized refers to point cloud data acquired from a scanned initial image. In the scanning process, a plurality of images to be optimized with a time sequence relation are obtained, and the optimization process is to predict and match the point cloud data of each image to be optimized into the point cloud data of one image with the same time node.
In this embodiment, after the point cloud data is acquired, X ═ X may be used1，x2，……，xnRepresenting the driving gesture of automatic driving in the image by point cloud data, representing the driving gesture by p, and using a space transformation function T (p, x) to transform the current image to be optimized relative to other images to be optimizedn) And (4) showing. After the spatial transformation information is determined, the displacement position of the point cloud data and the driving attitude transformation are correlated, namely, the result of image optimization can be finally obtained by solving the optimal attitude transformation or the optimal point cloud displacement position. The optimization for the image is essentially an optimal solution to the spatial transformation information.
102. Calculating an uncertainty value corresponding to the attitude transformation in the image by adopting a preset probability distribution optimization algorithm according to the spatial transformation information;
in the present embodiment, the spatial transformation information T (p, x) is obtained by identificationn) Then, two modes are available for obtaining the optimal solution of the spatial transformation information, wherein the first mode is to obtain the optimal displacement position of the point cloud, the second mode is to obtain the optimal attitude transformation, and the parameter optimization is respectively carried out on the point cloud displacement or the attitude transformation corresponding to the two modes, so that the optimal solution of the spatial transformation can be obtained.
Specifically, it is preferable to use the pose transformation as the parameter to be optimized, and then evaluate the parameter to be optimized with the result of maximum likelihood, and the result of optimization is represented by an uncertainty value of the pose transformation, that is, the higher the distribution probability is, the more the position matched by the image to be optimized meets the matching requirement.
103. And optimizing the point cloud data according to the uncertainty value, and generating an optimized image according to the optimized point cloud data.
In this embodiment, the point cloud data includes one or more point cloud semantic information, such as three-dimensional coordinates, color information (RGB), or reflection intensity information of the point cloud, and in the optimization process of the point cloud data, each point cloud semantic information of the point cloud data is optimized, that is, the point cloud is matched to the target image. When the point cloud is matched with the target image according to the uncertainty value, the position of point cloud migration is obtained according to the posture transformation, namely, the displacement position information of the point cloud data is optimized. After the displacement position information in the point cloud data is optimized, the position of the point cloud on the target image can be determined, and the optimized image combined with the target image can be obtained.
In the method, the uncertainty of posture transformation in the image is emphasized, the fine granularity of the image optimization based on the point cloud matching is estimated from point to posture at the same time, the estimation is carried out on a large number of point cloud distributions and is represented by uncertainty values, the obtained uncertainty values are applied to the image generation process to optimize the point cloud data for generating an optimized image, and the image optimization based on the point cloud matching is carried out on the image by utilizing the point cloud data of the image to be optimized and the uncertainty of the posture transformation, so that the calculation amount is greatly reduced, and the accuracy of the generation of the optimized image is ensured.
Referring to fig. 2, a second embodiment of the image optimization method based on point cloud matching according to the embodiment of the present invention includes:
201. dividing an image to be optimized into a plurality of unit structures, and extracting each point cloud structure in the image to be optimized;
202. identifying point cloud structures contained in the unit structures, and combining the point cloud structures contained in the unit structures respectively to obtain a plurality of point cloud sets;
203. respectively acquiring point cloud semantic information corresponding to each point cloud set, combining the point cloud semantic information to obtain point cloud data corresponding to the image, and identifying spatial transformation information corresponding to the attitude transformation in the image according to the point cloud data;
in the embodiment, the image to be optimized is obtainedInitializing (initialization) and dividing into a plurality of unit structures, namely subdividing the posture transformation corresponding to the image to be optimized into the posture transformation corresponding to the plurality of unit structures, simultaneously extracting each point cloud structure in the image, identifying the point cloud structure contained in each unit structure, and determining the relationship between the point cloud data and the posture transformation in each unit structure. Wherein each unit structure corresponds to a point cloud set comprising { x1，x2，……xkAnd f, collecting point clouds corresponding to the k unit structures.
204. Calculating lossless scores corresponding to the posture transformations in the images by adopting a Gaussian distribution algorithm according to the spatial transformation information;
205. calculating an n-step vector of the lossless score relative to a pose transformation in the image, wherein n is an integer greater than 1;
206. determining an uncertainty value corresponding to the posture transformation in the image according to the n-step degree vector;
in this embodiment, the lossless score corresponding to the pose transformation in the image refers to an influence score of each point cloud structure of the image on the corresponding pose transformation, or an influence score of each point cloud structure in the point cloud set on the corresponding pose transformation, or an influence of the pose transformation corresponding to the point cloud set on the overall pose transformation of the image. And subsequently, calculating an uncertainty value corresponding to the posture transformation in the image by calculating a first-order gradient vector and an n-step degree vector.
Preferably, the NDT (non-destructive Testing) algorithm is applied to the above calculation, and the point cloud displacement position is determined by uncertainty values for the point cloud matching of the current image to be optimized, thereby illustrating that the uncertainty values consist in solving the optimal solution of the spatial transformation information, and the maximum likelihood results at k unit structures are optimized for k unit structures:
wherein, P (T (P, x)k) I.e. corresponding transformation to the spatial transformation informationThe probability of (c). It is then equivalent to the optimization of the maximum likelihood result after taking the negative logarithm:
since in this equation the negative log-likelihood value of a normal distribution grows unbounded for points away from the mean. Therefore, the abnormal value in the acquired point cloud data may have a great influence on the overall likelihood result. Therefore, a linear combination of gaussian distribution and uniform distribution is used to replace the likelihood result of the unit structure corresponding to the posture change part, which is specifically shown as follows:
wherein, P0Is the expected ratio of outliers, c1And c2The constant term can be preset by the probability quantity of p (x) in the same unit structure being equal to 1.
And then can be represented by an approximate Gaussian distribution for finally calculating the lossless score of the posture transformation corresponding to the unit structure in the image, specifically:
wherein d is3＝-log(c2)；d1＝-log(c1+c2)-d3；d2＝-2log((-log(c1exp(-1/2)+c2)-d3)/d1)，μkAnd
is xkMean and covariance of the located cell structures.
In addition, after calculating the lossless score corresponding to the image posture transformation, returning to the likelihood result of the whole image space transformation information, the space can be transformedTransformation information T (p, x)k) Can be compared with xkTransform to p, perform the recombination of the initial maximum likelihood functions, and proceed with the computation of the maximum likelihood result:
because only the displacement related to the attitude transformation is concerned at the moment, the probability distribution of s (p) is the joint Gaussian distribution related to the displacement, then p (x) obtained by the derivation is substituted into s (p), and x is also substitutedk≡T(p，xk) That is, the conversion of the point cloud structure depends on the spatial transformation information, x ', corresponding to the pose transformation'kCorresponding to the central point of each unit structure, and setting a Hessian matrix H in the image according to each unit structureijAnd solving to equivalently calculate the distribution uncertainty of the point cloud set corresponding to each unit structure, and simultaneously calculating the lossless scores pi of the unit structures in the ith row and the lossless scores pj of the unit structures in the jth column, wherein the Hessian matrix comprises:
wherein HijNamely the uncertainty value of the unit structure of the ith row and the j column in the image. And here is a 2-step degree vector of the lossless score relative to the pose transform in the image, where the number of n depends on the transform function T ().
207. Calculating a first-order gradient vector of the lossless score relative to the posture transformation in the image, and constructing an optimal posture transformation equation by adopting the first-order gradient vector and the uncertainty value;
208. and solving the optimal attitude transformation equation to obtain optimal attitude transformation, and optimizing the point cloud data according to the optimal attitude transformation.
In this embodiment, when the point cloud data is optimized according to the uncertainty value, a first-order gradient vector is first solved:
then according to a formula H delta p ═ g, the displacement quantity delta p of attitude transformation is calculated by combining the first-order gradient vector and the second-order gradient vector, and the attitude transformation after final optimization can be obtained by delta p + p, and then T (p, x) is passedk) To calculate each point cloud xkDisplacement position information of (2).
Referring to fig. 3, a third embodiment of the image optimization method based on point cloud matching according to the embodiment of the present invention includes:
301. acquiring point cloud data corresponding to an image to be optimized, and constructing a conversion relation between corresponding attitude transformation and corresponding point cloud transformation in each unit structure in the image;
302. calculating the space displacement of the attitude transformation corresponding to each unit structure in the image by adopting the point cloud data according to the conversion relation to obtain the space transformation information corresponding to the image;
in the present embodiment, the conversion relationship between the estimation of the moving position of the point cloud structure in the image and the pose transformation of the image is converted into the spatial transformation information between the image and the pose transformation of each unit structure. Specifically, the conversion relation between the attitude transformation of each unit structure and the corresponding point cloud transformation is constructed, and then the spatial transformation information corresponding to the spatial displacement between the image and each unit structure is calculated according to the relation between the point cloud transformation and the image attitude transformation, namely, the secondary function T (p, x)n) To T (p, X)k)，XkThe point cloud sets corresponding to the k unit structures are used for improving the optimization efficiency.
303. Calculating lossless scores corresponding to the posture transformations in the images by adopting a Gaussian distribution algorithm according to the spatial transformation information;
304. calculating an uncertainty value corresponding to the attitude transformation in the image by adopting a preset probability distribution optimization algorithm according to the spatial transformation information;
305. calculating a first-order gradient vector of the lossless score relative to the posture transformation in the image, and constructing an optimal posture transformation equation by adopting the first-order gradient vector and the uncertainty value;
306. solving the optimal attitude transformation equation to obtain optimal attitude transformation, identifying each node to be optimized corresponding to the point cloud data, and identifying point cloud attributes corresponding to each node to be optimized;
307. and optimizing the point cloud attributes in each node to be optimized based on the optimal attitude transformation and a preset image optimization algorithm for point cloud matching.
In this embodiment, in each node to be optimized of the point cloud data, a corresponding point cloud attribute is optimized, for example, a coordinate value (x, y, z), a color value RGB, a reflection intensity value R, and the like of each point cloud structure in an image. According to xkAdjusting coordinate values (x, y, z), color values RGB, reflection intensity values R and the like, and finally adjusting the coordinate values, the color values RGB, the reflection intensity values R and the like according to each point cloud x after adjustment in the imagenAn optimized image is generated.
The above describes the image optimization method based on point cloud matching in the embodiment of the present invention, and the following describes the image optimization device based on point cloud matching in the embodiment of the present invention, with reference to fig. 4, an embodiment of the image optimization device based on point cloud matching in the embodiment of the present invention includes:
the identification module 401 is configured to acquire point cloud data corresponding to an image to be optimized, and identify spatial transformation information corresponding to posture transformation in the image according to the point cloud data;
a calculating module 402, configured to calculate an uncertainty value corresponding to the posture transformation in the image by using a preset probability distribution optimization algorithm according to the spatial transformation information;
and an optimizing module 403, configured to optimize the point cloud data according to the uncertainty value, and generate an optimized image according to the optimized point cloud data.
In the method, the uncertainty of posture transformation in the image is emphasized, the fine granularity of the image optimization based on the point cloud matching is estimated from point to posture at the same time, the estimation is carried out on a large number of point cloud distributions and is represented by uncertainty values, the obtained uncertainty values are applied to the image generation process to optimize the point cloud data for generating an optimized image, and the image optimization based on the point cloud matching is carried out on the image by utilizing the point cloud data of the image to be optimized and the uncertainty of the posture transformation, so that the calculation amount is greatly reduced, and the accuracy of the generation of the optimized image is ensured.
Referring to fig. 5, another embodiment of the image optimization apparatus based on point cloud matching according to the embodiment of the present invention includes:
the identification module 401 is configured to acquire point cloud data corresponding to an image to be optimized, and identify spatial transformation information corresponding to posture transformation in the image according to the point cloud data;
a calculating module 402, configured to calculate an uncertainty value corresponding to the posture transformation in the image by using a preset probability distribution optimization algorithm according to the spatial transformation information;
and an optimizing module 403, configured to optimize the point cloud data according to the uncertainty value, and generate an optimized image according to the optimized point cloud data.
Specifically, the calculating module 402 includes:
a score calculating unit 4021, configured to calculate a lossless score corresponding to the pose transformation in the image by using a gaussian distribution algorithm according to the spatial transformation information;
a vector calculation unit 4022, configured to calculate an n-step vector of the lossless score with respect to pose transformation in the image, where n is an integer greater than 1;
a determining unit 4023, configured to determine an uncertainty value corresponding to the pose transformation in the image according to the n-step vector.
Specifically, the optimization module 403 includes:
a function construction unit 4031, configured to calculate a first-order gradient vector of the lossless score relative to the pose transformation in the image, and construct an optimal pose transformation equation by using the first-order gradient vector and the uncertainty value;
and the optimizing unit 4032 is used for solving the optimal attitude transformation equation to obtain optimal attitude transformation, and optimizing the point cloud data according to the optimal attitude transformation.
Specifically, the optimization unit 4032 is further configured to:
identifying each node to be optimized corresponding to the point cloud data, and identifying point cloud attributes corresponding to each node to be optimized;
and optimizing the point cloud attributes in each node to be optimized based on the optimal attitude transformation and a preset image optimization algorithm for point cloud matching.
Specifically, the identifying module 401 includes:
the dividing unit 4011 is configured to divide an image to be optimized into a plurality of unit structures, and extract each point cloud structure in the image to be optimized;
a structure combination unit 4012, configured to identify a point cloud structure included in each unit structure, and combine the point cloud structures included in each unit structure, to obtain multiple point cloud sets;
and the information combination unit 4013 is configured to obtain point cloud semantic information corresponding to each point cloud set, and combine the point cloud semantic information to obtain point cloud data corresponding to the image.
Specifically, the identification module 401 further includes:
a relationship construction unit 4014, configured to construct a conversion relationship between a corresponding pose transformation and a corresponding point cloud transformation in each unit structure in the image;
and the displacement calculation unit 4015 is configured to calculate, according to the conversion relationship, a spatial displacement of the posture change corresponding to each unit structure in the image by using the point cloud data, so as to obtain spatial change information corresponding to the image.
Fig. 4 and 5 describe the image optimization device based on point cloud matching in the embodiment of the present invention in detail from the perspective of a modular functional entity, and the image optimization device based on point cloud matching in the embodiment of the present invention is described in detail from the perspective of hardware processing.
Fig. 6 is a schematic structural diagram of an image optimization apparatus based on point cloud matching according to an embodiment of the present invention, where the image optimization apparatus 600 based on point cloud matching may generate relatively large differences due to different configurations or performances, and may include one or more processors (CPUs) 610 (e.g., one or more processors) and a memory 620, one or more storage media 630 (e.g., one or more mass storage devices) storing an application program 633 or data 632. Memory 620 and storage medium 630 may be, among other things, transient or persistent storage. The program stored in the storage medium 630 may include one or more modules (not shown), each of which may include a series of instruction operations for the image optimization apparatus 600 based on point cloud matching. Still further, the processor 610 may be configured to communicate with the storage medium 630 and execute a series of instruction operations in the storage medium 630 on the image optimization apparatus 600 based on point cloud matching.
The point cloud matching-based image optimization apparatus 600 may also include one or more power supplies 640, one or more wired or wireless network interfaces 650, one or more input-output interfaces 660, and/or one or more operating systems 631, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, and the like. Those skilled in the art will appreciate that the configuration of the image optimization apparatus based on point cloud matching shown in fig. 6 does not constitute a limitation of the image optimization apparatus based on point cloud matching, and may include more or less components than those shown, or some components in combination, or a different arrangement of components.
The invention also provides an image optimization device based on point cloud matching, which comprises a memory and a processor, wherein computer readable instructions are stored in the memory, and when the computer readable instructions are executed by the processor, the processor executes the steps of the image optimization method based on point cloud matching in the embodiments.
The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium, and which may also be a volatile computer-readable storage medium, having stored therein instructions, which, when run on a computer, cause the computer to perform the steps of the point cloud matching-based image optimization method.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a read-only memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.
The above-mentioned embodiments are only used for illustrating the technical solutions of the present invention, and not for limiting the same; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (10)
1. An image optimization method based on point cloud matching is characterized by comprising the following steps:
acquiring point cloud data corresponding to an image to be optimized, and identifying spatial transformation information corresponding to posture transformation in the image according to the point cloud data;
calculating an uncertainty value corresponding to the attitude transformation in the image by adopting a preset probability distribution optimization algorithm according to the spatial transformation information;
and optimizing the point cloud data according to the uncertainty value, and generating an optimized image according to the optimized point cloud data.
2. The method for optimizing an image based on point cloud matching according to claim 1, wherein the calculating an uncertainty value corresponding to a pose transformation in the image by using a preset probability distribution optimization algorithm according to the spatial transformation information comprises:
calculating lossless scores corresponding to the posture transformations in the images by adopting a Gaussian distribution algorithm according to the spatial transformation information;
calculating an n-step vector of the lossless score relative to a pose transformation in the image, wherein n is an integer greater than 1;
and determining an uncertainty value corresponding to the attitude transformation in the image according to the n-step degree vector.
3. The method of point cloud matching-based image optimization according to claim 2, wherein said optimizing the point cloud data according to the uncertainty value comprises:
calculating a first-order gradient vector of the lossless score relative to the posture transformation in the image, and constructing an optimal posture transformation equation by adopting the first-order gradient vector and the uncertainty value;
and solving the optimal attitude transformation equation to obtain optimal attitude transformation, and optimizing the point cloud data according to the optimal attitude transformation.
4. The method of image optimization based on point cloud matching according to claim 3, wherein said optimizing the point cloud data according to the optimal pose transformation comprises:
identifying each node to be optimized corresponding to the point cloud data, and identifying point cloud attributes corresponding to each node to be optimized;
and optimizing the point cloud attributes in each node to be optimized based on the optimal attitude transformation and a preset image optimization algorithm for point cloud matching.
5. The method of claim 1, wherein the acquiring point cloud data corresponding to the image to be optimized comprises:
dividing an image to be optimized into a plurality of unit structures, and extracting each point cloud structure in the image to be optimized;
identifying point cloud structures contained in the unit structures, and combining the point cloud structures contained in the unit structures respectively to obtain a plurality of point cloud sets;
and respectively acquiring point cloud semantic information corresponding to each point cloud set, and combining the point cloud semantic information to obtain point cloud data corresponding to the image.
6. The method of claim 5, wherein the identifying spatial transformation information corresponding to pose transformation in the image according to the point cloud data comprises:
constructing a conversion relation between corresponding attitude transformation and corresponding point cloud transformation in each unit structure in the image;
and calculating the space displacement of the attitude transformation corresponding to each unit structure in the image by adopting the point cloud data according to the conversion relation to obtain the space transformation information corresponding to the image.
7. An image optimization device based on point cloud matching, characterized in that the image optimization device based on point cloud matching comprises:
the identification module is used for acquiring point cloud data corresponding to an image to be optimized and identifying spatial transformation information corresponding to posture transformation in the image according to the point cloud data;
the calculation module is used for calculating an uncertainty value corresponding to the attitude transformation in the image by adopting a preset probability distribution optimization algorithm according to the spatial transformation information;
and the optimization module is used for optimizing the point cloud data according to the uncertainty value and generating an optimized image according to the optimized point cloud data.
8. The point cloud matching-based image optimization device of claim 7, wherein the calculation module comprises:
the score calculating unit is used for calculating a lossless score corresponding to the posture transformation in the image by adopting a Gaussian distribution algorithm according to the space transformation information;
a vector calculation unit for calculating an n-step degree vector of the lossless score relative to the posture change in the image, wherein n is an integer greater than 1;
and the determining unit is used for determining an uncertainty value corresponding to the posture transformation in the image according to the n-step degree vector.
9. An image optimization apparatus based on point cloud matching, characterized in that the image optimization apparatus based on point cloud matching comprises: a memory and at least one processor, the memory having instructions stored therein;
the at least one processor invokes the instructions in the memory to cause the point cloud matching-based image optimization apparatus to perform the steps of the point cloud matching-based image optimization method of any one of claims 1-6.
10. A computer-readable storage medium having instructions stored thereon, wherein the instructions, when executed by a processor, implement the steps of the point cloud matching-based image optimization method according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111666665.2A CN114549605B (en) | 2021-12-31 | 2021-12-31 | Image optimization method, device, equipment and storage medium based on point cloud matching |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111666665.2A CN114549605B (en) | 2021-12-31 | 2021-12-31 | Image optimization method, device, equipment and storage medium based on point cloud matching |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114549605A true CN114549605A (en) | 2022-05-27 |
| CN114549605B CN114549605B (en) | 2023-08-04 |
Family
ID=81669746
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111666665.2A Active CN114549605B (en) | 2021-12-31 | 2021-12-31 | Image optimization method, device, equipment and storage medium based on point cloud matching |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114549605B (en) |
Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109949349A (en) * | 2019-01-24 | 2019-06-28 | 北京大学第三医院（北京大学第三临床医学院） | A kind of registration and fusion display methods of multi-modal 3-D image |
| US20190286915A1 (en) * | 2018-03-13 | 2019-09-19 | Honda Motor Co., Ltd. | Robust simultaneous localization and mapping via removal of dynamic traffic participants |
| US20200003869A1 (en) * | 2018-07-02 | 2020-01-02 | Beijing Didi Infinity Technology And Development Co., Ltd. | Vehicle navigation system using pose estimation based on point cloud |
| US20200026925A1 (en) * | 2018-07-23 | 2020-01-23 | Baidu Online Network Technology (Beijing) Co., Ltd. | Method, device and apparatus for generating electronic map, storage medium, and acquisition entity |
| CN111161412A (en) * | 2019-12-06 | 2020-05-15 | 苏州艾吉威机器人有限公司 | Three-dimensional laser mapping method and system |
| CN111383324A (en) * | 2018-12-29 | 2020-07-07 | 广州文远知行科技有限公司 | Point cloud map construction method and device, computer equipment and storage medium |
| CN111539999A (en) * | 2020-04-27 | 2020-08-14 | 深圳南方德尔汽车电子有限公司 | Point cloud registration-based 3D map construction method and device, computer equipment and storage medium |
| CN111812658A (en) * | 2020-07-09 | 2020-10-23 | 北京京东乾石科技有限公司 | Position determination method, device, system and computer readable storage medium |
| CN112837354A (en) * | 2021-02-02 | 2021-05-25 | 北京超星未来科技有限公司 | NDT point cloud registration algorithm and device based on GPU and electronic equipment |
| CN113177974A (en) * | 2021-05-19 | 2021-07-27 | 上海商汤临港智能科技有限公司 | Point cloud registration method and device, electronic equipment and storage medium |
| CN113192142A (en) * | 2021-05-27 | 2021-07-30 | 中国人民解放军国防科技大学 | High-precision map construction method and device in complex environment and computer equipment |
| CN113252051A (en) * | 2020-02-11 | 2021-08-13 | 北京图森智途科技有限公司 | Map construction method and device |
| WO2021218620A1 (en) * | 2020-04-30 | 2021-11-04 | 上海商汤临港智能科技有限公司 | Map building method and apparatus, device and storage medium |
-
2021
- 2021-12-31 CN CN202111666665.2A patent/CN114549605B/en active Active
Patent Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20190286915A1 (en) * | 2018-03-13 | 2019-09-19 | Honda Motor Co., Ltd. | Robust simultaneous localization and mapping via removal of dynamic traffic participants |
| CN111033299A (en) * | 2018-07-02 | 2020-04-17 | 北京嘀嘀无限科技发展有限公司 | Vehicle navigation system based on point cloud utilization pose estimation |
| US20200003869A1 (en) * | 2018-07-02 | 2020-01-02 | Beijing Didi Infinity Technology And Development Co., Ltd. | Vehicle navigation system using pose estimation based on point cloud |
| US20200026925A1 (en) * | 2018-07-23 | 2020-01-23 | Baidu Online Network Technology (Beijing) Co., Ltd. | Method, device and apparatus for generating electronic map, storage medium, and acquisition entity |
| CN111383324A (en) * | 2018-12-29 | 2020-07-07 | 广州文远知行科技有限公司 | Point cloud map construction method and device, computer equipment and storage medium |
| CN109949349A (en) * | 2019-01-24 | 2019-06-28 | 北京大学第三医院（北京大学第三临床医学院） | A kind of registration and fusion display methods of multi-modal 3-D image |
| CN111161412A (en) * | 2019-12-06 | 2020-05-15 | 苏州艾吉威机器人有限公司 | Three-dimensional laser mapping method and system |
| CN113252051A (en) * | 2020-02-11 | 2021-08-13 | 北京图森智途科技有限公司 | Map construction method and device |
| CN111539999A (en) * | 2020-04-27 | 2020-08-14 | 深圳南方德尔汽车电子有限公司 | Point cloud registration-based 3D map construction method and device, computer equipment and storage medium |
| WO2021218620A1 (en) * | 2020-04-30 | 2021-11-04 | 上海商汤临港智能科技有限公司 | Map building method and apparatus, device and storage medium |
| CN111812658A (en) * | 2020-07-09 | 2020-10-23 | 北京京东乾石科技有限公司 | Position determination method, device, system and computer readable storage medium |
| CN112837354A (en) * | 2021-02-02 | 2021-05-25 | 北京超星未来科技有限公司 | NDT point cloud registration algorithm and device based on GPU and electronic equipment |
| CN113177974A (en) * | 2021-05-19 | 2021-07-27 | 上海商汤临港智能科技有限公司 | Point cloud registration method and device, electronic equipment and storage medium |
| CN113192142A (en) * | 2021-05-27 | 2021-07-30 | 中国人民解放军国防科技大学 | High-precision map construction method and device in complex environment and computer equipment |
Non-Patent Citations (3)
| Title |
|---|
| MARTIN MAGNUSSON: "The Three-Dimensional Normal-Distributions Transform --- an Efficient Representation for Registration, Surface Analysis, and Loop Detection", 《HTTPS://WWW.RESEARCHGATE.NET/PUBLICATION/229213868》 * |
| MARTIN MAGNUSSON: "The Three-Dimensional Normal-Distributions Transform --- an Efficient Representation for Registration, Surface Analysis, and Loop Detection", 《HTTPS://WWW.RESEARCHGATE.NET/PUBLICATION/229213868》, 26 May 2014 (2014-05-26), pages 1 - 6 * |
| 赵凯: "基于改进NDT算法的城市场景三维点云配准", 《军事交通学院学报》, vol. 21, no. 03, pages 80 - 84 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114549605B (en) | 2023-08-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP2202672B1 (en) | Information processing apparatus, information processing method, and computer program | |
| CN107179768B (en) | Obstacle identification method and device | |
| US8861785B2 (en) | Information processing device, information processing method and program | |
| Sodhi et al. | In-field segmentation and identification of plant structures using 3D imaging | |
| Eade et al. | Monocular graph SLAM with complexity reduction | |
| US7747106B2 (en) | Method and system for filtering, registering, and matching 2.5D normal maps | |
| US10540813B1 (en) | Three-dimensional point data alignment | |
| Blanco et al. | Subjective local maps for hybrid metric-topological SLAM | |
| CN113420640A (en) | Mangrove hyperspectral image classification method and device, electronic equipment and storage medium | |
| US9816786B2 (en) | Method for automatically generating a three-dimensional reference model as terrain information for an imaging device | |
| CN115063762A (en) | Method, device and equipment for detecting lane line and storage medium | |
| CN112393735A (en) | Positioning method and device, storage medium and electronic device | |
| Román et al. | Evaluating the Robustness of Global Appearance Descriptors in a Visual Localization Task, under Changing Lighting Conditions. | |
| CN114549605B (en) | Image optimization method, device, equipment and storage medium based on point cloud matching | |
| Dreher et al. | Global localization in meshes | |
| CN110992372A (en) | Article grabbing method and device, storage medium and electronic device | |
| CN115375713B (en) | Ground point cloud segmentation method and device and computer readable storage medium | |
| CN114155281B (en) | Unmanned aerial vehicle target tracking automatic initialization method | |
| CN111586299B (en) | Image processing method and related equipment | |
| CN113178000B (en) | Three-dimensional reconstruction method and device, electronic equipment and computer storage medium | |
| Reinoso | Evaluating the Robustness of New Holistic Description Methods in Position Estimation of Mobile Robots | |
| Donnelly | Semi-dense stereo reconstruction from aerial imagery for improved obstacle detection | |
| CN113506338A (en) | 3D point selection method and device in optical motion capture scene and storage medium | |
| Wang et al. | Exploiting vehicle motion information in monocular SLAM | |
| Huber | Visual Navigation of an UAV based on Squared Planar Markers |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |