CN110287873B - Non-cooperative target pose measurement method and system based on deep neural network and terminal equipment - Google Patents

Non-cooperative target pose measurement method and system based on deep neural network and terminal equipment Download PDF

Info

Publication number
CN110287873B
CN110287873B CN201910555500.4A CN201910555500A CN110287873B CN 110287873 B CN110287873 B CN 110287873B CN 201910555500 A CN201910555500 A CN 201910555500A CN 110287873 B CN110287873 B CN 110287873B
Authority
CN
China
Prior art keywords
point cloud
point
feature
pose
neural network
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201910555500.4A
Other languages
Chinese (zh)
Other versions
CN110287873A (en
Inventor
刘厚德
刘兵
高学海
王学谦
梁斌
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen Graduate School Tsinghua University
Original Assignee
Shenzhen Graduate School Tsinghua University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen Graduate School Tsinghua University filed Critical Shenzhen Graduate School Tsinghua University
Priority to CN201910555500.4A priority Critical patent/CN110287873B/en
Publication of CN110287873A publication Critical patent/CN110287873A/en
Application granted granted Critical
Publication of CN110287873B publication Critical patent/CN110287873B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Molecular Biology (AREA)
  • Computational Linguistics (AREA)
  • Software Systems (AREA)
  • Mathematical Physics (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computing Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Human Computer Interaction (AREA)
  • Multimedia (AREA)
  • Image Analysis (AREA)

Abstract

The invention provides a non-cooperative target pose measurement method, a non-cooperative target pose measurement system and terminal equipment based on a deep neural network, wherein the method comprises the following steps: carrying out down-sampling on data point clouds and model point clouds of different angles of a non-cooperative target to obtain a point cloud; extracting a feature matrix containing the feature vector of each point and a global feature vector by using a trained PointNet network model; screening the feature points of the point cloud after down-sampling according to a preset feature point detection screening threshold, and performing feature point matching to obtain a feature point set; carrying out point cloud registration on the feature point set to obtain a pose transformation matrix; and applying the pose conversion matrix to the data point cloud after down sampling to obtain a new point cloud, and performing point cloud registration on the new point cloud and the model point cloud after down sampling to obtain a new pose conversion matrix. The method meets the real-time requirement of the pose measurement based on the space non-cooperative target on the basis of ensuring relatively high precision.

Description

Non-cooperative target pose measurement method and system based on deep neural network and terminal equipment
Technical Field
The invention relates to the technical field of short-distance non-cooperative target pose measurement, in particular to a short-distance non-cooperative target pose measurement method and system based on a deep neural network and a terminal device.
Background
Deep learning is to train a complex deep neural network on the premise of training samples with large data volume, so that the deep learning has a strong feature extraction function, and surpasses the traditional computer vision algorithm in many fields, particularly the computer vision field, such as tasks of classifying, identifying and detecting objects in ImageNet vision identification challenge games and the like. One of the reasons why deep learning succeeds in processing two-dimensional pictures is that a regular convolution kernel sharing mechanism can be used in a two-dimensional picture matrix which is regularly arranged in an array form, so that the parameter quantity of a neural network is greatly reduced. Due to the specific attributes of the three-dimensional point cloud: firstly, the point cloud data points are high in dimensionality, unstructured, large in scale, disordered and disordered, and the specific geometric attribute is difficult to directly utilize the existing mature convolutional neural network model and cannot be migrated. Secondly, the point cloud data are unevenly distributed along with the change of environmental illumination, and the difficulty of point cloud processing is increased due to incomplete structure caused by shielding or scanning angles; finally, although the three-dimensional sensor is rapidly developed, there still exist noise caused by error generation and other factors in the environment, and the massive nature of the point cloud data has great challenges to the processing efficiency.
The method for popularizing the neural network in the two-dimensional picture to the three-dimensional point cloud at present is to solve the problem of point cloud sequence standardization, for example, VoxNet and Voxception-ResNet apply three-dimensional convolution to voxelized point cloud. Another solution is to use a mature deep neural network for processing two-dimensional pictures, which first converts three-dimensional disordered point clouds into two-dimensional pictures, such as Multi-view CNN, by rendering the three-dimensional point clouds or projecting the three-dimensional point clouds onto two-dimensional pictures, and then apply a mature 2D convolutional network to the resulting two-dimensional pictures. However, both methods have certain limitations, the method based on voxelization can only process point cloud data with small data volume and small resolution, and the method based on multi-view loses a certain amount of spatial information, and has poor robustness to some point cloud data which are partially lost. Starting from a PointNet network proposed by Stanford university in 2017, a deep neural network directly processing disordered point cloud data logs on a historical stage, and max points are used as a symmetric function, so that the characteristics of the point cloud can be effectively extracted.
Most of the existing point cloud features are manually established features aiming at specific tasks. These features are basically encoding some specific geometric features or statistical properties and are designed to promote robustness and invariance to specific transformations. But these features are not easy to find an optimal combination of features for unknown tasks. Although the manual features are mature, such as SHOT, FPFH, etc., this feature-based method cannot exhaust all the basic feature vectors in the vector space of the point cloud data, and can only find a proper feature description in a limited feature vector space. Therefore, such methods must have a bottleneck in convergence and accuracy.
Disclosure of Invention
The invention provides a method, a system and a terminal device for measuring the short-distance non-cooperative target pose based on a deep neural network, aiming at solving the problem of a point cloud data processing method in the prior art.
In order to solve the above problems, the technical solution adopted by the present invention is as follows:
the invention provides a non-cooperative target pose measurement method based on a deep neural network, which comprises the following steps: s1: carrying out down-sampling on the data point cloud P and the model point cloud Q of the non-cooperative target at different angles to obtain point clouds P ', Q', S2: carrying out feature extraction on the point cloud P ', Q' by using the trained PointNet network model to obtain a feature matrix A containing the feature vector of each pointn×1024And a global feature vector B1×1024(ii) a S3: screening the feature points of the point clouds P 'and Q' according to a preset feature point detection screening threshold, and screening the feature points according to the global feature vector B1×1024A feature matrix A of the feature vectors associated with said points of said point cloud P', Qn×1024Matching the characteristic points to obtain a characteristic point set P ', Q'; s4: point cloud registration is carried out on the feature point sets P 'and Q' to obtain a pose transformation matrix T1＝[R1,t1](ii) a S5: converting the pose to a matrix T1Acting on the point cloud P' to obtain the point cloud
Figure GDA0002938692980000021
And the point cloud
Figure GDA0002938692980000022
Carrying out point cloud registration on the point cloud Q' to obtain a pose transformation matrix T2＝[R2,t2]。
Preferably, the down-sampling is performed according to curvature features, density of the point cloud, or normal in step S1.
Preferably, the number of sampling points for performing the down-sampling in step S1 is 1024.
Preferably, the feature point matching is performed by a TrICP algorithm in step S3.
Preferably, the feature point matching is performed by a TrICP algorithm in step S4.
The invention also provides a non-cooperative target pose measurement system based on the deep neural network, which comprises the following steps: a first unit: the method comprises the following steps of carrying out down-sampling on data point clouds P and model point clouds Q of non-cooperative targets at different angles to obtain point clouds P ', Q', and a second unit: carrying out feature extraction on the point cloud P ', Q' by using the trained PointNet network model to obtain a feature matrix A containing the feature vector of each pointn×1024And a global feature vector B1×1024(ii) a A third unit: screening the feature points of the point clouds P 'and Q' according to a preset feature point detection screening threshold, and screening the feature points according to the global feature vector B1×1024With the feature matrix A of the point cloud P ', Q' containing the feature vector of each pointn×1024Matching the characteristic points to obtain a characteristic point set P ', Q'; a fourth unit: point cloud registration is carried out on the feature point sets P 'and Q' to obtain a pose transformation matrix T1＝[R1,t1](ii) a A fifth unit: converting the pose to a matrix T1Acting on the point cloud P' to obtain the point cloud
Figure GDA0002938692980000031
And the point cloud
Figure GDA0002938692980000032
Carrying out point cloud registration on the point cloud Q' to obtain a bitAttitude transformation matrix T2＝[R2,t2]。
Preferably, the down-sampling is performed in terms of curvature features, density of the point cloud, or normal.
Preferably, the number of sampling points for performing the down-sampling is 1024; and performing the feature point matching through a TrICP algorithm.
The invention further provides a non-cooperative target pose measurement terminal device based on a deep neural network, which comprises a memory, a processor and a computer program stored in the memory and capable of running on the processor, wherein the processor realizes the steps of any one of the methods when executing the computer program.
The invention further provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, carries out the steps of the method as set forth in any of the above.
The invention has the beneficial effects that: the method, the system and the terminal device for measuring the short-distance non-cooperative target pose based on the deep neural network are characterized in that the deep neural network is subjected to principle analysis, then a data set is manufactured according to the pose measurement of the spatial non-cooperative target, the integrity of point cloud can be guaranteed as far as possible on the basis of simplifying data and removing redundancy, a deep neural network model is trained, the feature vector and the global feature vector of each point are effectively extracted, and the algorithm speed and the matching precision are improved. On the basis of ensuring relatively high precision, the real-time requirement of pose measurement based on the space non-cooperative target is met.
Drawings
FIG. 1 is a schematic diagram of a non-cooperative target pose measurement method based on a deep neural network in an embodiment of the present invention.
FIG. 2 is a schematic diagram of a non-cooperative target pose measurement system based on a deep neural network in the embodiment of the invention.
FIG. 3 is a diagram illustrating raw data before pose estimation of non-cooperative targets in an embodiment of the present invention.
Fig. 4 is a diagram illustrating a result of detecting a feature point of a non-cooperative target in the embodiment of the present invention.
FIG. 5 is a diagram illustrating results of coarse matching and fine matching of non-cooperative targets in an embodiment of the present invention.
Detailed Description
In order to make the technical problems, technical solutions and advantageous effects to be solved by the embodiments of the present invention more clearly apparent, the present invention is further described in detail below with reference to the accompanying drawings and the embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
It will be understood that when an element is referred to as being "secured to" or "disposed on" another element, it can be directly on the other element or be indirectly on the other element. When an element is referred to as being "connected to" another element, it can be directly connected to the other element or be indirectly connected to the other element. The connection may be for fixation or for circuit connection.
It is to be understood that the terms "length," "width," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," and the like are used in an orientation or positional relationship indicated in the drawings for convenience in describing the embodiments of the present invention and to simplify the description, and are not intended to indicate or imply that the referenced device or element must have a particular orientation, be constructed in a particular orientation, and be in any way limiting of the present invention.
Furthermore, the terms "first", "second" and "first" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of the embodiments of the present invention, "a plurality" means two or more unless specifically limited otherwise.
Example 1
The PointNet network aims at the tasks of point cloud classification and segmentation in the field of stereoscopic vision, and obtains good results in each mainstream database. The starting point of the algorithm design is to solve the problem of point cloud data disorder. In the initial phase, the processing of each point is the same and independent, and in the basic setup, each point is composed of its three coordinates. The key of the method is that max posing is used as a symmetric function, so that the extracted feature vector can ignore the disorder of the point cloud data.
Due to the problem of point cloud data disorder, it is difficult for the network to learn a consistent mapping from input to output. In order to make the neural network model immune to input data ordering, there may be three solution strategies: the first is to sort the input data in a canonical order, which sounds simple but does not actually have a stable ordering of point perturbations in a high dimensional space; secondly, an input sequence is used as a sequence for training RNN, training data are added through various arrangement modes, however, the RNN has better robustness on input sequencing of small-length sequences (dozens), but is difficult to expand to thousands of input data and more, and obviously is not suitable for massive point cloud data; the third is to use a simple symmetric function to aggregate the information of each point, i.e. input n vectors, pass through a symmetric function and output a new vector which is not affected by the input sequence, and satisfy:
f({x1,x2,...,xn})＝Γ(MAX(h(x1),...,h(xn))) (1)
wherein, given an unordered set of points { x }1,x2,…,xn}，xi∈R3The set function Γ is defined to map a set of point cloud points to a feature vector. The method comprises the steps of selecting max posing in a PointNet network as a symmetric function, selecting a simple multilayer perceptron (MLP) for functions h and Γ, processing point by point to obtain a high-dimensional vector of each point, and finally obtaining a global feature vector through the max posing symmetric function. Because the input point cloud data is easy to applyRigid or affine transformations, and each point is transformed independently. Therefore, a space transformation network T-Net depending on multi-dimensional data input is added in the network, the T-Net is a transformation matrix in a feature vector space and is mini-PointNet, the original point cloud is used as input, and output returns to a 3 x 3 matrix or a 64 x 64 matrix. The function of the method is to normalize the data once before processing the input points, so that the spatial transformation can be invariant, and the result is further improved.
Point cloud classification networks are described here with emphasis: in the starting stage, firstly, the initial n × 3-dimensional point cloud data is subjected to a normalization operation of a spatial transformation network T-Net with the dimensionality of 3 × 3, then each point of the point cloud initial data is raised to n × 64 dimensionality by a shared MLP (64, 64) network, then each point of the point cloud initial data is raised to n × 1024 dimensionality by the normalization operation of the spatial transformation network T-Net with the dimensionality of 64 × 64, and finally a global feature vector is obtained through a max pooling symmetric function. And finally obtaining the score value of the k classification by using the global feature vector through a two-layer MLP (512,256) full-connection network to judge the classification result. It can be seen that the classification network is based on the finally obtained global feature vector, and the global feature vector is obtained by an n × 1024 matrix composed of 1024-dimensional vectors corresponding to each point through a max power symmetric function. All but the last layer include the choice of the ReLU activation function and batch normalization (batch normalization) is applied. The final classification accuracy of the PointNet network in the public data set ModelNet40 reaches 89.2%, which is far higher than 85.9% of VoxNet.
In order to realize that the spatial non-cooperative target ultra-close range pose measurement pays attention to accuracy and real-time performance, the invention extracts the point cloud based on the deep neural network. The method comprises the steps of firstly, making a data set aiming at the pose measurement of the space non-cooperative target, training a deep neural network model, and then applying the trained deep neural network model to the pose measurement of the short-distance non-cooperative target, and aims to provide pose data of the non-cooperative target in real time for a space on-orbit task.
As shown in fig. 1, the invention provides a non-cooperative target pose measurement method based on a deep neural network, comprising the following steps:
s1: carrying out down-sampling on the data point cloud P and the model point cloud Q of the non-cooperative target at different angles to obtain point clouds P 'and Q';
in mass point cloud data, for any point x, there are generally two types of geometric features describing the point, namely, a feature value and a corresponding feature vector. The curvature characteristics are an important basis for characteristic identification, the curvature value of a point reflects the concave-convex degree of the point on the surface of the point cloud, and a matching point can be effectively searched for two scattered point clouds. The algorithm provided by the invention is based on the curvature information of the three-dimensional point cloud data, so that the down-sampling is selected according to the curvature characteristics. And the two scattered point clouds P and Q are subjected to secondary sampling, so that the accuracy and the integrity of the curvature characteristic after sampling can be ensured.
It is understood that the down-sampling can be performed according to geometric features such as density, normal and the like of the point cloud in addition to the curvature features.
In an embodiment of the present invention, the PointNet network model used is trained based on 1024 point cloud data, so that 1024 points are obtained after down-sampling.
S2: carrying out feature extraction on the point cloud P ', Q' by using the trained PointNet network model to obtain a feature matrix A containing the feature vector of each pointn×1024And a global feature vector B1×1024；
The PointNet network model has stronger feature extraction capability, and each point in the point cloud P ', Q' is subjected to two layers of T-Net network conversion and five layers of MLP feature extraction layers, so that the feature of each point is finally promoted to 1024 dimensions from a three-dimensional coordinate. The characteristics finally extracted by the PointNet network comprise a characteristic matrix A of the characteristic vector of each pointn×1024And a global feature vector B describing the point cloud as a whole1×1024。
The global feature vector can effectively observe points with obvious features in the point cloud, namely, the point cloud can be used as a feature point detection algorithm to enable a smaller number of feature points to represent the whole point cloud, and therefore the calculation complexity of the algorithm can be greatly reduced; meanwhile, the feature matrix of the point feature vector of each point can be used for feature point matching. The most straightforward approach to feature point matching is based on global feature vectors, which is very efficient for registration.
S3: screening the feature points of the point clouds P 'and Q' according to a preset feature point detection screening threshold, and screening the feature points according to the global feature vector B1×1024A feature matrix A of the feature vectors associated with said points of said point cloud P', Qn×1024Matching the characteristic points to obtain a characteristic point set P ', Q';
the main reasons influencing the efficiency of the point cloud pose estimation algorithm include the number of point clouds after feature point detection, calculation of feature descriptors, feature point pairing and point cloud matching. The characteristic point detection is a very main step, the most representative minimum characteristic points can be accurately detected, redundant points are eliminated, and the method is the basis for improving the algorithm efficiency. Global feature vector B describing the whole point cloud obtained in step S21×1024Setting a feature point detection screening threshold tau to screen P 'and Q', and describing a subvector B according to global features1×1024Feature matrix A of feature vectors for each point of feature point set P', Qn×1024And matching the feature points to obtain a feature point set P ', Q'.
S4: point cloud registration is carried out on the feature point sets P 'and Q' to obtain a pose transformation matrix T1＝[R1,t1]；
The registration of two scattered point clouds is essentially to find rigid transformation containing a rotation matrix R and a translation vector t, and the transformation can transform two pieces of mass scattered point cloud data in different coordinate systems into the same coordinate system and realize accurate registration and coincidence. TrICP (trimmed ICP) is a new robust improved version of a traditional ICP algorithm, the method sorts the square errors of all matching point pairs, only optimizes a certain number of smaller values, the number is obtained according to the initial overlapping rate of two pieces of point clouds, only data smaller than the median of the square error sorting is taken to participate in optimization relative to an LMedS algorithm, and the TrICP has better convergence rate and robustness.
S5: converting the pose to a matrix T1Acting on the point cloud P' to obtain the point cloud
Figure GDA0002938692980000071
And the point cloud
Figure GDA0002938692980000072
Carrying out point cloud registration on the point cloud Q' to obtain a pose transformation matrix T2＝[R2,t2]。
And (4) applying the pose data obtained by the coarse registration to the original data, and then further performing accurate registration by using TrICP (TrICP), so that the pose data obtained by the accurate registration can be obtained. Through the two steps, the pose with 6 degrees of freedom of the whole measuring process can be obtained. The invention adds the distance weighting method into the TrICP (trained ICP) algorithm, thereby not only obtaining a more accurate matching point pair number, but also further strengthening the function of correct matching point pairs, weakening the influence of wrong matching point pairs, simplifying the calculation amount of mass data and improving the accuracy of results.
Compared with the prior art, the invention has the innovativeness that:
a. the method of sampling according to the geometric features is applied to the mass data, and the integrity of the point cloud can be ensured as far as possible on the basis of simplifying the data and removing redundancy;
b. in the process of extracting the point cloud data features, a trained PointNet deep neural network is utilized, so that the feature vector of each point can be effectively extracted, and the algorithm speed and robustness are improved;
c. in the point matching search strategy, the algorithm process is further accelerated by using the global feature vector extracted based on the trained PointNet deep neural network;
d. the method of weighting according to the distance is added into a TrICP (trained ICP) algorithm for fine matching, so that not only can a more accurate matching point pair number be obtained, but also the effect of a correct matching point pair can be further enhanced, the influence of an incorrect matching point pair is weakened, and the precision and the robustness of the result are greatly improved.
Example 2
As shown in fig. 2, the present invention further provides a non-cooperative target pose measurement system based on a deep neural network, including:
a first unit: carrying out down-sampling on the data point cloud P and the model point cloud Q of the non-cooperative target at different angles to obtain point clouds P 'and Q';
a second unit: carrying out feature extraction on the point cloud P ', Q' by using the trained PointNet network model to obtain a feature matrix A containing the feature vector of each pointn×1024And a global feature vector B1×1024；
A third unit: screening the feature points of the point clouds P 'and Q' according to a preset feature point detection screening threshold, and screening the feature points according to the global feature vector B1×1024With the feature matrix A of the point cloud P ', Q' containing the feature vector of each pointn×1024Matching the characteristic points to obtain a characteristic point set P ', Q';
a fourth unit: point cloud registration is carried out on the feature point sets P 'and Q' to obtain a pose transformation matrix T1＝[R1,t1]；
A fifth unit: converting the pose to a matrix T1Acting on the point cloud P' to obtain the point cloud
Figure GDA0002938692980000081
And the point cloud
Figure GDA0002938692980000082
Carrying out point cloud registration on the point cloud Q' to obtain a pose transformation matrix T2＝[R2,t2]。
In one embodiment of the invention, the first unit performs said down-sampling according to curvature features, density of the point cloud or normal. The number of sampling points for carrying out the downsampling is 1024; and the fourth unit and the fifth unit carry out feature point matching through a TrICP algorithm.
The deep neural network-based non-cooperative target pose measurement system of this embodiment further includes: a processor, a memory, and a computer program stored in the memory and executable on the processor, such as a program that down-samples a data point cloud P and a model point cloud Q from different angles of a non-cooperative target to obtain a point cloud P ', Q'. The processor, when executing the computer program, implements the steps in each of the deep neural network-based non-cooperative object pose measurement method embodiments described above, such as steps S1-S5 shown in fig. 1. Alternatively, the processor, when executing the computer program, implements the functions of the modules/units in the above device embodiments, for example, the first unit: and (3) carrying out down-sampling on the data point cloud P and the model point cloud Q of the non-cooperative target at different angles to obtain point clouds P 'and Q'.
Illustratively, the computer program may be divided into one or more units, the five units being merely exemplary. The one or more modules/units are stored in the memory and executed by the processor to implement the present invention. The one or more units may be a series of instruction segments of a computer program capable of performing specific functions, and the instruction segments are used for describing the execution process of the computer program in the deep neural network-based non-cooperative target pose measurement system.
The non-cooperative target pose measurement system based on the deep neural network can also comprise, but is not limited to, a processor and a memory. Those skilled in the art will appreciate that the schematic diagram is merely an example of a deep neural network based non-cooperative object pose measurement system, and does not constitute a limitation of a deep neural network based non-cooperative object pose measurement system, and may include more or fewer components than those shown, or some components in combination, or different components, for example, the deep neural network based non-cooperative object pose measurement system may further include input-output devices, network access devices, buses, etc.
The Processor may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can be any conventional processor and the like, the processor is a control center of the non-cooperative target pose measurement system based on the deep neural network, and various interfaces and lines are used for connecting all parts of the non-cooperative target pose measurement system based on the deep neural network.
The memory may be used to store the computer programs and/or modules, and the processor may implement the various functions of the deep neural network-based non-cooperative target pose measurement system by running or executing the computer programs and/or modules stored in the memory and invoking data stored in the memory. The memory may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required by at least one function (such as a sound playing function, an image playing function, etc.), and the like; the storage data area may store data (such as audio data, a phonebook, etc.) created according to the use of the cellular phone, and the like. In addition, the memory may include high speed random access memory, and may also include non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), at least one magnetic disk storage device, a Flash memory device, or other volatile solid state storage device.
The integrated unit of the non-cooperative target pose measurement system based on the deep neural network can be stored in a computer readable storage medium if the integrated unit is realized in the form of a software functional unit and is sold or used as an independent product. Based on such understanding, all or part of the flow of the method according to the embodiments of the present invention may also be implemented by a computer program, which may be stored in a computer-readable storage medium, and when the computer program is executed by a processor, the steps of the method embodiments may be implemented. Wherein the computer program comprises computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, usb disk, removable hard disk, magnetic disk, optical disk, computer Memory, Read-Only Memory (ROM), Random Access Memory (RAM), electrical carrier wave signals, telecommunications signals, software distribution medium, and the like. It should be noted that the computer readable medium may contain content that is subject to appropriate increase or decrease as required by legislation and patent practice in jurisdictions, for example, in some jurisdictions, computer readable media does not include electrical carrier signals and telecommunications signals as is required by legislation and patent practice.
Example 3
The embodiment is used for carrying out simulation verification on a non-cooperative target pose measurement method based on a deep neural network. The performance of the algorithm is evaluated by adopting the Bunny three-dimensional point cloud data in the Stanford University graphical Laboratory data set. In the experiment, firstly, the detection result of the characteristic points is visualized, and then the comparison of the algorithm with the traditional geometric characteristic descriptor FPFH is carried out to verify the real-time performance and the accuracy of the algorithm.
Fig. 3 is a schematic diagram of raw data before pose estimation in this embodiment. A new point cloud is obtained by down-sampling the original data in fig. 3.
Fig. 4 is a schematic diagram showing the result of feature point detection in the preferred embodiment of the present invention. Feature point detection is carried out on the point cloud obtained by down sampling through the global feature vector obtained by the PointNet network model, the outline of which the feature point is approximately the original point cloud data is obtained, the feature points of the point cloud can be well extracted, the number of points is reduced, and the number of the point cloud in the graph is respectively reduced from 1024 to 103 and 109, which is about ten times reduced.
As shown in fig. 5, pose estimation of the non-cooperative target is obtained after coarse registration and fine registration. Only one more accurate matching point pair number can be obtained, the effect of correct matching point pairs can be further enhanced, the influence of wrong matching point pairs is weakened, and the precision and the robustness of results are greatly improved.
From the performance comparison of the two algorithms in table 1, the conventional manual feature FPFH descriptor cannot cope with a small amount of point cloud data only containing feature points, because the abstract capability of the FPFH descriptor is reduced due to the loss of local information, and the time efficiency of coarse registration is reduced. The algorithm provided by the invention is based on the PointNet neural network, has strong feature extraction capability, and has higher efficiency than the traditional manual feature when the trained model is directly applied to the feature extraction of the point cloud. As can be seen from table 1, the total time consumed for one pose estimation is 0.216s, so that the pose result can be returned about 4 times per second in actual measurement, and the real-time requirement is met.
TABLE 1 Performance comparison of the three algorithms
Figure GDA0002938692980000111
With the rapid development of the technology, the number of the acquired cloud points of the three-dimensional point is more and more huge, and how to process massive three-dimensional data and achieve rapid and high-precision registration becomes a difficult point for research. The traditional point cloud processing method based on the geometric feature descriptor has great limitation in universality because the point cloud processing method is manually established for a specific task. However, the deep neural network based method does not have such limitation, and the feature extraction capability of the deep neural network based method is stronger and stronger as the data set is increased. Therefore, in the field of point cloud data processing, a deep neural network-based method will become mainstream in the future.
The foregoing is a more detailed description of the invention in connection with specific preferred embodiments and it is not intended that the invention be limited to these specific details. For those skilled in the art to which the invention pertains, several equivalent substitutions or obvious modifications can be made without departing from the spirit of the invention, and all the properties or uses are considered to be within the scope of the invention.

Claims (10)

1. A non-cooperative target pose measurement method based on a deep neural network is characterized by comprising the following steps:
s1: carrying out down-sampling on the data point cloud P and the model point cloud Q of the non-cooperative target at different angles to obtain point clouds P 'and Q';
s2: carrying out feature extraction on the point cloud P ', Q' by using the trained PointNet network model to obtain a feature matrix A containing the feature vector of each pointn×1024And a global feature vector B1×1024；
S3: according to the global feature vector B1×1024Screening the feature points of the point clouds P 'and Q' according to a preset feature point detection screening threshold, and screening the feature points according to the global feature vector B1×1024A feature matrix A of the feature vectors associated with said points of said point cloud P', Qn×1024Matching the characteristic points to obtain a characteristic point set P ', Q';
s4: point cloud registration is carried out on the feature point sets P 'and Q' to obtain a pose transformation matrix T1＝[R1,t1]Where R is a rotation matrix and t is a translation vector;
s5: converting the pose to a matrix T1Acting on the point cloud P' to obtain the point cloud
Figure FDA0003078516100000011
And the point cloud
Figure FDA0003078516100000012
Carrying out point cloud registration on the point cloud Q' to obtain a pose transformation matrix T2＝[R2,t2]And applying the pose data obtained by coarse registration to the original data, further carrying out accurate registration and obtaining the pose data of accurate registration.
2. The deep neural network-based non-cooperative object pose measurement method according to claim 1, wherein the down-sampling is performed according to a curvature feature, a density of a point cloud, or a normal in step S1.
3. The deep neural network-based non-cooperative object pose measurement method according to claim 1, wherein the number of sampling points for performing the down-sampling in step S1 is 1024.
4. The deep neural network-based non-cooperative target pose measurement method according to claim 1, wherein the feature point matching is performed by a TrICP algorithm in step S3.
5. The deep neural network-based non-cooperative target pose measurement method according to claim 1, wherein the point cloud registration is performed by a TrICP algorithm in step S4.
6. A non-cooperative target pose measurement system based on a deep neural network is characterized by comprising:
a first unit: the data point cloud P and the model point cloud Q of different angles of the non-cooperative target are down sampled to obtain the point clouds P ', Q',
a second unit: carrying out feature extraction on the point cloud P ', Q' by using the trained PointNet network model to obtain a feature matrix A containing the feature vector of each pointn×1024And a global feature vector B1×1024；
A third unit: according to the global feature vector B1×1024Screening the feature points of the point clouds P 'and Q' according to a preset feature point detection screening threshold, and screening the feature points according to the global feature vector B1×1024With the feature matrix A of the point cloud P ', Q' containing the feature vector of each pointn×1024Matching the characteristic points to obtain a characteristic point set P ', Q';
a fourth unit: point cloud registration is carried out on the feature point sets P 'and Q' to obtain a pose transformation matrix T1＝[R1,t1]Where R is a rotation matrix and t is a translation vector;
a fifth unit: converting the pose to a matrix T1Acting on the point cloud P' to obtain the point cloud
Figure FDA0003078516100000021
And the point cloud
Figure FDA0003078516100000022
Carrying out point cloud registration on the point cloud Q' to obtain a pose transformation matrix T2＝[R2,t2]And applying the pose data obtained by coarse registration to the original data, further carrying out accurate registration and obtaining the pose data of accurate registration.
7. The deep neural network-based non-cooperative object pose measurement system of claim 6, wherein the down-sampling is performed in terms of curvature features, density of point clouds, or normals.
8. The deep neural network-based non-cooperative target pose measurement system of claim 6, wherein the number of sampling points for the down-sampling is 1024; and performing the feature point matching through a TrICP algorithm.
9. A non-cooperative target pose measurement terminal device based on a deep neural network, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method according to any one of claims 1 to 5 when executing the computer program.
10. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 5.
CN201910555500.4A 2019-06-25 2019-06-25 Non-cooperative target pose measurement method and system based on deep neural network and terminal equipment Active CN110287873B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910555500.4A CN110287873B (en) 2019-06-25 2019-06-25 Non-cooperative target pose measurement method and system based on deep neural network and terminal equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910555500.4A CN110287873B (en) 2019-06-25 2019-06-25 Non-cooperative target pose measurement method and system based on deep neural network and terminal equipment

Publications (2)

Publication Number Publication Date
CN110287873A CN110287873A (en) 2019-09-27
CN110287873B true CN110287873B (en) 2021-06-29

Family

ID=68005745

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910555500.4A Active CN110287873B (en) 2019-06-25 2019-06-25 Non-cooperative target pose measurement method and system based on deep neural network and terminal equipment

Country Status (1)

Country Link
CN (1) CN110287873B (en)

Families Citing this family (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111047631B (en) * 2019-12-04 2023-04-07 广西大学 Multi-view three-dimensional point cloud registration method based on single Kinect and round box
CN111144483B (en) 2019-12-26 2023-10-17 歌尔股份有限公司 Image feature point filtering method and terminal
CN111223136B (en) * 2020-01-03 2024-04-23 三星（中国）半导体有限公司 Depth feature extraction method and device for sparse 2D point set
CN111402328B (en) * 2020-03-17 2023-11-10 北京图森智途科技有限公司 Pose calculation method and device based on laser odometer
CN111832473A (en) * 2020-07-10 2020-10-27 星际空间（天津）科技发展有限公司 Point cloud feature identification processing method and device, storage medium and electronic equipment
CN112017225B (en) * 2020-08-04 2023-06-09 华东师范大学 Depth image matching method based on point cloud registration
CN112559959B (en) * 2020-12-07 2023-11-07 中国西安卫星测控中心 Space-based imaging non-cooperative target rotation state resolving method based on feature vector
CN112700455A (en) * 2020-12-28 2021-04-23 北京超星未来科技有限公司 Laser point cloud data generation method, device, equipment and medium
CN113034439B (en) * 2021-03-03 2021-11-23 北京交通大学 High-speed railway sound barrier defect detection method and device
CN114310873B (en) * 2021-12-17 2024-05-24 上海术航机器人有限公司 Pose conversion model generation method, control method, system, equipment and medium
CN117152245B (en) * 2023-01-31 2024-09-03 荣耀终端有限公司 Pose calculation method and device
CN116363217B (en) * 2023-06-01 2023-08-11 中国人民解放军国防科技大学 Method, device, computer equipment and medium for measuring pose of space non-cooperative target

Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104048648A (en) * 2014-05-27 2014-09-17 清华大学深圳研究生院 Relative pose measurement method for large size non-cooperative target
CN106780459A (en) * 2016-12-12 2017-05-31 华中科技大学 A kind of three dimensional point cloud autoegistration method
CN107449402A (en) * 2017-07-31 2017-12-08 清华大学深圳研究生院 A kind of measuring method of the relative pose of noncooperative target
CN108133458A (en) * 2018-01-17 2018-06-08 视缘(上海)智能科技有限公司 A kind of method for automatically split-jointing based on target object spatial point cloud feature
CN108376408A (en) * 2018-01-30 2018-08-07 清华大学深圳研究生院 A kind of three dimensional point cloud based on curvature feature quickly weights method for registering
CN109458994A (en) * 2018-10-24 2019-03-12 北京控制工程研究所 A kind of space non-cooperative target laser point cloud ICP pose matching correctness method of discrimination and system
CN109523552A (en) * 2018-10-24 2019-03-26 青岛智能产业技术研究院 Three-dimension object detection method based on cone point cloud
CN109523501A (en) * 2018-04-28 2019-03-26 江苏理工学院 One kind being based on dimensionality reduction and the matched battery open defect detection method of point cloud data
CN109887028A (en) * 2019-01-09 2019-06-14 天津大学 A kind of unmanned vehicle assisted location method based on cloud data registration
CN109919984A (en) * 2019-04-15 2019-06-21 武汉惟景三维科技有限公司 A kind of point cloud autoegistration method based on local feature description's

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105976353B (en) * 2016-04-14 2020-01-24 南京理工大学 Spatial non-cooperative target pose estimation method based on model and point cloud global matching
CN109308737A (en) * 2018-07-11 2019-02-05 重庆邮电大学 A kind of mobile robot V-SLAM method of three stage point cloud registration methods
CN109102547A (en) * 2018-07-20 2018-12-28 上海节卡机器人科技有限公司 Robot based on object identification deep learning model grabs position and orientation estimation method
CN109801337B (en) * 2019-01-21 2020-10-02 同济大学 6D pose estimation method based on instance segmentation network and iterative optimization

Patent Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104048648A (en) * 2014-05-27 2014-09-17 清华大学深圳研究生院 Relative pose measurement method for large size non-cooperative target
CN106780459A (en) * 2016-12-12 2017-05-31 华中科技大学 A kind of three dimensional point cloud autoegistration method
CN107449402A (en) * 2017-07-31 2017-12-08 清华大学深圳研究生院 A kind of measuring method of the relative pose of noncooperative target
CN108133458A (en) * 2018-01-17 2018-06-08 视缘(上海)智能科技有限公司 A kind of method for automatically split-jointing based on target object spatial point cloud feature
CN108376408A (en) * 2018-01-30 2018-08-07 清华大学深圳研究生院 A kind of three dimensional point cloud based on curvature feature quickly weights method for registering
CN109523501A (en) * 2018-04-28 2019-03-26 江苏理工学院 One kind being based on dimensionality reduction and the matched battery open defect detection method of point cloud data
CN109458994A (en) * 2018-10-24 2019-03-12 北京控制工程研究所 A kind of space non-cooperative target laser point cloud ICP pose matching correctness method of discrimination and system
CN109523552A (en) * 2018-10-24 2019-03-26 青岛智能产业技术研究院 Three-dimension object detection method based on cone point cloud
CN109887028A (en) * 2019-01-09 2019-06-14 天津大学 A kind of unmanned vehicle assisted location method based on cloud data registration
CN109919984A (en) * 2019-04-15 2019-06-21 武汉惟景三维科技有限公司 A kind of point cloud autoegistration method based on local feature description's

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation;Charles R. Qi et al.;《arXiv》;20170410;第1-19页、图2 *
PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space;Charles R. Qi et al.;《31st Conference on Neural Information Processing Systems》;20171231;全文 *

Also Published As

Publication number Publication date
CN110287873A (en) 2019-09-27

Similar Documents

Publication Publication Date Title
CN110287873B (en) Non-cooperative target pose measurement method and system based on deep neural network and terminal equipment
Yang et al. A fast and robust local descriptor for 3D point cloud registration
Buch et al. Rotational subgroup voting and pose clustering for robust 3d object recognition
JP5705147B2 (en) Representing 3D objects or objects using descriptors
Guo et al. A novel local surface feature for 3D object recognition under clutter and occlusion
CN111795704A (en) Method and device for constructing visual point cloud map
Yang et al. Multi-attribute statistics histograms for accurate and robust pairwise registration of range images
Rodrigues et al. 6D pose estimation of textureless shiny objects using random ferns for bin-picking
Albarelli et al. Fast and accurate surface alignment through an isometry-enforcing game
Keselman et al. Many-to-many graph matching via metric embedding
CN104090972A (en) Image feature extraction and similarity measurement method used for three-dimensional city model retrieval
JP2011238204A (en) Method for recognition and position attitude determination of three-dimensional object at three-dimensional scene
WO2015002114A1 (en) Method and apparatus for determining pose of object in scene
Guo et al. 3D free form object recognition using rotational projection statistics
CN113160285A (en) Point cloud matching method based on local depth image criticality
Guo et al. 3D object recognition from cluttered and occluded scenes with a compact local feature
CN114332172A (en) Improved laser point cloud registration method based on covariance matrix
Srivastava et al. Drought stress classification using 3D plant models
CN115661218B (en) Virtual super-point-based laser point cloud registration method and system
Liu et al. Deep learning of directional truncated signed distance function for robust 3D object recognition
Song Local voxelizer: A shape descriptor for surface registration
CN109887012B (en) Point cloud registration method combined with self-adaptive search point set
Zhou et al. Hough-space-based hypothesis generation and hypothesis verification for 3D object recognition and 6D pose estimation
Sa et al. Depth grid-based local description for 3D point clouds
Liu et al. An improved local descriptor based object recognition in cluttered 3D point clouds

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant