CN112949380B - Intelligent underwater target identification system based on laser radar point cloud data - Google Patents
Intelligent underwater target identification system based on laser radar point cloud data Download PDFInfo
- Publication number
- CN112949380B CN112949380B CN202110023522.3A CN202110023522A CN112949380B CN 112949380 B CN112949380 B CN 112949380B CN 202110023522 A CN202110023522 A CN 202110023522A CN 112949380 B CN112949380 B CN 112949380B
- Authority
- CN
- China
- Prior art keywords
- point cloud
- cloud data
- dimensional
- data
- neural network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
- 238000012549 training Methods 0.000 claims abstract description 75
- 238000013527 convolutional neural network Methods 0.000 claims abstract description 30
- 238000007781 pre-processing Methods 0.000 claims abstract description 24
- 238000005070 sampling Methods 0.000 claims description 30
- 238000000034 method Methods 0.000 claims description 27
- 238000013528 artificial neural network Methods 0.000 claims description 20
- 230000008569 process Effects 0.000 claims description 20
- 238000012360 testing method Methods 0.000 claims description 14
- 239000013598 vector Substances 0.000 claims description 12
- 230000004913 activation Effects 0.000 claims description 8
- 230000000694 effects Effects 0.000 claims description 6
- 238000010606 normalization Methods 0.000 claims description 6
- 238000003062 neural network model Methods 0.000 claims description 5
- 238000012545 processing Methods 0.000 claims description 5
- 230000005540 biological transmission Effects 0.000 claims description 3
- 238000011478 gradient descent method Methods 0.000 claims description 3
- 238000005516 engineering process Methods 0.000 abstract description 14
- 239000010410 layer Substances 0.000 description 46
- 230000006870 function Effects 0.000 description 14
- 238000010586 diagram Methods 0.000 description 9
- 238000000605 extraction Methods 0.000 description 3
- 238000011160 research Methods 0.000 description 3
- 230000000007 visual effect Effects 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000007123 defense Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 210000002569 neuron Anatomy 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 238000002203 pretreatment Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 239000002344 surface layer Substances 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 238000000844 transformation Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
- G01S17/88—Lidar systems specially adapted for specific applications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/047—Probabilistic or stochastic networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
- G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Biophysics (AREA)
- Multimedia (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Biomedical Technology (AREA)
- Health & Medical Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Computational Linguistics (AREA)
- Probability & Statistics with Applications (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Computer Networks & Wireless Communication (AREA)
- Electromagnetism (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Image Processing (AREA)
Abstract
The invention discloses an intelligent underwater target recognition system based on laser radar point cloud data. And the laser radar scans an underwater target object and stores the obtained point cloud data into the database. The upper computer comprises a data preprocessing module, a cloud element coding module, a model training module, a model updating module and a result display module. The point cloud data are obtained by using the laser radar, the speed is high, the precision is high, the anti-interference capability is strong, the three-dimensional convolutional neural network directly uses the three-dimensional point cloud data to train the model, the characteristics of the point cloud data are utilized to the maximum extent, and the identification accuracy can be obviously improved. The invention provides an underwater target recognition instrument with strong intelligence, which brings remarkable synergy for an underwater target recognition technology.
Description
Technical Field
The invention relates to the field of radar data processing and the field of computer vision, in particular to an intelligent underwater target identification technology based on laser radar point cloud data, which is a new identification instrument for underwater targets.
Background
The object recognition is to realize the human visual recognition function by using a computer, and the research goal is to enable the computer to have the capability of recognizing the surrounding environment from one or more images or videos, including the perception, recognition and understanding of the three-dimensional environment of the objective world. The basic principle of target identification is that target feature information in radar echo is utilized, the size, the shape, the weight and physical characteristic parameters of a surface layer of a target are estimated through various mathematical multi-dimensional space transformations, and finally a specific object is identified in a classifier according to a discriminant function obtained by a large number of training samples. With the rapid development of computer vision technology in recent years, the target recognition technology is also widely applied to the fields of national economy, space technology, national defense and the like. Although the use of object recognition technology for identifying objects on the ground has become relatively mature, there are many shortcomings in identifying underwater objects, which make object recognition technology difficult to work underwater. Because underwater environments, particularly marine environments, are complex, many challenges, such as limited field of view, are associated with target recognition.
Due to the potential opportunities and application value of underwater object recognition technology, research on object recognition technology in underwater environments has also increased recently. Among the various technologies currently under study and applied to underwater target recognition, visual cameras, sonar sensors, GPS and lidar are representative. The visual camera may not work in low light conditions, while the sonar sensor is mainly used for the measurement of close objects and the detection of close range. GPS data requires a high and stable network connection, which is affected by weather conditions, real-time location, etc., and therefore, the results may be inaccurate in some cases. The laser radar has been most widely used due to its advantages of high resolution, strong active interference resistance, small size and light weight. The laser radar working principle is that laser is used as a signal source, pulse laser emitted by a laser device can cause scattering when the laser radar strikes an object, a part of light wave can be reflected to a receiver of the laser radar, the distance from the laser radar to a target point can be obtained according to calculation of a laser ranging principle, the pulse laser continuously scans a target object, data of all the target points on the target object can be obtained, and accurate three-dimensional images can be obtained after the data is used for imaging processing. Can be collected quickly and accurately, and can be operated well in the daytime and at night.
Three-dimensional stereo images obtained with lidar are typically represented as point cloud data, which refers to a collection of vectors in a three-dimensional coordinate system. These vectors are typically expressed in terms of X, Y, Z three-dimensional coordinates, which represent the shape of the external surface of an object. In addition to the geometric position information, the point cloud data may also represent RGB color, gray value, depth, and segmentation result of one point. However, in the existing technology of object recognition using point cloud data, the point cloud is usually mapped into a two-dimensional space to form a 2D image or three-dimensional point cloud data is converted into vectors, and then classified using a conventional neural network. However, the extraction capability of the features is limited during dimension reduction, which causes the loss of the features in the point cloud data, and the identification accuracy is reduced. Meanwhile, the existing underwater target recognition technology can only detect the target and then classify and calibrate the target manually, so that the consumption of human resources is high, and the intelligence is poor. Therefore, the existing underwater target identification technology has many defects, is a difficult point and a hot point for identifying targets at home and abroad, and has important academic research value and engineering practice value.
Disclosure of Invention
The invention aims to overcome the defects of poor characteristic extraction capability, low identification accuracy and low intelligence of the existing underwater target identification technology, and provides an intelligent underwater target identification system instrument which is strong in characteristic extraction capability and high in intelligence and is based on laser radar point cloud data.
The technical scheme adopted by the invention for solving the technical problems is as follows: an intelligent underwater target recognition system based on laser radar point cloud data comprises a laser radar, a database and an upper computer. The laser radar, the database and the upper computer are connected in sequence, and the underwater target of the recognizer is recognized according to the following processes:
1) scanning underwater objects needing to be identified by using a laser radar to obtain a laser radar point cloud data set, wherein each point cloud data set is a set of vectors which take the laser radar as a coordinate origin and are in a three-dimensional coordinate system, and the vectors are expressed in the form of X, Y and Z three-dimensional coordinates, namely
Pij＝(x,y,z)
Wherein, PijA j point of the ith point cloud data;
2) storing point cloud data acquired by scanning an underwater object by using a laser radar into a database of the identification instrument, wherein the point cloud data in the database is called by an upper computer;
the host computer of discernment appearance includes:
the data preprocessing module is used for preprocessing the point cloud data in the database and comprises the following steps:
3) acquiring N point cloud data from database to obtain training sample set
Itrain＝{D1,D2,···,DN}
Di＝{Pi,Oi}
Wherein D isiFor each sample used for training, PiFor the ith input point cloud data, OiThe output of the ith training sample represents the type of the recognized object;
4) performing geometric normalization processing on each point cloud data in the training sample set, wherein the geometric normalization is performed according to the following steps:
4.1) extracting the coordinate value X with the minimum numerical value in the X, Y and Z directions in the point cloud datamin，ymin，zminAnd the coordinate value x with the largest numerical valuemax，ymax，zmax；
4.2) mixing (x)min,ymin,zmin) As a new origin of coordinates, the coordinates of the X, Y,length of side taken in Z direction
HX＝(xmax-xmin)，HY＝(ymax-ymin)，HZ＝(zmax-zmin)
Obtaining a point cloud cuboid V which surrounds all points in the point cloud data;
4.3) obtaining new coordinate values of each point in the point cloud data by the following formula:
x'＝x-xmin,y'＝y-ymin,z'＝z-zmin
(x ', y ', z ') is the new coordinate value of the point cloud data which is obtained and is preprocessed by the preprocessing module;
the cloud element coding module is used for coding the preprocessed point cloud data to obtain a three-dimensional tensor of the input neural network, and the process is completed by adopting the following steps:
5) regarding the cuboid V surrounding the point cloud data obtained in the step 4), the cuboid V is respectively arranged in the X direction, the Y direction and the Z direction
Is divided into intervals of HX，HY，HZThe length of the cuboid V in the X, Y and Z directions respectively, namely the cuboid V is divided into H3Small cuboids, each small cuboid being called a cloud block;
6) encoding each cloud block, wherein the encoding process is carried out according to the following steps:
6.1) calculating the coordinate mean value of each point in the X, Y and Z directions for each cloud block:
wherein muX、μY、μZAre respectively the coordinate mean value, x 'of each point in the direction of X, Y, Z'j、y'j、z'jRespectively passing through the pretreatment module in the direction of X, Y, ZNew coordinate value of block pre-treatment, T is the number of points in each cloud block, then (μ)X,μY,μZ) As a block center;
6.2) sampling each cloud block, taking the T 'point closest to the mass center, taking all points if the T' points are insufficient in the cloud block, and calculating the Euclidean distance from each point to the block center:
summing the Euclidean distances of each point to the block center:
wherein ρjThe Euclidean distance from the jth point in the block to the block center, CiClouding element called i-th cloud piece, CiThe size of the point cloud data is the cloud element value of the ith cloud block, and each point cloud data is converted into a point cloud data set H3The individual cloud elements represent three-dimensional tensors, and a training sample set which converts point cloud data into cloud elements is obtained;
the model training module is used for training the three-dimensional convolutional neural network by using a training sample set represented by the encoded cloud element, and the model training module is completed by adopting the following processes:
7) setting a topological structure of a three-dimensional convolutional neural network, wherein the three-dimensional convolutional neural network consists of a characteristic layer, a down-sampling layer and a full-connection layer, the size and the number of convolutional cores need to be set, an activation function can be optionally set, the number of nodes of each layer of the full-connection layer is set, and each hyper-parameter of the neural network is set;
8) initializing a weight value and a bias value of a neural network;
9) with training sample set I'train＝{D'1,D'2,···,D'NTrain convolutional neural network, where'trainIs a training sample set D 'processed by a data preprocessing module'1,D'2,···,D'NIs processed by a data preprocessing module in the training sample setThe training process of each training sample is carried out according to the following steps:
9.1) inputting training sample set data into an input layer of a three-dimensional convolutional neural network, wherein the input data is a three-dimensional tensor;
9.2) carrying out convolution operation on input data through a convolution kernel, adding a convolution weighted sum and an offset term to obtain a value, obtaining a corresponding cloud pixel value in a characteristic layer through an activation function according to the value, wherein the convolution step length is 1, the number of three-dimensional characteristic tensors obtained in a first characteristic layer is equal to the number of convolution kernels of the layer, and a schematic diagram of the convolution operation is shown in FIG. 3;
9.3) down-sampling each three-dimensional feature tensor obtained in the first feature layer, and taking the three-dimensional tensor 2 of the feature layer3The maximum cloud pixel value in the region is used as the corresponding cloud pixel value in the down-sampling layer, the sampling step length is 2, and the down-sampling operation schematic diagram is shown in figure 4;
9.4) carrying out convolution operation on the three-dimensional tensors in the first down-sampling layer to obtain a second characteristic layer, wherein the number of the three-dimensional characteristic tensors in the second characteristic layer is equal to the number of convolution kernel cores of the layer;
9.5) down-sampling each three-dimensional feature tensor of the first feature layer to obtain a second down-sampling layer in the same step 9.3);
9.6) carrying out convolution operation on each three-dimensional feature tensor of the second down-sampling layer to obtain a one-dimensional feature vector which is used as the input of the fully-connected neural network;
9.7) obtaining a prediction output through the fully-connected neural network in the step 9.6), wherein a prediction output value and an actual value are both expressed by one-hot processed by softmax, and an error between an output result and an expected value is calculated by using a loss function;
9.8) when the error does not meet the requirement, carrying out a reverse propagation process, carrying out reverse transmission on the error, calculating the error of each layer, and then updating the weight and the offset value of the convolution kernel and the full connection layer by using a random gradient descent method;
9.9) repeating the steps 9.1) to 9.8) until the error precision requirement is met or the preset iteration times are reached, obtaining a trained three-dimensional convolution neural network model after the training is finished, and finishing the underwater target recognition task by using the model.
Alternatively, as a preferred scheme, the loss function J of the three-dimensional convolutional neural network in the model training module can be a cross entropy function
Wherein N is the number of training samples, K is the number of identification target types required by the task, PijThe j value, y, represented by the i-th data actual output value one-hotijThe j value represented by the ith data prediction output value one-hot is represented as the neural network output;
optionally, as a preferred scheme, the method further comprises:
10) m data are taken from point cloud data except the training set to form a test set I'test＝{D'1,D'2,···,D'MInputting the test sample set into a trained three-dimensional convolutional neural network in a model training module for testing, and calculating the recognition rate;
11) and evaluating the target recognition effect, finishing a target recognition task by using the model if the recognition accuracy meets the requirement, taking data from the database by using the data preprocessing module if the recognition accuracy does not meet the requirement, updating the training set, training the three-dimensional convolutional neural network model again, and updating the target recognition model.
As another preferable scheme, the method further comprises:
12) in the step 9) and the step 10), displaying the identification accuracy results of the model on the training set and the test set on a screen of an upper computer;
13) and displaying the result of the underwater object target identification on a screen of the upper computer.
The technical conception of the invention is as follows: the method comprises the steps of intelligently identifying underwater targets, preprocessing and coding point cloud data obtained by a laser radar, directly training a three-dimensional convolutional neural network by using the three-dimensional point cloud data, establishing an underwater target identification model, and obtaining a model meeting task requirements after evaluation to realize intelligent underwater target identification.
The invention has the following beneficial effects: 1. the laser radar is used for acquiring point cloud data, so that the speed is high, the precision is high, and the anti-interference capability is strong; 2. the model is trained by directly utilizing the three-dimensional point cloud data, the characteristics of the point cloud data are utilized to the maximum extent, and the identification accuracy is obviously improved; 3. the underwater target can be classified while being recognized, and the intelligent performance is strong.
Drawings
FIG. 1 is a functional block diagram of an upper computer of the identification instrument provided by the invention;
FIG. 2 is a diagram for explaining an example of a network structure of a three-dimensional convolutional neural network proposed by the present invention;
FIG. 3 is a schematic diagram of the convolution operation of the proposed three-dimensional convolutional neural network;
fig. 4 is a schematic diagram of the down-sampling operation of the three-dimensional convolutional neural network proposed in the present invention.
Detailed Description
The present invention will be described in detail below with reference to the accompanying drawings.
The laser radar, the database and the upper computer are connected in sequence. The recognizer recognizes underwater targets by adopting the following processes:
1) scanning underwater objects needing to be identified by using a laser radar to obtain a laser radar point cloud data set, wherein each point cloud data set is a set of vectors which take the laser radar as a coordinate origin and are in a three-dimensional coordinate system, and the vectors are expressed in the form of X, Y and Z three-dimensional coordinates, namely
Pij＝(x,y,z)
Wherein, PijA j point of the ith point cloud data;
2) storing point cloud data acquired by scanning an underwater object by using a laser radar into a database of the identification instrument, wherein the point cloud data in the database is used by an upper computer;
fig. 1 is a functional module schematic diagram of an upper computer of the identification instrument, the upper computer comprising:
the data preprocessing module 1 is used for preprocessing point cloud data in a database, and is completed by adopting the following processes:
3) acquiring N point cloud data from database to obtain training sample set
Itrain＝{D1,D2,···,DN}
Di＝{Pi,Oi}
Wherein D isiFor each sample used for training, PiFor the ith input point cloud data, OiThe output of the ith training sample represents the type of the recognized object;
4) performing geometric normalization processing on each point cloud data in the training sample set, wherein the geometric normalization is performed according to the following steps:
4.1) extracting the coordinate value X with the minimum numerical value in the X, Y and Z directions in the point cloud datamin，ymin，zminAnd the coordinate value x with the largest numerical valuemax，ymax，zmax；
4.2) mixing (x)min,ymin,zmin) As a new coordinate origin, the side length is taken in the X, Y and Z directions
HX＝(xmax-xmin)，HY＝(ymax-ymin)，HZ＝(zmax-zmin)
Obtaining a point cloud cuboid V which surrounds all points in the point cloud data;
4.3) obtaining new coordinate values of each point in the point cloud data by the following formula:
x'＝x-xmin,y'＝y-ymin,z'＝z-zmin
(x ', y ', z ') is the new coordinate value of the point cloud data which is obtained and is preprocessed by the preprocessing module;
the cloud element encoding module 2 is used for encoding the preprocessed point cloud data to obtain a three-dimensional tensor of the input neural network, and the process is completed by adopting the following steps:
5) regarding the cuboid V surrounding the point cloud data obtained in the step 4), the cuboid V is respectively arranged in the X direction, the Y direction and the Z direction
Is divided into intervals of HX，HY，HZThe length of the cuboid V in the X, Y and Z directions respectively, namely the cuboid V is divided into H3Small cuboids, each of which is called a cloud block, in this example, H is 18;
6) coding each cloud block, wherein the coding process comprises the following steps:
6.1) calculating the coordinate mean value of each point in the X, Y and Z directions for each cloud block:
wherein muX、μY、μZAre respectively the coordinate mean value, x 'of each point in the direction of X, Y, Z'j、y'jZ' j are new coordinate values pre-processed by the pre-processing module in direction X, Y, Z, T is the number of points in each cloud block, and then (μ) will beX,μY,μZ) As a block center;
6.2) sampling each cloud block, taking 100 points closest to the mass center, taking all points if less than 100 points exist in the cloud block, and calculating the Euclidean distance from each point to the block center:
summing the Euclidean distances of each point to the block center:
where ρ isjThe Euclidean distance from the jth point in the block to the block center, CiThe cloud element referred to as the i-th cloud chunk,Cithe size of the point cloud data is the cloud element value of the ith cloud block, and each point cloud data is converted into a point cloud data set H3The individual cloud elements represent three-dimensional tensors, and a training sample set which converts point cloud data into cloud elements is obtained;
the model training module 3 is used for training the three-dimensional convolutional neural network by using a training sample set represented by the encoded cloud element, and the method is completed by adopting the following processes:
7) a topological structure of a three-dimensional convolutional neural network is set, and FIG. 2 shows the three-dimensional convolutional neural network used in the present example, which is composed of two feature layers, two down-sampling layers and three full-connection layers. Optionally setting the size and the number of convolution kernels, setting an activation function and the number of nodes of each layer of the full connection layer, and setting each hyper-parameter of the neural network;
8) initializing the weight value and the bias value of the neural network, and in the embodiment, randomly initializing each weight value and bias value to a value with a value range of [ -1,1 ];
9) with training sample set I'train＝{D'1,D'2,···,D'NTrain convolutional neural network, where'trainIs a training sample set D 'processed by a data preprocessing module'1,D'2,···,D'NFor each training sample processed by the data preprocessing module in the training sample set, the training process is carried out according to the following steps:
9.1) inputting training sample set data into an input layer of the three-dimensional convolutional neural network, wherein the input data is 183The three-dimensional tensor of (a);
9.2) performing a convolution operation on the input data by a convolution kernel having a size of 3 in the first feature layer3The number of convolution kernels is 6, the convolution weighted sum and an offset term are added to obtain a value, the value is used for obtaining a corresponding cloud pixel value in the first characteristic layer through an optional ReLU activation function, the convolution step size is 1, and 6 values with the size of 16 are obtained in the first characteristic layer3The schematic diagram of the convolution operation is shown in fig. 3;
9.3) down-sampling each three-dimensional feature tensor obtained in the first feature layer, and taking the three-dimensional tensor 2 of the feature layer3The maximum cloud pixel value in the region is used as the corresponding cloud pixel value in the down-sampling layer, the sampling step length is 2, and the down-sampling operation schematic diagram is shown in FIG. 4;
9.4) convolution of the three-dimensional tensors in the first downsampling layer, the convolution kernel size in the second eigen layer also being 33The number of convolution kernels is 18, the activation function also adopts a ReLU activation function, and 18 sizes in the second feature layer are obtained and are 63The three-dimensional tensor of (a);
9.5) down-sampling each three-dimensional feature tensor of the first feature layer to obtain a second down-sampling layer in the same step 9.3);
9.6) performing convolution operation on the three-dimensional feature tensors of the second down-sampling layer, wherein the size of a convolution kernel is 33The number of convolution kernels is 120, and a one-dimensional feature vector is obtained and used as the input of the fully-connected neural network, and the input layer is the fully-connected layer 1 shown in fig. 2. Then, a full connection layer 2 (as a hidden layer) containing 80 neurons is passed, and a 1 × 10 output matrix is obtained by softmax operation on an output layer, namely, the model shown in the example can be used for identifying 10 underwater targets;
9.7) optionally taking a cross entropy function as a loss function J to calculate the error between the output result and the expected value, wherein the error calculation formula is
Wherein N is the number of training samples, K is the number of identification target types required by the task, PijThe j value, y, represented by the i-th data actual output value one-hotijThe j value represented by the ith data prediction output value one-hot is represented as the neural network output;
9.8) when the error does not meet the requirement, carrying out a back propagation process, carrying out back transmission on the error, calculating the error of each layer, and then updating the weight and offset value of the convolution kernel and the full-connection layer by using a random gradient descent method;
9.9) repeating the steps 9.1) to 9.8) until the error precision requirement is met or the preset iteration times are reached, obtaining a trained three-dimensional convolution neural network model after the training is finished, and finishing the underwater target recognition task by using the model.
The model updating module 4 is used for evaluating the effect of the three-dimensional neural network model obtained by the model training module, and the method is completed by adopting the following processes:
10) m data are taken from point cloud data except the training set to form a test set I'test＝{D'1,D'2,···,D'MInputting the test sample set into a trained three-dimensional convolutional neural network in a model training module for testing, and calculating the recognition rate;
11) and evaluating the target recognition effect, finishing a target recognition task by using the model if the recognition accuracy meets the requirement, taking data from the database by using the data preprocessing module if the recognition accuracy does not meet the requirement, updating the training set, training the three-dimensional convolutional neural network model again, and updating the target recognition model.
The result display module 5: the method is used for displaying the result of target identification on an upper computer and comprises the following steps:
12) in the step 9) and the step 10), displaying the identification accuracy results of the model on the training set and the test set on a screen of an upper computer;
13) and displaying the result of the target identification of the underwater object on a screen of the upper computer.
The hardware part of the upper computer consists of the following parts: the program memory is used for storing the implementation programs of all the modules; the data memory is used for storing data samples acquired by the laser radar and various parameters and hyper-parameters of the neural network; the arithmetic unit is used for executing the program and realizing corresponding functions; the I/O element is used for collecting data and transmitting information; and the display module is used for displaying the model training result and the target recognition result on the upper computer.
The above-described embodiments are intended to illustrate rather than to limit the invention, and any modifications and variations of the present invention are within the spirit of the invention and the scope of the appended claims.
Claims (5)
1. An intelligent underwater target recognition system based on laser radar point cloud data comprises a laser radar, a database and an upper computer, wherein the laser radar, the database and the upper computer are sequentially connected; the method is characterized in that: scanning an underwater object to be identified by a laser radar to obtain laser radar point cloud data, and then storing the obtained point cloud data in a database; the data stored in the database are used by an upper computer, and the upper computer comprises a data preprocessing module, a cloud element coding module, a model training module, a model updating module and a result display module which are sequentially connected; the data preprocessing module is used for preprocessing point cloud data in a database; the cloud element coding module is used for coding the point cloud data preprocessed by the data preprocessing module to obtain a three-dimensional tensor of the input neural network; the model training module trains a three-dimensional convolutional neural network by using a training sample set represented by the cloud element obtained after the encoding of the cloud element encoding module; the model updating module evaluates the effect of the three-dimensional convolutional neural network model obtained by the model training module; the result display module displays the result of the target identification on the upper computer; the data preprocessing module is used for preprocessing point cloud data in a database and comprises the following steps:
(3.1) acquiring N point cloud data in a database to obtain a training sample set:
Itrain＝{D1,D2,···,DN}
Di＝{Pi,Oi}
wherein D isiFor each sample used for training, PiFor the ith input point cloud data, OiThe output of the ith training sample represents the type of the recognized object;
(3.2) carrying out geometric normalization processing on each point cloud data in the training sample set, wherein the geometric normalization is carried out according to the following steps:
(3.2.1) extracting the coordinate value X with the minimum numerical value in the X, Y and Z directions in the point cloud datamin，ymin，zminAnd the coordinate value x with the largest numerical valuemax，ymax，zmax；
(3.2.2) mixing (x)min,ymin,zmin) As a new coordinate origin, the side length is taken in the X, Y and Z directions
HX＝(xmax-xmin)，HY＝(ymax-ymin)，HZ＝(zmax-zmin)
Obtaining a point cloud cuboid V which surrounds all points in the point cloud data;
(3.2.3) obtaining new coordinate values of each point in the point cloud data by the following formula:
x'＝x-xmin,y'＝y-ymin,z'＝z-zmin
(x ', y ', z ') is the new coordinate value of the point cloud data which is obtained and is preprocessed by the preprocessing module;
the cloud element coding module is used for coding the preprocessed point cloud data to obtain a three-dimensional tensor of the input neural network, and the process is completed by adopting the following steps:
(3.3) regarding the cuboid V surrounding the point cloud data obtained in the step (3.2), respectively pressing the cuboid V in the X, Y and Z directions
Is divided into intervals of HX，HY，HZThe length of the cuboid V in the X, Y and Z directions respectively, namely the cuboid V is divided into H3Small cuboids, each small cuboid being called a cloud block;
(3.4) encoding each cloud block, wherein the encoding process is carried out according to the following steps:
(3.4.1) for each cloud block, calculating the coordinate mean value of each point in the X, Y and Z directions:
wherein muX、μY、μZAre respectively the coordinate mean value, x 'of each point in the direction of X, Y, Z'j、y'j、z'jNew coordinate values pre-processed by the pre-processing module in the direction X, Y, Z, respectively, T is the number of points in each cloud block, and then (μ) will beX,μY,μZ) As a block center;
(3.4.2) sampling each cloud block, taking the T 'point nearest to the centroid, if there are less than T' points in the cloud block, taking all the points, and calculating the Euclidean distance from each point to the centroid:
summing the Euclidean distances of each point to the block center:
where ρ isjThe Euclidean distance from the jth point in the block to the block center, CiCloud element called i-th cloud piece, CiThe size of the point cloud data is the cloud element value of the ith cloud block, and each point cloud data is converted into a point cloud data set H3The individual cloud elements represent three-dimensional tensors, and a training sample set which converts the point cloud data into cloud elements is obtained.
2. The lidar point cloud data underwater target identification system of claim 1, wherein the lidar scans underwater objects to be identified to obtain lidar point cloud data by the steps of:
(2.1) scanning underwater objects to be identified by using the laser radar to obtain a laser radar point cloud data set, wherein each point cloud data set is a set of vectors which take the laser radar as a coordinate origin and are in a three-dimensional coordinate system, and the vectors are expressed in the form of X, Y and Z three-dimensional coordinates, namely
Pij＝(x,y,z)
Wherein, PijA j point of the ith point cloud data;
and (2.2) storing point cloud data acquired by scanning underwater objects by using a laser radar into a database of the identification system, wherein the point cloud data in the database is called by an upper computer.
3. The lidar point cloud data underwater target recognition system of claim 1, wherein the model training module is configured to train a three-dimensional convolutional neural network by using a training sample set represented by encoded cloud elements, and the training is performed by the following processes:
(4.1) setting a topological structure of a three-dimensional convolutional neural network, wherein the three-dimensional convolutional neural network consists of a characteristic layer, a down-sampling layer and a full connection layer, the size and the number of convolutional kernels need to be set, an activation function is set, the number of nodes of each layer of the full connection layer is set, and each hyper-parameter of the neural network is set;
(4.2) initializing a weight value and a bias value of the neural network;
(4.3) with training sample set I'train＝{D'1,D'2,···,D'NTrain convolutional neural network, where'trainIs a training sample set, D ', processed by a cloud coding module'1,D'2,···,D'NFor each training sample processed by the cloud coding module in the training sample set, the training process is carried out according to the following steps:
(4.3.1) inputting the training sample set data into an input layer of the three-dimensional convolutional neural network, wherein the input data is a three-dimensional tensor;
(4.3.2) performing convolution operation on input data through a convolution kernel, adding a convolution weighted sum and an offset term to obtain a value, obtaining a corresponding cloud pixel value in a characteristic layer through an activation function according to the value, wherein the convolution step length is 1, and the number of three-dimensional characteristic tensors obtained in a first characteristic layer is equal to the number of convolution kernel cores of the layer;
(4.3.3) down-sampling each three-dimensional feature tensor obtained in the first feature layer, and taking the three-dimensional tensor 2 of the first feature layer3The maximum cloud pixel value in the area is used as the corresponding cloud pixel value in the down-sampling layer, and the sampling step length is 2, so that a first down-sampling layer is obtained;
(4.3.4) performing convolution operation on the three-dimensional tensors in the first downsampling layer to obtain a second feature layer, wherein the number of the three-dimensional feature tensors in the second feature layer is equal to the number of convolution kernel cores in the layer;
(4.3.5) downsampling each three-dimensional feature tensor of the second feature layer, and taking the three-dimensional tensor 2 of the second feature layer3The maximum cloud pixel value in the area is used as the corresponding cloud pixel value in the down-sampling layer, the sampling step length is 2, and a second down-sampling layer is obtained;
(4.3.6) performing convolution operation on each three-dimensional feature tensor of the second down-sampling layer to obtain a one-dimensional feature vector which is used as the input of the fully-connected neural network;
(4.3.7 obtaining a predicted output through the fully-connected neural network described in the step (4.3.6), wherein the predicted output value and the actual value are both expressed by one-hot processed by softmax, and the error between the output result and the expected value is calculated by using a loss function;
(4.3.8) when the error does not meet the requirement, carrying out a back propagation process, carrying out back transmission on the error, calculating the error of each layer, and then carrying out updating on the weight and the offset value of the convolution kernel and the full connection layer by using a random gradient descent method;
and (4.3.9) repeating the iteration of the steps (4.3.1) - (4.3.8) until the error precision requirement is met or the preset iteration number is reached, and obtaining a trained three-dimensional convolution neural network model after the training is finished.
4. The lidar point cloud data underwater target recognition system of claim 3, wherein the loss function J of the three-dimensional convolutional neural network in the model training module is a cross entropy function:
wherein N is the number of training samples, K is the number of identification target types required by the task, PijThe j value, y, represented by the i-th data actual output value one-hotijAnd the output of the neural network represents the j value represented by the ith data prediction output value one-hot.
5. The lidar point cloud data underwater target identification system of claim 1, wherein the upper computer further comprises: the model updating module is used for evaluating the effect of the three-dimensional neural network model obtained by the model training module and is completed by adopting the following processes:
(6.1) taking M data from point cloud data except the training set to form a test set I'test＝{D'1,D'2,···,D'MInputting the test sample set into a trained three-dimensional convolutional neural network in a model training module for testing, and calculating the recognition rate;
and (6.2) evaluating the target recognition effect, finishing a target recognition task by using the model if the recognition accuracy meets the requirement, and enabling the data preprocessing module to take data from the database again if the recognition accuracy does not meet the requirement, updating the training set and training the three-dimensional convolutional neural network model again.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110023522.3A CN112949380B (en) | 2021-01-08 | 2021-01-08 | Intelligent underwater target identification system based on laser radar point cloud data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110023522.3A CN112949380B (en) | 2021-01-08 | 2021-01-08 | Intelligent underwater target identification system based on laser radar point cloud data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112949380A CN112949380A (en) | 2021-06-11 |
| CN112949380B true CN112949380B (en) | 2022-06-10 |
Family
ID=76235112
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110023522.3A Expired - Fee Related CN112949380B (en) | 2021-01-08 | 2021-01-08 | Intelligent underwater target identification system based on laser radar point cloud data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112949380B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113870359A (en) * | 2021-09-23 | 2021-12-31 | 厦门大学 | Laser radar external parameter calibration method and device based on random gradient descent |
| CN114545400B (en) * | 2022-04-27 | 2022-08-05 | 陕西欧卡电子智能科技有限公司 | Global repositioning method of water surface robot based on millimeter wave radar |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106600622A (en) * | 2016-12-06 | 2017-04-26 | 西安电子科技大学 | Point cloud data partitioning method based on hyper voxels |
| CN109426828A (en) * | 2017-08-28 | 2019-03-05 | 中国科学院深圳先进技术研究院 | Point cloud classifications method, apparatus, equipment and storage medium |
| CN109816714A (en) * | 2019-01-15 | 2019-05-28 | 西北大学 | A kind of point cloud object type recognition methods based on Three dimensional convolution neural network |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP3525000B1 (en) * | 2018-02-09 | 2021-07-21 | Bayerische Motoren Werke Aktiengesellschaft | Methods and apparatuses for object detection in a scene based on lidar data and radar data of the scene |
-
2021
- 2021-01-08 CN CN202110023522.3A patent/CN112949380B/en not_active Expired - Fee Related
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106600622A (en) * | 2016-12-06 | 2017-04-26 | 西安电子科技大学 | Point cloud data partitioning method based on hyper voxels |
| CN109426828A (en) * | 2017-08-28 | 2019-03-05 | 中国科学院深圳先进技术研究院 | Point cloud classifications method, apparatus, equipment and storage medium |
| CN109816714A (en) * | 2019-01-15 | 2019-05-28 | 西北大学 | A kind of point cloud object type recognition methods based on Three dimensional convolution neural network |
Non-Patent Citations (2)
| Title |
|---|
| 《Point-Voxel CNN for Efficient 3D Deep Learning》;Zhijian Liu，et al;《NeurIPS 2019》;20191231;第1-11页 * |
| 《基于激光雷达的无人驾驶障碍物检测与辨识》;谭勇;《中国优秀博硕士学位论文全文数据库(硕士)工程科技Ⅱ辑》;20200815(第08期);第C035-403页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112949380A (en) | 2021-06-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110472627B (en) | End-to-end SAR image recognition method, device and storage medium | |
| CN110210463B (en) | Precise ROI-fast R-CNN-based radar target image detection method | |
| CN109871902B (en) | SAR small sample identification method based on super-resolution countermeasure generation cascade network | |
| US9639748B2 (en) | Method for detecting persons using 1D depths and 2D texture | |
| CN108052942B (en) | Visual image recognition method for aircraft flight attitude | |
| Fayaz et al. | Underwater object detection: architectures and algorithms–a comprehensive review | |
| CN112149591B (en) | SSD-AEFF automatic bridge detection method and system for SAR image | |
| CN112949380B (en) | Intelligent underwater target identification system based on laser radar point cloud data | |
| CN103353988A (en) | Method for evaluating performance of heterogeneous SAR (synthetic aperture radar) image feature matching algorithm | |
| CN115049821A (en) | Three-dimensional environment target detection method based on multi-sensor fusion | |
| CN116486287A (en) | Target detection method and system based on environment self-adaptive robot vision system | |
| CN112487900A (en) | SAR image ship target detection method based on feature fusion | |
| Yu et al. | Dual-branch framework: AUV-based target recognition method for marine survey | |
| CN116630937A (en) | Multimode fusion 3D target detection method | |
| CN114627183A (en) | Laser point cloud 3D target detection method | |
| CN114565753A (en) | Unmanned aerial vehicle small target identification method based on improved YOLOv4 network | |
| Wang et al. | Underwater object detection by fusing features from different representations of sonar data | |
| CN117523428B (en) | Ground target detection method and device based on aircraft platform | |
| CN114283296B (en) | Target identification self-evaluation method based on structural similarity | |
| CN117173549B (en) | Multi-scale target detection method and system for synthetic aperture sonar image under complex scene | |
| CN112926383B (en) | Automatic target identification system based on underwater laser image | |
| CN112906458B (en) | Group intelligent optimized underwater laser multi-target end-to-end automatic identification system | |
| Farooq et al. | A Review of Monocular Depth Estimation Methods Based on Deep Learning | |
| You et al. | Small Target Detection Optimization Algorithm Based on Improved YOLOv5 | |
| Sun et al. | Infrared Small-Target Detection Based on Multi-level Local Contrast Measure |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20220610 |