CN117830399B - Positioning method and device in autonomous docking process of underwater vehicle - Google Patents
Positioning method and device in autonomous docking process of underwater vehicle Download PDFInfo
- Publication number
- CN117830399B CN117830399B CN202311725714.4A CN202311725714A CN117830399B CN 117830399 B CN117830399 B CN 117830399B CN 202311725714 A CN202311725714 A CN 202311725714A CN 117830399 B CN117830399 B CN 117830399B
- Authority
- CN
- China
- Prior art keywords
- feature
- matching
- local
- global
- features
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 122
- 230000008569 process Effects 0.000 title claims abstract description 76
- 238000003032 molecular docking Methods 0.000 title claims abstract description 60
- 238000001514 detection method Methods 0.000 claims abstract description 72
- 239000013598 vector Substances 0.000 claims abstract description 49
- 238000013527 convolutional neural network Methods 0.000 claims abstract description 28
- 239000011159 matrix material Substances 0.000 claims abstract description 25
- 238000005259 measurement Methods 0.000 claims abstract description 10
- 238000006243 chemical reaction Methods 0.000 claims abstract description 6
- 238000012549 training Methods 0.000 claims description 12
- 238000003860 storage Methods 0.000 claims description 10
- 238000004590 computer program Methods 0.000 claims description 8
- 230000006870 function Effects 0.000 claims description 8
- 230000009466 transformation Effects 0.000 claims description 3
- 238000010586 diagram Methods 0.000 description 9
- 238000004891 communication Methods 0.000 description 8
- 230000007613 environmental effect Effects 0.000 description 8
- 238000012360 testing method Methods 0.000 description 6
- 238000012545 processing Methods 0.000 description 5
- 238000001914 filtration Methods 0.000 description 3
- 238000011084 recovery Methods 0.000 description 3
- 238000013461 design Methods 0.000 description 2
- 239000007787 solid Substances 0.000 description 2
- 230000005856 abnormality Effects 0.000 description 1
- 238000010521 absorption reaction Methods 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000005520 cutting process Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 238000007667 floating Methods 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 210000001503 joint Anatomy 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000011524 similarity measure Methods 0.000 description 1
- 230000001502 supplementing effect Effects 0.000 description 1
- 230000009897 systematic effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S15/00—Systems using the reflection or reradiation of acoustic waves, e.g. sonar systems
- G01S15/88—Sonar systems specially adapted for specific applications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S15/00—Systems using the reflection or reradiation of acoustic waves, e.g. sonar systems
- G01S15/02—Systems using the reflection or reradiation of acoustic waves, e.g. sonar systems using reflection of acoustic waves
- G01S15/06—Systems determining the position data of a target
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0464—Convolutional networks [CNN, ConvNet]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
- G06V10/757—Matching configurations of points or features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/761—Proximity, similarity or dissimilarity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Remote Sensing (AREA)
- Radar, Positioning & Navigation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Medical Informatics (AREA)
- Databases & Information Systems (AREA)
- Computer Networks & Wireless Communication (AREA)
- Acoustics & Sound (AREA)
- Multimedia (AREA)
- Biomedical Technology (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Biophysics (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Molecular Biology (AREA)
- Measurement Of Velocity Or Position Using Acoustic Or Ultrasonic Waves (AREA)
Abstract
The invention provides a positioning method and a positioning device in an autonomous docking process of an underwater vehicle, wherein the method comprises the following steps: inputting the sound image acquired by the multi-beam sonar into a target detection depth network, and acquiring a first output result, wherein the first output result comprises a boundary frame coordinate set and a feature confidence coefficient set which are respectively corresponding to global features and local features; based on a feature matching reference threshold, matching the global feature and the local feature to obtain a first matching result, wherein the first matching result comprises a boundary frame coordinate set of the global feature and the matched local feature; tracking the first matching result by using a Kalman filter to obtain a position prediction result; inputting the first matching result into a convolutional neural network, and determining a feature vector; and constructing a correlation measurement weight matrix, matching the feature confidence of the tracking feature by combining with the Hungary algorithm, selecting global features and local features, performing coordinate conversion based on the image resolution of the multi-beam sonar, and determining the positioning result of the underwater vehicle in the autonomous docking process.
Description
Technical Field
The invention belongs to the field of automatic control of aircrafts, and particularly relates to a positioning method and device in an autonomous docking process of an underwater vehicle.
Background
The underwater vehicle plays a vital role in the operations of marine structure maintenance, submarine surveying and mapping and the like, and has high requirements on the continuous operation capability of the underwater vehicle in most scenes. Autonomous docking is an important technical means for improving the continuous operation capability of an underwater vehicle, and long-time long-voyage Cheng Zuoye of the underwater vehicle under the limited reserve energy can be realized by temporarily supplementing energy and exchanging data.
The positioning precision and the positioning continuity of the underwater vehicle in the autonomous docking process directly influence the control precision of the underwater vehicle, thereby influencing the high efficiency of task execution and the task completion amount. The existing autonomous docking technology faces the difficulties of strong underwater environment signal absorption, low visibility, complex acoustic conditions and the like, so that the positioning precision of the underwater vehicle is difficult to guarantee, and the positioning result is easy to be interfered by environment noise. Therefore, how to ensure positioning accuracy and robustness in autonomous docking of an underwater vehicle is a problem to be solved.
Disclosure of Invention
Aiming at the defects of the prior art, the invention aims to improve the positioning precision and the continuous positioning capability in the autonomous docking process of the underwater vehicle and aims to solve the problem that the positioning precision and the robustness in the autonomous docking process of the current underwater vehicle are difficult to guarantee.
In order to achieve the above purpose, the invention provides a positioning method and a positioning device in the autonomous docking process of an underwater vehicle.
In a first aspect, the present invention provides a positioning method in an autonomous docking process of an underwater vehicle, including:
acquiring an acoustic image in the autonomous docking process of the underwater vehicle based on multi-beam sonar;
inputting the sound image to a target detection depth network to obtain a first output result; the first output result comprises a boundary frame coordinate set and a feature confidence coefficient set which correspond to the global feature and the local feature respectively, and the target detection depth network is obtained by offline training based on a sample sound image and boundary frame coordinate labels of the global feature and the local feature which are determined in advance;
based on the first output result and a predetermined feature matching reference threshold, matching the global feature with the local feature to obtain a first matching result; the first matching result comprises a boundary frame coordinate set of the global feature and the matched local feature;
tracking the first matching result by using a Kalman filter to obtain a position prediction result of a tracking feature; inputting the first matching result into a convolutional neural network, and determining a feature vector;
determining a motion matching degree based on the first matching result and the position prediction result; determining a feature matching degree based on the feature vector extracted by the convolutional neural network; constructing an associated measurement weight matrix based on the motion matching degree and the feature matching degree;
based on the association metric weight matrix and the Hungary algorithm, matching the first matching result with the position prediction result, and matching the feature confidence coefficient of the tracking feature;
selecting global features and local features based on feature confidence of the tracking features;
And based on the image resolution of the multi-beam sonar, performing coordinate transformation on the selected global features and local features, and determining a positioning result of the underwater vehicle in the autonomous docking process.
In some embodiments, the feature matching reference threshold is determined based on the steps of:
determining the average feature ratio of local features in the global features in the sample sound image graph and the average feature cross ratio of the local features and the global features;
Determining the average feature duty cycle and the average feature cross-over ratio as the feature matching reference threshold.
In some embodiments, said matching said global feature and said local feature comprises:
traversing the boundary frame coordinate set of the global feature through outer circulation, and traversing the boundary frame coordinate set of the local feature through inner circulation;
And in the inner loop traversal process corresponding to the current outer loop, if the feature ratio of the local feature in the global feature corresponding to the current outer loop is larger than or equal to the average feature ratio, or the feature cross ratio of the local feature and the global feature corresponding to the current outer loop is larger than or equal to the average feature cross ratio, determining that the corresponding local feature is the matched local feature.
In some embodiments, the selected global features and local features include:
determining a final confidence value of the tracking feature based on the feature confidence of the tracking feature calibration and a predetermined additional confidence;
And selecting the global feature with the highest confidence coefficient final value and the matched local feature.
In some embodiments, the additional confidence is determined based on a motion path of the tracked feature in the sound image map over the tracking period, satisfying:
Wherein Δδ represents the additional confidence, k represents the harmonic coefficient, e t represents the exponential function of the time step t, d t represents the image distance from the central position of the tracked feature to the multi-beam sonar-base coordinate system at the time step t, and d 0 represents the image distance from the central position of the tracked feature to the multi-beam sonar-base coordinate system at the time of first tracking to the tracked feature.
In some embodiments, the determining a degree of motion matching based on the first matching result and the position prediction result includes:
Determining a mahalanobis distance between the first matching result and the position prediction result, the mahalanobis distance being used to characterize the degree of motion matching.
In some embodiments, the determining the feature matching degree based on the feature vector extracted by the convolutional neural network includes:
The feature vector comprises a first feature vector corresponding to a global feature and a second feature vector corresponding to a local feature, and a cosine distance between the first feature vector and the second feature vector is determined and used for representing the feature matching degree.
In a second aspect, the present invention provides a positioning device in an autonomous docking process of an underwater vehicle, comprising:
the image acquisition unit is used for acquiring an acoustic image in the autonomous docking process of the underwater vehicle based on the multi-beam sonar;
The target detection unit is used for inputting the sound image to a target detection depth network and obtaining a first output result; the first output result comprises a boundary frame coordinate set and a feature confidence coefficient set which correspond to the global feature and the local feature respectively, and the target detection depth network is obtained by offline training based on a sample sound image and boundary frame coordinate labels of the global feature and the local feature which are determined in advance;
the feature matching unit is used for matching the global feature with the local feature based on the first output result and a predetermined feature matching reference threshold value to obtain a first matching result; the first matching result comprises a boundary frame coordinate set of the global feature and the matched local feature;
The characteristic tracking unit is used for tracking the first matching result by using a Kalman filter to acquire a position prediction result of a tracking characteristic; inputting the first matching result into a convolutional neural network, and determining a feature vector; determining a motion matching degree based on the first matching result and the position prediction result; determining a feature matching degree based on the feature vector extracted by the convolutional neural network; constructing an associated measurement weight matrix based on the motion matching degree and the feature matching degree; based on the association metric weight matrix and the Hungary algorithm, matching the first matching result with the position prediction result, and matching the feature confidence coefficient of the tracking feature; selecting global features and local features based on feature confidence of the tracking features;
And the positioning acquisition unit is used for carrying out coordinate conversion on the selected global features and local features based on the image resolution of the multi-beam sonar, and determining the positioning result of the underwater vehicle in the autonomous docking process.
In a third aspect, the present invention provides an industrial personal computer, including: at least one memory for storing a program; at least one processor for executing a memory-stored program, which when executed is adapted to carry out the method described in the first aspect or any one of the possible implementations of the first aspect.
In a fourth aspect, the present invention provides a computer readable storage medium storing a computer program which, when run on a processor, causes the processor to perform the method described in the first aspect or any one of the possible implementations of the first aspect.
In a fifth aspect, the invention provides a computer program product which, when run on a processor, causes the processor to perform the method described in the first aspect or any one of the possible implementations of the first aspect.
It will be appreciated that the advantages of the second to fifth aspects may be found in the relevant description of the first aspect, and are not described here again.
In general, the above technical solutions conceived by the present invention have the following beneficial effects compared with the prior art:
According to the positioning method and device in the autonomous docking process of the underwater vehicle, noise characteristics are filtered through target detection, characteristic matching and characteristic tracking, the characteristics of the underwater vehicle are tracked in real time, and positioning accuracy and positioning continuity are guaranteed; the positioning of the underwater vehicle is realized by combining the global features and the local features of the underwater vehicle, so that the positioning precision is improved; the feature matching reference threshold is counted by utilizing the sample sound image, and the feature matching reference threshold is utilized to perform preliminary filtering of false detection features in the feature matching process on the basis of target detection, so that the influence of environmental noise on a positioning result is reduced to a certain extent; and the Kalman filter and the depth correlation measurement are combined to track the global features and the local features after feature matching respectively, the additional confidence is designed to reduce the confidence of noise features of the false motion trend, the confidence of real features is improved, and the positioning accuracy and the robustness are further improved.
Drawings
In order to more clearly illustrate the invention or the technical solutions of the prior art, the following description will briefly explain the drawings used in the embodiments or the description of the prior art, and it is obvious that the drawings in the following description are some embodiments of the invention, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.
Fig. 1 is a schematic view of an autonomous docking scenario of an underwater vehicle according to an embodiment of the present invention;
FIG. 2 is a schematic flow chart of a positioning method in autonomous docking of an underwater vehicle according to an embodiment of the present invention;
FIG. 3 is a second flow chart of a positioning method in an autonomous docking process of an underwater vehicle according to an embodiment of the present invention;
FIG. 4 is a schematic diagram illustrating the identification of a target detection layer in a real boat docking test process according to an embodiment of the present invention;
FIG. 5 is a schematic diagram illustrating the identification of a feature tracking layer in a real boat docking test process according to an embodiment of the present invention;
Fig. 6 is a schematic structural diagram of a positioning device in an autonomous docking process of an underwater vehicle according to an embodiment of the present invention;
Fig. 7 is a schematic structural diagram of an industrial personal computer according to an embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present invention more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
Next, the technical solutions provided in the embodiments of the present invention will be described with reference to fig. 1 to 7.
Fig. 1 is a schematic view of an autonomous docking scenario of an underwater vehicle according to an embodiment of the present invention, and as shown in fig. 1, an autonomous docking test platform of an underwater vehicle based on underwater acoustic communication and multi-beam sonar positioning is shown. In the test platform, the recovery device is fixed below the towing platform, and the carrying industrial personal computer on the platform is directly connected with the multi-beam sonar fixed on the recovery device, so that the positioning method in the autonomous docking process of the underwater vehicle is realized, and the positioning result is sent to the on-board controller of the underwater vehicle through underwater acoustic communication to complete docking control.
It is easy to think that besides the positioning of the autonomous docking process of the underwater vehicle, the positioning method provided by the embodiment of the invention can be further expanded to other operation tasks for positioning the underwater vehicle based on multi-beam sonar.
Fig. 2 is one of flow diagrams of a positioning method in an autonomous docking process of an underwater vehicle according to an embodiment of the present invention, and as shown in fig. 2, an execution main body of the method is an industrial personal computer, and at least includes the following steps (Step):
s201, acquiring a sound image in the autonomous docking process of the underwater vehicle based on multi-beam sonar.
Specifically, the positioning method in the autonomous docking process of the underwater vehicle provided by the invention can be roughly divided into five steps of image acquisition, target detection, feature matching, feature tracking and positioning acquisition.
In the image acquisition process, the multi-beam sonar is deployed on a recovery device, and a multi-beam sonar image map in the autonomous docking process of the underwater vehicle is acquired. The sound image comprises global characteristics and local characteristics of the underwater vehicle. The global features mainly refer to hull features, and the local features can be selected as bow features, stern features and the like aiming at the autonomous docking features of the underwater vehicle. Hereinafter, the hull feature is taken as a global feature, and the bow feature is taken as a local feature.
S202, inputting an acoustic image to a target detection depth network to obtain a first output result; the first output result comprises a boundary frame coordinate set and a feature confidence coefficient set which correspond to the global feature and the local feature respectively, and the target detection depth network is obtained through offline training based on a sample sound image and boundary frame coordinate labels of the global feature and the local feature which are determined in advance.
Specifically, in the target detection process, the sound image is input to a target detection depth network, and a first output result is obtained. The input of the target detection depth network is a complete multi-beam sonar image graph, and the output, namely a first output result, comprises: and classifying the global features and the local features of the identified underwater vehicle, and collecting the boundary frame coordinates of the global features and the local features and the corresponding feature confidence coefficient sets. The confidence of the obtained features is low, and the false detection features need to be further filtered.
The target detection depth network is trained offline in advance, and the specific training process comprises the following steps: based on the acquired sample acoustic image, calibrating global features and local features of the underwater vehicle in the historical acoustic image to form a data set S, and training a target detection depth network-deconvolution network (Deconvolutional Networks, DN) based on the data set S. The network input of the target detection depth network is a complete multi-beam sonar image graph, and the network output is the recognized boundary frame coordinates of the ship body characteristics and the ship bow characteristics of the underwater vehicle and the corresponding characteristic confidence coefficient set.
Meanwhile, in the target detection process, global features and local features of the underwater vehicle are considered at the same time, and positioning in the autonomous docking process of the underwater vehicle is realized by the global features or the local features instead of the global features or the local features singly, so that the positioning accuracy and the robustness are better.
S203, matching the global feature with the local feature based on the first output result and a predetermined feature matching reference threshold value to obtain a first matching result; the first matching result includes a set of bounding box coordinates of the global feature and the matched local feature.
Specifically, in the feature matching process, feature matching is performed by using the potential relationship between features on the basis of target detection. And specifically, matching the global features and the local features obtained by target detection by combining a predetermined feature matching reference threshold value, and outputting a first matching result meeting the condition. The first matching result includes a set of bounding box coordinates of the global feature and the matched local feature.
For global features, the probability of false detection is considered to be low; whereas for local features the probability of false detection is considered to be higher. In the process of feature matching, the false detection features in the local features are primarily filtered by utilizing a predetermined feature matching reference threshold.
S204, tracking the first matching result by using a Kalman filter to obtain a position prediction result of the tracking feature; and inputting the first matching result into a convolutional neural network to determine a feature vector.
S205, determining a motion matching degree based on the first matching result and the position prediction result; determining feature matching degree based on feature vectors extracted by the convolutional neural network; and constructing an association metric weight matrix based on the motion matching degree and the feature matching degree.
S206, matching the first matching result and the position prediction result based on the association metric weight matrix and the Hungary algorithm, and matching the feature confidence of the tracking feature.
S207, selecting global features and local features based on the feature confidence of the tracking features.
Specifically, in the feature tracking process, a first matching result obtained in the feature matching process is tracked by using a Kalman filter, and a position prediction result of each tracking feature is obtained. The tracking feature is a global feature or a local feature in the first matching result. And then determining the motion matching degree by using the first matching result of the feature matching and the position prediction result obtained by the Kalman filter.
Further, the first matching result is input to a trained convolutional neural network (Convolutional Neural Networks, CNN), feature vectors are extracted, including a first feature vector of global features and a second feature vector of local features.
The convolutional neural network is trained offline in advance, and the training process comprises the following steps: based on the collected historical sound image, calibrating global features and local features of the underwater vehicle in the historical sound image to form a data set S, locally cutting the sound image in the data set S, respectively obtaining local images containing the global features and the local features, and manufacturing corresponding labels to distinguish the local images to form the data set S body and the data set S head. Convolutional neural networks CN body and CN head are trained based on data sets S body and S head, respectively. The network input of the convolutional neural network is a global feature local image or a local feature local image of the underwater vehicle. The network output is a feature vector of input global features or local features.
And obtaining a global feature vector and a local feature vector according to the output of the convolutional neural network, thereby determining the feature matching degree. The steps of extracting the feature vector and obtaining the position prediction result are not limited to the sequence, and can be synchronously performed.
Further, an associated metric weight matrix is constructed according to the motion matching degree and the feature matching degree. And matching a first matching result obtained by characteristic matching and a position prediction result obtained by tracking by a Kalman filter based on the correlation metric weight matrix by using a Hungary algorithm, so as to match the characteristic confidence of each tracking characteristic. The feature confidence of each tracking feature is contained in the feature confidence set output by the target detection depth network. Optionally, each tracking feature is marked with an identification number and a corresponding feature confidence.
And selecting global features and local features as the global features and the local features which are finally acquired in the current frame sound image or the current time interval according to the feature confidence of each matched tracking feature.
S208, based on the image resolution of the multi-beam sonar, coordinate conversion is carried out on the selected global features and the local features, and the positioning result of the underwater vehicle in the autonomous docking process is determined.
Specifically, after the global features and the local features are selected, based on the image resolution of the multi-beam sonar, the selected global features and the local features are subjected to coordinate conversion, and the specific operation of the coordinate conversion is to convert the central coordinates of the selected features under the image coordinates into the actual relative distances of the underwater vehicle relative to the multi-beam sonar, so that the positioning of the underwater vehicle in the autonomous docking process is completed.
According to the positioning method in the autonomous docking process of the underwater vehicle, noise characteristics are filtered through three processes of target detection, characteristic matching and characteristic tracking, and the characteristics of the underwater vehicle are tracked in real time, and global characteristics and local images in an acoustic image are extracted through a target detection depth network; then setting a feature matching reference threshold on the basis of target detection, and performing feature matching and preliminary filtering of false detection features; and finally, respectively determining the motion matching degree and the feature matching degree through a Kalman filter and a convolutional neural network in the feature tracking process, determining an associated measurement weight matrix according to the motion matching degree and the feature matching degree, selecting global features and local features by using a Hungary algorithm and the associated measurement weight matrix, completing the extraction of the global features and the local features in the acoustic image map, and finally, completing the positioning of the underwater vehicle in the autonomous docking process by using the selected global features and the local features and combining the image resolution of the multi-beam sonar.
In some embodiments, the feature matching reference threshold is determined based on the steps of:
Determining the average feature ratio of local features in global features in a sample sound image graph and the average feature cross ratio of the local features and the global features;
And determining the average characteristic occupation ratio and the average characteristic crossing ratio as characteristic matching reference threshold values.
Specifically, the global features and the local features output by the target detection depth network are initially screened by utilizing a predetermined feature matching reference threshold value, and the local features with obvious abnormality are filtered. The feature matching reference threshold value can be obtained by training a sample sound image graph of a target detection depth network for statistics, specifically, the average feature duty ratio of the local feature obtained by statistics in the global feature and the average feature cross ratio of the local feature and the global feature.
Taking global features as hull features and local features as bow features as examples, counting the average feature ratio of the bow features in the hull features in the acoustic image mapThe method meets the following conditions:
average characteristic crossing ratio of hull characteristic and bow characteristic in statistical sound image The method meets the following conditions:
wherein N is the number of sound image graphs of the relevant characteristics of the underwater vehicle captured in the training data set S of the target detection depth network, For characteristic boundary frame coordinates of the bow of the boatIs used for the image area of the (c),Boundary frame coordinates characteristic of a hullIs a picture area of the picture frame.
In some embodiments, matching the global feature with the local feature in S203 includes:
traversing the boundary frame coordinate set of the global feature through the outer loop, and traversing the boundary frame coordinate set of the local feature through the inner loop;
And in the inner loop traversal process corresponding to the current outer loop, if the feature ratio of the local feature in the global feature corresponding to the current outer loop is larger than or equal to the average feature ratio, or the feature cross ratio of the local feature and the global feature corresponding to the current outer loop is larger than or equal to the average feature cross ratio, determining that the corresponding local feature is the matched local feature.
Specifically, for a plurality of local features and a plurality of global features, feature matching is required, and local features matched with the global features are determined according to the feature bit shape relation. Meanwhile, considering the influence of environmental noise, the local features of the target detection output have more false detection features, so that the false detection features need to be filtered. In the feature matching process, the false detection features are primarily filtered through a predetermined feature matching reference threshold.
Meanwhile, in the process of feature matching, the invention designs a double-circulation flow of external circulation and internal circulation. On the basis of target detection, traversing the boundary frame coordinate set of the global feature through outer circulation, and traversing the boundary frame coordinate set of the local feature through inner circulation.
And in the inner loop traversal process corresponding to each outer loop, calculating the feature ratio of each local feature in the global feature corresponding to the current outer loop, and comparing the feature cross ratio of each local feature and the feature cross ratio with a feature matching reference threshold value to determine whether the local feature is a false detection feature.
Specifically, the outer circulation traverses the boundary frame coordinate set { bbox body } of the hull feature of the underwater vehicle output by the target detection depth network, and the inner circulation traverses the boundary frame coordinate set { bbox head } of the hull feature;
Extracting boundary frame coordinates bbox body of the hull features and boundary frame coordinates bbox head of the bow features corresponding to the current outer circulation in the inner circulation traversal process, and calculating the bow feature ratio:
Then, calculating the characteristic crossing ratio of the bow characteristic and the hull characteristic:
average characteristic ratio of lambda 1 to boat bow characteristic Compare and compare lambda 2 to the average feature cross ratio of the boat bow and hull featuresAnd (5) comparing. If it isAnd is also provided withJudging the boundary frame coordinate bbox head of the current boat bow characteristic as the false detection characteristic and entering the next outer layer circulation processing, if the condition is not satisfied, namelyOr (b)Then bbox head is added to the matched boat bow feature set bbox head}f.
After the double-layer traversal is completed, the feature matching outputs a boundary frame coordinate set { bbox body } of the hull feature of the underwater vehicle and a boundary frame coordinate set { bbox head}f of the matched hull feature.
According to the positioning method in the autonomous docking process of the underwater vehicle, the global features and the local features of the underwater vehicle are matched through internal and external circulation, so that the fact that the global features have the local features matched with the global features is ensured, meanwhile, in the feature matching process, false detection features in the local features are primarily filtered by means of the feature matching reference threshold, and the influence of environmental noise on positioning results is reduced to a certain extent.
In some embodiments, determining the motion matching degree in S205 based on the first matching result and the position prediction result specifically includes:
a mahalanobis distance between the first matching result and the position prediction result is determined, the mahalanobis distance being used to characterize the degree of motion matching.
Specifically, a Kalman filter is adopted in the feature tracking process to track a first matching result output by feature matching, namely a boundary frame coordinate set of the global feature and the matched local feature, so that a position prediction result of each tracking feature is obtained. The tracking state includes the execution position, width, and aspect ratio of the bounding box of the tracking feature.
The motion matching degree can be expressed by using the mahalanobis distance between the actual detection result and the prediction result obtained by the Kalman filtering. According to the method, the first matching result of feature matching output and the position prediction result obtained by tracking through a Kalman filter are combined, and the motion matching degree of a tracking boundary box of the tracking feature and the motion matching degree of a prediction boundary box are determined.
Specifically, the feature tracking adopts a Kalman filter to track a boundary frame coordinate set { bbox body } of the hull feature output by feature matching and a boundary frame coordinate set { bbox head}f of the matched hull bow feature, and the tracking state comprises the center position, the width and the aspect ratio of the feature boundary frame;
The position prediction result of the Kalman filter on the tracking feature, namely the position prediction result of the boundary frame coordinates, is respectively And (3) withExtracting feature center coordinates p and of a first matching result of feature matching and a position prediction result of a Kalman filterCalculating the mahalanobis distanceThe method meets the following conditions:
wherein, sigma is the covariance matrix of the high-dimensional random variable.
In addition to mahalanobis distance, the degree of motion matching may also be measured by euclidean distance, manhattan distance, hamming distance, etc. The mahalanobis distance may well exclude interference of the correlation between variables.
In some embodiments, determining the feature matching degree based on the feature vector extracted by the convolutional neural network in S205 includes:
The feature vector comprises a first feature vector corresponding to the global feature and a second feature vector corresponding to the local feature, and the cosine distance between the first feature vector and the second feature vector is determined and used for representing the feature matching degree.
Specifically, the cosine distance between feature vectors may be used to represent the feature matching degree. And calculating the cosine distance between the first feature vector of the global feature and the second feature vector of the local feature output by the convolutional neural network as the feature matching degree.
Specifically, the first matching result of the feature matching is input into trained convolutional neural networks CN body and CN head, corresponding feature vectors x f and x kf are extracted, and cosine distance D cos(xf,xkf) is calculated, where:
In addition to cosine distance, feature matching degree can be measured by adjusting cosine similarity, pearson correlation coefficient, jaccard similarity coefficient and the like. The similarity measure of cosine distance is not affected by the index scale.
In some embodiments, constructing an association metric weight matrix based on the motion matching degree and the feature matching degree in S205 includes:
superimposed mahalanobis distance And cosine distance D cos(xf,xkf), and constructing an association metric weight matrix.
Specifically, the correlation metric weight matrix is determined by the motion matching degree, the characteristic matching degree and the weight coefficient of the motion matching degree and the characteristic matching degree. In the embodiment of the invention, the Markov distance and the cosine distance are simply overlapped to obtain the association measurement weight matrix.
In some embodiments, selecting global features and local features in S207 includes:
determining a final confidence value of the tracking feature based on the feature confidence of the tracking feature calibration and a predetermined additional confidence;
and selecting the global feature and the local feature with the highest confidence end value.
In particular, for each frame of the sound image, there is theoretically only a pair of matching global and local features for the same underwater vehicle. However, due to the influence of environmental noise and the like, multiple global features and local features may be extracted for the same frame of sound image in the target detection process, and the global features and local features output after target detection and feature matching are performed on multiple sound image images in a certain period of time are multiple. Thus, there is a need to select global and local features that are ultimately used for underwater vehicle positioning.
Firstly, the invention considers the feature confidence of the target detection depth network output as the basis to select the global feature and the local feature. For example, the global feature and the local feature with the highest feature confidence are selected.
However, considering systematic errors, the invention further designs additional confidence levels for reducing the feature confidence levels of the environmental noise of the wrong motion trend. Feature confidence levels of all tracking features tracked by the Kalman filter (the feature confidence levels are output by a target detection depth network and matched to all the tracking features through a Hungary algorithm) and predetermined additional confidence levels are overlapped, and global features and local features with highest confidence level final values are selected as confidence level final values of the tracking features.
Optionally, the additional confidence level is determined based on a motion path of the tracked feature in the sound image map over the tracking period.
In the feature tracking process, when a new detection feature is tracked, the image distance d 0 from the central position of the tracking feature to the multi-beam sonar-based coordinate system is extracted and timing is started until the feature is tracked until the feature is lost.
In the continuous tracking process, the additional confidence delta of the tracking characteristic when the time step is t is calculated through a nonlinear function, and the following conditions are satisfied:
Wherein Δδ represents the additional confidence, k represents the harmonic coefficient, e t represents the exponential function of the time step t, d t represents the image distance from the central position of the tracked feature to the multi-beam sonar-base coordinate system at the time step t, and d 0 represents the image distance from the central position of the tracked feature to the multi-beam sonar-base coordinate system at the time of first tracking to the tracked feature.
When the tracking feature is far from the origin of the multibeam sonar-based coordinate system, i.e., d t＞d0, at time step t relative to the first tracking, the additional confidence Δδ is negative. The additional confidence delta is positive when the tracking feature is near the origin of the multibeam sonar-based coordinate system, i.e., d t＜d0, at time step t relative to the first tracking.
Alternatively, the maximum absolute value of the additional confidence Δδ is limited to 0.2.
Alternatively, the additional confidence Δδ may be calculated by a nonlinear motion trend evaluator.
According to the positioning method in the autonomous docking process of the underwater vehicle, which is provided by the embodiment of the invention, the feature confidence level output by the target detection depth network is not used as the unique basis for selecting the global feature and the local feature, the additional confidence level is determined based on the motion path of the tracking feature in the sound image graph in the tracking process, the confidence level of the noise feature of the wrong motion trend is reduced, the confidence level of the real feature is improved, and the positioning precision and the robustness are further improved.
The technical scheme provided by the invention is further described below through a specific example.
Fig. 3 is a second flow chart of a positioning method in an autonomous docking process of an underwater vehicle according to an embodiment of the present invention, as shown in fig. 3, the method at least includes the following steps:
First, a sonar image is acquired.
In the target detection layer, the target detection depth network is trained based on a multi-beam sonar image data set, and the average feature ratio and the average feature cross ratio of the boat bow features of the underwater vehicle are respectively counted based on the following formula:
wherein N is the number of sound image graphs of the relevant characteristics of the underwater vehicle captured in the training data set S of the target detection depth network, For characteristic boundary frame coordinates of the bow of the boatIs used for the image area of the (c),Boundary frame coordinates characteristic of a hullIs a picture area of the picture frame.
And at the feature matching layer, traversing the feature boundary frame coordinate set { bbox body } of the underwater vehicle hull output by the target detection layer in an outer circulation manner, and traversing the feature boundary frame coordinate set { bbox head } of the hull in an inner circulation manner. Extracting boat body characteristic boundary frame coordinates bbox body and boat bow characteristic boundary frame coordinates bbox head corresponding to the current outer circulation in the inner circulation traversal process, and calculating the boat bow characteristic ratio:
Then, calculating the characteristic crossing ratio of the bow characteristic and the hull characteristic:
average characteristic ratio of lambda 1 to boat bow characteristic Compare and compare lambda 2 to the average feature cross ratio of the boat bow and hull featuresAnd (5) comparing. If it isAnd is also provided withJudging the boundary frame coordinate bbox head of the current boat bow characteristic as the false detection characteristic and entering the next outer layer circulation processing, if the condition is not satisfied, namelyOr (b)Then bbox head is added to the matched boat bow feature set bbox head}f.
After the double-layer traversal is completed, the feature matching outputs a boundary frame coordinate set { bbox body } of the hull feature of the underwater vehicle and a boundary frame coordinate set { bbox head}f of the matched hull feature.
And in the feature tracking layer, tracking a hull feature boundary frame coordinate set { bbox body } output by the feature matching layer and a matched hull bow feature boundary frame coordinate set { bbox head}f by adopting a Kalman filter, wherein the tracking state comprises the center position, the width and the aspect ratio of the feature boundary frame.
The result of the Kalman filter for predicting the position of the tracking characteristic boundary box is respectively as followsAnd (3) withExtracting feature center coordinates p and characteristic center coordinates p of output results and Kalman filter position prediction results of feature matching layers of underwater vehiclesCalculating the mahalanobis distance
Wherein, sigma is the covariance matrix of the high-dimensional random variable. Inputting the output result of the feature matching layer into a convolutional neural network to respectively obtain corresponding feature vectors x f and x kf and calculating cosine distance D cos(xf,xkf):
Superposition And D cos(xf,xkf) forming an association metric weight matrix, adopting a Hungary algorithm to match the output result of the underwater vehicle feature matching unit with the Kalman filter prediction result based on the weight matrix, updating the identification number of the tracking feature and marking the corresponding feature confidence;
At the feature tracking layer, each time a new detected feature is tracked, the nonlinear motion trend evaluator extracts the image distance d 0 from the center position of the feature to the multi-beam sonar-based coordinate system and starts timing until the tracked feature is lost.
In the process of continuously tracking the features, the nonlinear motion trend judger calculates the additional confidence delta of the target features at the time step t through a nonlinear function:
Wherein Δδ represents the additional confidence, k represents the harmonic coefficient, e t represents the exponential function of the time step t, d t represents the image distance from the central position of the tracked feature to the multi-beam sonar-base coordinate system at the time step t, and d 0 represents the image distance from the central position of the tracked feature to the multi-beam sonar-base coordinate system at the time of first tracking to the tracked feature.
The maximum absolute value of the additional confidence delta is limited to 0.2. When the target feature is far from the origin of the multi-beam sonar-based coordinate system, i.e. d t>d0, relative to the start of tracking at time step t, the nonlinear motion trend evaluator outputs an additional confidence delta that is negative. When the target feature approaches the origin of the multi-beam sonar-based coordinate system, i.e., d t<d0, at time step t relative to the start of tracking, the nonlinear motion trend evaluator outputs an additional confidence delta that is positive.
And the additional confidence coefficient delta output by the nonlinear motion trend judger is overlapped with the target detection confidence coefficient to obtain the final target tracking feature confidence coefficient delta. And finally, selecting the boat bow features and the boat body features with highest confidence delta from all target tracking features, and extracting the center coordinates of the feature boundary frames. Based on the multi-beam sonar image resolution, the image coordinates are converted into actual relative distances, and positioning is completed. And the positioning result is sent to the underwater vehicle on-board controller through underwater acoustic communication to complete the docking control.
After the underwater vehicle enters the multi-beam sonar view field, the industrial personal computer acquires the image stream from the multi-beam sonar and inputs the image stream to the deployed system program. The target detection layer firstly calls a pre-trained target detection depth network to identify the boat bow characteristics and the boat body characteristics of the underwater vehicle, the characteristics are further processed by the characteristic tracking layer after being processed by the characteristic matching layer, and the false detection characteristics are further processed and target tracking is realized until the multi-beam sonar loses the target characteristics due to the fact that the underwater vehicle is in butt joint or in emergency floating.
Fig. 4 is a schematic diagram of identifying a target detection layer in a real boat docking test process according to an embodiment of the present invention, and as shown in fig. 4, an identifying result of the target detection layer is shown, ① is environmental noise, ② is real boat characteristics of an underwater vehicle, and ③ is an output result of the target detection layer. Because the upward movement of the environmental noise in the sonar image is partially overlapped with the real boat characteristic of the underwater vehicle, a plurality of false detection results exist in the target detection layer depth network.
Fig. 5 is a schematic diagram of identification of a feature tracking layer in a real boat docking test process provided by the embodiment of the invention, and as shown in fig. 5, an output result of the feature tracking layer is displayed, it can be seen that the environmental noise moving upwards is successfully filtered through further processing of the feature matching layer and the feature tracking layer, a bounding box of the real boat feature of the underwater vehicle is accurately output, and finally, the image coordinate in the center of the bounding box is converted into an actual relative position and distance based on sonar image resolution. The reference real position of the target characteristic is compared, the average positioning error of the technical scheme provided by the embodiment of the invention is less than 0.2m, and the whole-course positioning of the underwater vehicle in the docking process can be realized.
Fig. 6 is a schematic structural diagram of a positioning device in an autonomous docking process of an underwater vehicle according to an embodiment of the present invention, where, as shown in fig. 6, the device at least includes:
The image acquisition unit 601 is used for acquiring an acoustic image in the autonomous docking process of the underwater vehicle based on multi-beam sonar;
The target detection unit 602 is configured to input the sound image to a target detection depth network, and obtain a first output result; the first output result comprises a boundary frame coordinate set and a feature confidence coefficient set which correspond to the global feature and the local feature respectively, and the target detection depth network is obtained by offline training based on a sample sound image and boundary frame coordinate labels of the global feature and the local feature which are determined in advance;
A feature matching unit 603, configured to match the global feature and the local feature based on the first output result and a predetermined feature matching reference threshold, to obtain a first matching result; the first matching result comprises a boundary frame coordinate set of the global feature and the matched local feature;
A feature tracking unit 604, configured to track the first matching result by using a kalman filter, and obtain a position prediction result of the tracking feature; inputting the first matching result into a convolutional neural network, and determining a feature vector; determining a motion matching degree based on the first matching result and the position prediction result; determining feature matching degree based on feature vectors extracted by the convolutional neural network; constructing an association measurement weight matrix based on the motion matching degree and the feature matching degree; based on the association metric weight matrix and the Hungary algorithm, matching the first matching result with the position prediction result, and matching the feature confidence of the tracking feature; selecting global features and local features based on feature confidence of the tracked features;
The positioning obtaining unit 605 is configured to perform coordinate transformation on the selected global feature and the local feature based on the image resolution of the multi-beam sonar, and determine a positioning result of the underwater vehicle in the autonomous docking process.
In some embodiments, the feature matching reference threshold is determined based on the steps of:
Determining the average feature ratio of local features in global features in a sample sound image graph and the average feature cross ratio of the local features and the global features;
And determining the average characteristic occupation ratio and the average characteristic crossing ratio as characteristic matching reference threshold values.
In some embodiments, matching the global features and the local features includes:
traversing the boundary frame coordinate set of the global feature through the outer loop, and traversing the boundary frame coordinate set of the local feature through the inner loop;
And in the inner loop traversal process corresponding to the current outer loop, if the feature ratio of the local feature in the global feature corresponding to the current outer loop is larger than or equal to the average feature ratio, or the feature cross ratio of the local feature and the global feature corresponding to the current outer loop is larger than or equal to the average feature cross ratio, determining that the corresponding local feature is the matched local feature.
In some embodiments, selecting the global feature and the local feature includes:
determining a final confidence value of the tracking feature based on the feature confidence of the tracking feature calibration and a predetermined additional confidence;
And selecting the global feature with the highest confidence end value and the matched local feature.
In some embodiments, the additional confidence is determined based on a motion path of the tracked feature in the sound image map over the tracking period, satisfying:
Wherein Δδ represents the additional confidence, k represents the harmonic coefficient, e t represents the exponential function of the time step t, d t represents the image distance from the central position of the tracked feature to the multi-beam sonar-base coordinate system at the time step t, and d 0 represents the image distance from the central position of the tracked feature to the multi-beam sonar-base coordinate system at the time of first tracking to the tracked feature.
In some embodiments, determining a degree of motion matching based on the first matching result and the position prediction result comprises:
a mahalanobis distance between the first matching result and the position prediction result is determined, the mahalanobis distance being used to characterize the degree of motion matching.
In some embodiments, determining a feature match based on the feature vector extracted by the convolutional neural network comprises:
The feature vector comprises a first feature vector corresponding to the global feature and a second feature vector corresponding to the local feature, and the cosine distance between the first feature vector and the second feature vector is determined and used for representing the feature matching degree.
It should be understood that the detailed functional implementation of each unit/module may be referred to the description of the foregoing method embodiment, and will not be repeated herein.
It should be understood that, the foregoing apparatus is used to perform the method in the foregoing embodiment, and corresponding program modules in the apparatus implement principles and technical effects similar to those described in the foregoing method, and reference may be made to corresponding processes in the foregoing method for the working process of the apparatus, which are not repeated herein.
Based on the method in the above embodiment, the embodiment of the invention provides an industrial personal computer. The apparatus may include: at least one memory for storing programs and at least one processor for executing the programs stored by the memory. Wherein the processor is adapted to perform the method described in the above embodiments when the program stored in the memory is executed.
Fig. 7 is a schematic structural diagram of an industrial personal computer according to an embodiment of the present invention, as shown in fig. 7, the industrial personal computer may include: a processor 701, a communication interface (Communications Interface) 720, a memory 703 and a communication bus 704, wherein the processor 701, the communication interface 702 and the memory 703 communicate with each other through the communication bus 704. The processor 701 may call software instructions in the memory 703 to perform the methods described in the above embodiments.
Based on the method in the above embodiment, the embodiment of the present invention provides a computer-readable storage medium storing a computer program, which when executed on a processor, causes the processor to perform the method in the above embodiment.
Based on the method in the above embodiments, an embodiment of the present invention provides a computer program product, which when run on a processor causes the processor to perform the method in the above embodiments.
It is to be appreciated that the Processor in embodiments of the invention may be a central processing unit (Central Processing Unit, CPU), other general purpose Processor, digital signal Processor (DIGITAL SIGNAL Processor, DSP), application SPECIFIC INTEGRATED Circuit (ASIC), field programmable gate array (Field Programmable GATE ARRAY, FPGA), or other programmable logic device, transistor logic device, hardware component, or any combination thereof. The general purpose processor may be a microprocessor, but in the alternative, it may be any conventional processor.
The steps of the method in the embodiment of the present invention may be implemented by hardware, or may be implemented by executing software instructions by a processor. The software instructions may be comprised of corresponding software modules that may be stored in random access Memory (Random Access Memory, RAM), flash Memory, read-Only Memory (ROM), programmable ROM (PROM), erasable Programmable ROM (EPROM), electrically Erasable Programmable ROM (EEPROM), registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor such the processor can read information from, and write information to, the storage medium. In the alternative, the storage medium may be integral to the processor. The processor and the storage medium may reside in an ASIC.
In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, produces a flow or function in accordance with embodiments of the present invention, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in or transmitted across a computer-readable storage medium. The computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by a wired (e.g., coaxial cable, fiber optic, digital Subscriber Line (DSL)), or wireless (e.g., infrared, wireless, microwave, etc.). The computer readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that contains an integration of one or more available media. The usable medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (Solid STATE DISK, SSD)), etc.
It will be appreciated that the various numerical numbers referred to in the embodiments of the present invention are merely for ease of description and are not intended to limit the scope of the embodiments of the present invention.
It will be readily appreciated by those skilled in the art that the foregoing description is merely a preferred embodiment of the invention and is not intended to limit the invention, but any modifications, equivalents, improvements or alternatives falling within the spirit and principles of the invention are intended to be included within the scope of the invention.
Claims (8)
1. A positioning method in an autonomous docking process of an underwater vehicle, comprising:
acquiring an acoustic image in the autonomous docking process of the underwater vehicle based on multi-beam sonar;
inputting the sound image to a target detection depth network to obtain a first output result; the first output result comprises a boundary frame coordinate set and a feature confidence coefficient set which correspond to the global feature and the local feature respectively, and the target detection depth network is obtained by offline training based on a sample sound image and boundary frame coordinate labels of the global feature and the local feature which are determined in advance;
based on the first output result and a predetermined feature matching reference threshold, matching the global feature with the local feature to obtain a first matching result; the first matching result comprises a boundary frame coordinate set of the global feature and the matched local feature;
tracking the first matching result by using a Kalman filter to obtain a position prediction result of a tracking feature; inputting the first matching result into a convolutional neural network, and determining a feature vector;
determining a motion matching degree based on the first matching result and the position prediction result; determining a feature matching degree based on the feature vector extracted by the convolutional neural network; constructing an associated measurement weight matrix based on the motion matching degree and the feature matching degree;
based on the association metric weight matrix and the Hungary algorithm, matching the first matching result with the position prediction result, and matching the feature confidence coefficient of the tracking feature;
selecting global features and local features based on feature confidence of the tracking features;
Based on the image resolution of the multi-beam sonar, coordinate transformation is carried out on the selected global features and local features, and a positioning result of the underwater vehicle in the autonomous docking process is determined;
wherein the feature matching reference threshold is determined based on the steps of:
determining the average feature ratio of local features in the global features in the sample sound image graph and the average feature cross ratio of the local features and the global features;
Determining the average feature duty cycle and the average feature cross ratio as the feature matching reference threshold;
Wherein the selected global features and local features comprise:
determining a final confidence value of the tracking feature based on the feature confidence of the tracking feature calibration and a predetermined additional confidence; the additional confidence is determined based on the motion path of the tracking feature in the sound image graph in the tracking time period;
And selecting the global feature with the highest confidence coefficient final value and the matched local feature.
2. The method of positioning in an autonomous docking process for an underwater vehicle according to claim 1, wherein said matching said global features and said local features comprises:
traversing the boundary frame coordinate set of the global feature through outer circulation, and traversing the boundary frame coordinate set of the local feature through inner circulation;
And in the inner loop traversal process corresponding to the current outer loop, if the feature ratio of the local feature in the global feature corresponding to the current outer loop is larger than or equal to the average feature ratio, or the feature cross ratio of the local feature and the global feature corresponding to the current outer loop is larger than or equal to the average feature cross ratio, determining that the corresponding local feature is the matched local feature.
3. The positioning method in autonomous docking of an underwater vehicle according to claim 1, wherein the additional confidence level is determined based on a motion path of the tracking feature in the sound image map in a tracking period, satisfying:
Wherein Δδ represents the additional confidence, k represents the harmonic coefficient, e t represents the exponential function of the time step t, d t represents the image distance from the central position of the tracked feature to the multi-beam sonar-base coordinate system at the time step t, and d 0 represents the image distance from the central position of the tracked feature to the multi-beam sonar-base coordinate system at the time of first tracking to the tracked feature.
4. The positioning method in autonomous docking of an underwater vehicle according to claim 1, wherein the determining a degree of motion matching based on the first matching result and the position prediction result includes:
Determining a mahalanobis distance between the first matching result and the position prediction result, the mahalanobis distance being used to characterize the degree of motion matching.
5. The positioning method in autonomous docking of an underwater vehicle according to claim 1, wherein the determining a feature matching degree based on the feature vector extracted by the convolutional neural network comprises:
The feature vector comprises a first feature vector corresponding to a global feature and a second feature vector corresponding to a local feature, and a cosine distance between the first feature vector and the second feature vector is determined and used for representing the feature matching degree.
6. A positioning device in autonomous docking of an underwater vehicle, comprising:
the image acquisition unit is used for acquiring an acoustic image in the autonomous docking process of the underwater vehicle based on the multi-beam sonar;
The target detection unit is used for inputting the sound image to a target detection depth network and obtaining a first output result; the first output result comprises a boundary frame coordinate set and a feature confidence coefficient set which correspond to the global feature and the local feature respectively, and the target detection depth network is obtained by offline training based on a sample sound image and boundary frame coordinate labels of the global feature and the local feature which are determined in advance;
the feature matching unit is used for matching the global feature with the local feature based on the first output result and a predetermined feature matching reference threshold value to obtain a first matching result; the first matching result comprises a boundary frame coordinate set of the global feature and the matched local feature;
The characteristic tracking unit is used for tracking the first matching result by using a Kalman filter to acquire a position prediction result of a tracking characteristic; inputting the first matching result into a convolutional neural network, and determining a feature vector; determining a motion matching degree based on the first matching result and the position prediction result; determining a feature matching degree based on the feature vector extracted by the convolutional neural network; constructing an associated measurement weight matrix based on the motion matching degree and the feature matching degree; based on the association metric weight matrix and the Hungary algorithm, matching the first matching result with the position prediction result, and matching the feature confidence coefficient of the tracking feature; selecting global features and local features based on feature confidence of the tracking features;
the positioning acquisition unit is used for carrying out coordinate conversion on the selected global features and local features based on the image resolution of the multi-beam sonar, and determining a positioning result of the underwater vehicle in the autonomous docking process;
wherein the feature matching reference threshold is determined based on the steps of:
determining the average feature ratio of local features in the global features in the sample sound image graph and the average feature cross ratio of the local features and the global features;
Determining the average feature duty cycle and the average feature cross ratio as the feature matching reference threshold;
Wherein the selected global features and local features comprise:
determining a final confidence value of the tracking feature based on the feature confidence of the tracking feature calibration and a predetermined additional confidence; the additional confidence is determined based on the motion path of the tracking feature in the sound image graph in the tracking time period;
And selecting the global feature with the highest confidence coefficient final value and the matched local feature.
7. An industrial personal computer, characterized by comprising:
at least one memory for storing a program;
At least one processor for executing the memory-stored program, which processor is adapted to perform the method according to any of claims 1-5, when the memory-stored program is executed.
8. A computer readable storage medium storing a computer program, characterized in that the computer program, when run on a processor, causes the processor to perform the method according to any one of claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311725714.4A CN117830399B (en) | 2023-12-14 | 2023-12-14 | Positioning method and device in autonomous docking process of underwater vehicle |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311725714.4A CN117830399B (en) | 2023-12-14 | 2023-12-14 | Positioning method and device in autonomous docking process of underwater vehicle |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117830399A CN117830399A (en) | 2024-04-05 |
| CN117830399B true CN117830399B (en) | 2024-07-16 |
Family
ID=90507020
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311725714.4A Active CN117830399B (en) | 2023-12-14 | 2023-12-14 | Positioning method and device in autonomous docking process of underwater vehicle |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117830399B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111611918A (en) * | 2020-05-20 | 2020-09-01 | 重庆大学 | Traffic flow data set acquisition and construction method based on aerial photography data and deep learning |
| CN113269098A (en) * | 2021-05-27 | 2021-08-17 | 中国人民解放军军事科学院国防科技创新研究院 | Multi-target tracking positioning and motion state estimation method based on unmanned aerial vehicle |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109816690A (en) * | 2018-12-25 | 2019-05-28 | 北京飞搜科技有限公司 | Multi-target tracking method and system based on depth characteristic |
| CN110516556B (en) * | 2019-07-31 | 2023-10-31 | 平安科技（深圳）有限公司 | Multi-target tracking detection method and device based on Darkflow-deep Sort and storage medium |
| JP2022550785A (en) * | 2019-10-07 | 2022-12-05 | 日本電気株式会社 | Visual object tracking method, visual object tracking system, learning method, and learning system |
| CN115995063A (en) * | 2021-10-18 | 2023-04-21 | 中车株洲电力机车研究所有限公司 | Work vehicle detection and tracking method and system |
| CN115147576A (en) * | 2022-06-30 | 2022-10-04 | 浙江大学 | Underwater robot docking monocular vision guiding method based on key characteristics |
| CN116311353A (en) * | 2023-02-09 | 2023-06-23 | 哈尔滨工业大学 | Intensive pedestrian multi-target tracking method based on feature fusion, computer equipment and storage medium |
-
2023
- 2023-12-14 CN CN202311725714.4A patent/CN117830399B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111611918A (en) * | 2020-05-20 | 2020-09-01 | 重庆大学 | Traffic flow data set acquisition and construction method based on aerial photography data and deep learning |
| CN113269098A (en) * | 2021-05-27 | 2021-08-17 | 中国人民解放军军事科学院国防科技创新研究院 | Multi-target tracking positioning and motion state estimation method based on unmanned aerial vehicle |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117830399A (en) | 2024-04-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112308881B (en) | Ship multi-target tracking method based on remote sensing image | |
| CN109186610B (en) | Robust BSLAM method for AUV terrain matching navigation | |
| CN109190544B (en) | Human identity recognition method based on sequence depth image | |
| CN110610165A (en) | Ship behavior analysis method based on YOLO model | |
| CN114564545A (en) | System and method for extracting ship experience course based on AIS historical data | |
| CN115512152B (en) | Ship track classification method and system based on CNN and LSTM neural network combination | |
| CN115131393A (en) | Trajectory prediction method, collision detection method, apparatus, electronic device, and medium | |
| CN116306790B (en) | Offshore ship track real-time prediction method, system, equipment and medium based on CNN-GRU and attention mechanism | |
| CN116580056B (en) | Ship detection and tracking method and device, electronic equipment and storage medium | |
| CN113283653A (en) | Ship track prediction method based on machine learning and AIS data | |
| CN114119659A (en) | Multi-sensor fusion target tracking method | |
| CN110378411A (en) | Maneuvering target tracking method under a kind of support vector machines auxiliary water based on interactive multi-model | |
| CN116381672A (en) | X-band multi-expansion target self-adaptive tracking method based on twin network radar | |
| Li et al. | Target recognition and detection in side-scan sonar images based on YOLO v3 model | |
| CN117830399B (en) | Positioning method and device in autonomous docking process of underwater vehicle | |
| CN116704037A (en) | Satellite lock-losing repositioning method and system based on image processing technology | |
| CN113554072B (en) | Flight action dividing method, computer readable storage medium and terminal equipment | |
| CN111860613B (en) | Multi-target tracking and state predicting method based on multi-source heterogeneous signals | |
| CN114140497A (en) | Target vehicle 3D real-time tracking method and system | |
| CN114898202A (en) | Underwater video target scale space discriminant tracking system and method based on multi-model fusion | |
| Oliveira et al. | Probabilistic positioning of a mooring cable in sonar images for in-situ calibration of marine sensors | |
| CN117111013B (en) | Radar target tracking track starting method, device, equipment and medium | |
| CN117542004B (en) | Offshore man-ship fitting method, device, equipment and storage medium | |
| US20240134009A1 (en) | Method and apparatus of filtering dynamic objects in radar-based ego-emotion estimation | |
| CN118378793B (en) | Marine unmanned equipment search and rescue method, device and equipment based on drifting track prediction |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |