CN117131394A

CN117131394A - Ship track clustering method and equipment based on AIS data and storage equipment

Info

Publication number: CN117131394A
Application number: CN202310975968.5A
Authority: CN
Inventors: 冯宇浩; 奚大平; 江文萍; 刘一飞; 王楚元
Original assignee: China University of Geosciences
Current assignee: China University of Geosciences
Priority date: 2023-08-03
Filing date: 2023-08-03
Publication date: 2023-11-28

Abstract

The invention provides a ship track clustering method, equipment and storage equipment based on AIS data, which comprises the following steps: extracting historical track data, preprocessing abnormal values and redundant values existing in the historical track data, resampling missing data by using a linear interpolation algorithm, and segmenting the processed track; extracting starting and ending points of each track, clustering the starting and ending points by using an improved DBSCAN algorithm, and dividing the tracks with the same class cluster as the starting and ending points into classes; for the tracks of the same type, calculating the similarity between the tracks by using an improved DTW algorithm, and further dividing the track types; and extracting the characteristic track of each type of track, and calculating the similarity of the tracks by utilizing an improved DTW algorithm to obtain a final clustering result. Compared with the traditional DBSCAN algorithm or the DTW algorithm, the method can improve the efficiency and accuracy of track clustering, can well process dense data sets, and can better adapt to clustering results of any shape route through calculation of track similarity.

Description

Ship track clustering method and equipment based on AIS data and storage equipment

Technical Field

The invention relates to the field of ship navigation track analysis, in particular to a ship track clustering method, equipment and storage equipment based on AIS data.

Background

With the continuous development of domestic and foreign economy, the number of ships is continuously increased, and the development trend of large-scale, high-speed and intelligent is presented. The navigation density of water vessels such as coastal ports, estuaries and the like is continuously improved, the navigation environment of the water is more complex, and higher requirements are put forward on the navigation management capacity of the water. The ship automatic identification system (Automatic Identification System, AIS) is used as a carrier of ship motion information, provides massive ship positioning data for related research, and track data positioned through AIS equipment can only reflect information such as longitude and latitude, positioning time, instantaneous speed and heading of a ship, and can not reflect characteristic information of ship navigation, however, the information is very important for planning, managing and controlling maritime traffic. Therefore, the AIS data is mined, the ship track information is analyzed in a clustering way, and intelligent decision support can be provided for maritime supervision and ship traffic.

Clustering serves as an important research direction of the data mining technology, and reasonable classification of the original data can be achieved, so that the internal rules of things are found.

Disclosure of Invention

In order to solve the problems, the invention provides a ship track clustering method, equipment and storage equipment based on AIS data, wherein the ship track clustering method based on AIS data mainly comprises the following steps:

s1: extracting ship track data from the AIS database, and preprocessing abnormal data;

s2: carrying out sectional processing on the track data of each ship after pretreatment, and extracting a navigation track;

s3: clustering starting and ending points of the navigation track by adopting an improved DBSCAN algorithm to obtain a plurality of class clusters;

s4: calculating the similarity distance of each class cluster track by adopting a DTW algorithm, and judging whether the class clusters need to be further divided;

s5: extracting a characteristic track from each class cluster;

s6: and calculating the similarity distance of the feature tracks by using an improved DTW algorithm, and obtaining a final cluster-like set.

Further, in step S1, the abnormal data includes: the preprocessing comprises longitude and latitude error correction, repeated point cleaning and drift point filtering of the abnormal data.

Further, in step S2, the segmentation process includes: and calculating the maximum positioning time in the neighborhood of all the positions within the specified distance according to the positions and the positioning time of the ship, if the two times exceed a time threshold, considering that the ship stays at the position, segmenting the track according to the stay points, and extracting the navigation track.

Further, in step S3, the specific process of obtaining the plurality of clusters is as follows:

s31: extracting a starting point and an ending point of a ship navigation track, respectively constructing a Ball-Tree for a point set of the starting point and the ending point, and obtaining a data set D so as to improve the scanning speed of boundary points in a neighborhood of a core point;

s32: inputting a data set D, a scanning radius R and a minimum inclusion point Pmin;

s33: taking an unoccupied point P in the target data set, judging whether the number of the points P is greater than or equal to Pmin in the scanning radius of the point P, if not, marking the point as a noise point, and repeating the step S33; if yes, judging whether the point P belongs to a certain cluster C, and if not, establishing a new cluster;

s34: step S33 is repeated for all points in the class cluster C that have not been queried until the last point is calculated, and then a set of all clusters is output.

Further, in step S4, the specific procedure of calculating the similarity distance by using the modified DTW algorithm is as follows:

s41: let the sequence of the two tracks be a= { a1, a2, … …, an }, b= { B1, B2, … …, bm };

s42: calculating to obtain a distance matrix Ei= { d (ai, B1), d (ai, B2), … …, d (ai, bm) }, i= {1,2, … …, n }, wherein Ei is the ith row of the distance matrix E, d is the distance between two points, n is the length of the sequence A, and m is the length of the sequence B;

s43: accumulating the passed distances from the upper left corner to the lower right corner according to the distance matrix E, minimizing the accumulated distances, dividing the accumulated distances by the accumulated steps, namely, the average accumulated distance, and taking the distance as the similarity distance between two track sequences;

s44: and setting a proper threshold according to the obtained similarity distances among all track sequences and judging whether the track clusters need to be further divided.

Further, in step S5, for the track similarity distance calculated in step S4, a track with the smallest sum of the track distances from other tracks is found, and the track is the feature track.

Further, in step S6, the similarity distances between all the feature tracks are calculated by using the feature tracks extracted in step S5, so as to obtain a final clustering result.

The storage equipment stores instructions and data for realizing a ship track clustering method based on AIS data.

An AIS data based ship track clustering apparatus comprising: a processor and the storage device; and the processor loads and executes the instructions and the data in the storage device to realize a ship track clustering method based on AIS data.

The technical scheme provided by the invention has the beneficial effects that: according to the method, historical track data are extracted from an AIS database, missing values, abnormal values, redundant values and data formats existing in the historical track data are preprocessed, and the processed track is segmented; the method comprises the steps of extracting the starting point and the end point of each track, clustering the ship tracks in the research range by using an improved DBSCAN algorithm instead of clustering the starting point and the end point of the tracks to obtain track clustering clusters, so that the space complexity of the algorithm is greatly reduced, large-scale data can be effectively clustered, and then the tracks with the same starting and end point clustering results are classified into one type according to the clustering results; for the tracks of the same type, calculating the similarity distance between the tracks by using an improved DTW algorithm, and further dividing the track types; the characteristic track of each type of track is extracted, the similarity distance of the tracks is calculated by utilizing an improved DTW algorithm, and the clustering result of the DBSCAN algorithm is corrected, so that the accuracy of the clustering result can be improved better, the track clustering method with high clustering speed and low complexity can be realized, the behavior of the ship deviating from the navigation channel can be recognized well, and the method can be applied to the fields of ship track monitoring, offshore traffic planning and the like.

Drawings

The invention will be further described with reference to the accompanying drawings and examples, in which:

fig. 1 is a flowchart of a ship track clustering method based on AIS data in an embodiment of the invention.

Fig. 2 is a schematic diagram of extracting a ship track stay point in an embodiment of the invention.

Fig. 3 is a schematic diagram of extracting a navigation track of a ship according to an embodiment of the present invention.

Fig. 4 is a schematic diagram of a DBSCAN algorithm in an embodiment of the present invention.

Fig. 5 is a schematic diagram of the Ball-Tree construction of the trace points in the embodiment of the present invention.

Fig. 6 is a schematic diagram of an improved DTW distance calculation in an embodiment of the invention.

Fig. 7 is a schematic diagram of an improved DTW distance calculation in an embodiment of the invention.

Fig. 8 is a schematic diagram of feature trajectory extraction in an embodiment of the invention.

FIG. 9 is a schematic diagram of the operation of a hardware device in an embodiment of the invention.

Detailed Description

For a clearer understanding of technical features, objects and effects of the present invention, a detailed description of embodiments of the present invention will be made with reference to the accompanying drawings.

The embodiment of the invention provides a ship track clustering method, equipment and storage equipment based on AIS data.

Referring to fig. 1, fig. 1 is a flowchart of a ship track clustering method based on AIS data in an embodiment of the present invention, which specifically includes:

s1, after extracting the historical track data of the ship from the AIS database, preprocessing missing values, abnormal values, redundant values and the like in the data. The judging method of the missing value comprises the following steps: in the time interval of two continuous positioning, the speed of the ship exceeds a speed threshold; the abnormal value judging method comprises the following steps: for the continuous tertiary positioning point, if the running speed between the second point and the first and third points exceeds the speed threshold, but the running speeds of the first point and the third point are within the speed threshold, the second point is considered to be an abnormal point; the redundancy value judging method comprises the following steps: the positions and times of two consecutive positioning are the same. The preprocessing comprises longitude and latitude error correction, repeated point cleaning and drift point filtering processing on missing values, abnormal values, redundant values and the like existing in the data.

S2, traversing all track points according to the positions and the positioning time of the ships, carrying out segmentation processing on the track data of each ship after pretreatment, finding out the segmentation points of the track, and extracting the navigation track. The method comprises the following two modes: (1) Calculating the maximum positioning time in all the position designated neighbors, if the time exceeds a time threshold, considering that the ship stays at the position, marking the first point entering the neighborhood as a stay point, and marking the first point exiting the neighborhood as the starting point of the next track, as shown in fig. 2; (2) As shown in fig. 3, the time difference between two adjacent positioning points is calculated, and if the time threshold is exceeded, the two points are considered as navigation track dividing points and marked as the end point of the previous track and the start point of the next track respectively. And the ship stays at the position, the track is segmented according to the stay points, and the navigation track is extracted.

S3, extracting starting and ending points of all tracks for the navigation track data obtained in the step S2, and clustering the set of the starting and ending points by using an improved DBSCAN algorithm, wherein the specific process is as follows:

(1) The scanning mode of the DBSCAN algorithm to the neighborhood points of the track points is as follows: the distance from all other points to the point is traversed, so that the time complexity of the method is high, and a Ball-Tree is respectively constructed for the track starting point and the track ending point before the DBSCAN algorithm is executed, and a data set D is obtained. As shown in fig. 4, as a result of constructing some sample points on a two-dimensional plane by using a Ball-Tree algorithm, in fig. 4, a large circle contains the whole data set, a middle circle divides data in the large circle into two, a small circle divides data in each middle circle into two, and the small circle is one because at least two points can form a circle, only three points in the middle circle are divided into two parts, only 2+1 can be used, and all two data can form a circle, and one data is the circle.

(2) Inputting a data set D, a scanning radius R and a minimum inclusion point Pmin;

(3) Taking an unoccupied point P in the target data set, judging whether the number of the points P is greater than or equal to Pmin in the scanning radius of the point P, if not, marking the point P as a noise point, and repeating the step (3); if yes, judging whether the point P belongs to a certain cluster C, and if not, establishing a new cluster;

(4) Repeating the step (3) for all points which are not queried in the class cluster C until the last point is calculated, and then outputting a set of all clusters.

As shown in fig. 5, the point P is selected as the first point in the data set D, the number of points included in the scan radius R (in gray circle) is calculated, if the number of points is equal to or greater than the minimum included point Pmin (with a value of 3), it is explained that the point P (core point) belongs to a certain cluster C, and then the above operation is repeated for other points in the cluster C, such as the point Q (density direct), until a set of all the points belonging to the cluster C (density direct) is found. For point Q2, the number of points within the scan radius R is smaller than Pmin, so Q2 is a noise point.

S4, regarding the tracks obtained in the step S2 and the clustering results of the starting points and the ending points obtained in the step S3, if the starting point and the ending point of one or more tracks are in the same cluster, classifying the tracks into one cluster. Then, for the tracks in each class cluster, calculating the track similarity distance between every two tracks, wherein the specific process of calculating the track similarity distance is as follows:

(1) Let the sequence of the two tracks be a= { a1, a2, … …, an }, b= { B1, B2, … …, b=m };

(2) The distance matrix ei= { d (ai, B1), d (ai, B2), … …, d (ai, bm) }, i= {1,2, … …, n }, where Ei is the i-th row of the distance matrix E, d is the distance between two points, n is the length of the sequence a, and m is the length of the sequence B, is calculated.

(3) According to the distance matrix E, the distance covered is accumulated from the upper left corner to the lower right corner, and the accumulated distance is minimized, and then divided by the accumulated number of steps, i.e. the average accumulated distance, which is taken as the similarity distance between the two track sequences. Since any point in the two sequences cannot be skipped during matching, a certain point in one sequence is matched with a plurality of points in the other sequence, so that the DTW distance of the two sequences with low similarity is reduced, and a cross-point matching method is provided to avoid the condition of point matching and multiple points, as shown in fig. 6.

(4) And setting a proper threshold according to the obtained similarity distances among all the tracks, and judging whether the track clusters need to be further divided.

As shown in fig. 7, for two vessel track sequences A, B, the distances from all points in sequence a to all points in sequence B are calculated, then all points in sequence a are traversed, the shortest distance from all points in sequence B is found (dashed line in the figure), these distances are accumulated, and an average accumulated distance is obtained, which is the similarity distance of sequence a to sequence B.

S5, finding out a track with the minimum sum of the track distances from one track to the other tracks according to the track similarity distance calculated in the step S4, and obtaining the track as the characteristic track. As shown in fig. 8, distances d (A1, A2), d (A1, A3), d (A2, A3) between the three track sequences are calculated, and the total distance from the sequence A2 to the sequences A1, A3 is the smallest, so the sequence A2 is the characteristic track.

S6, calculating the similarity among all the characteristic tracks for the characteristic tracks extracted in the step S5, and setting a proper threshold value to obtain a final clustering result.

Referring to fig. 9, fig. 9 is a schematic working diagram of a hardware device according to an embodiment of the present invention, where the hardware device specifically includes: an AIS data-based ship track clustering device 401, a processor 402 and a storage device 403.

Ship track clustering device 401 based on AIS data: the ship track clustering method based on the AIS data is used for achieving the ship track clustering method based on the AIS data.

Processor 402: the processor 402 loads and executes the instructions and data in the storage device 403 to implement the ship track clustering method based on AIS data.

Storage device 403: the storage device 403 stores instructions and data; the storage device 403 is configured to implement the ship track clustering method based on AIS data.

The beneficial effects of the invention are as follows: according to the method, historical track data are extracted from an AIS database, missing values, abnormal values, redundant values and data formats existing in the historical track data are preprocessed, and the processed track is segmented; the method comprises the steps of extracting the starting point and the end point of each track, clustering the ship tracks in the research range by using an improved DBSCAN algorithm instead of clustering the starting point and the end point of the tracks to obtain track clustering clusters, so that the space complexity of the algorithm is greatly reduced, large-scale data can be effectively clustered, and then the tracks with the same starting and end point clustering results are classified into one type according to the clustering results; for the tracks of the same type, calculating the similarity distance between the tracks by using an improved DTW algorithm, and further dividing the track types; the characteristic track of each type of track is extracted, the similarity distance of the tracks is calculated by utilizing an improved DTW algorithm, and the clustering result of the DBSCAN algorithm is corrected, so that the accuracy of the clustering result can be improved better, the track clustering method with high clustering speed and low complexity can be realized, the behavior of the ship deviating from the navigation channel can be recognized well, and the method can be applied to the fields of ship track monitoring, offshore traffic planning and the like.

The foregoing description of the preferred embodiments of the invention is not intended to limit the invention to the precise form disclosed, and any such modifications, equivalents, and alternatives falling within the spirit and scope of the invention are intended to be included within the scope of the invention.

Claims

1. A ship track clustering method based on AIS data is characterized in that: comprising the following steps:

s4: calculating the similarity distance of the tracks in each class cluster by using an improved DTW algorithm, and dividing the tracks with large similarity distance into different class clusters;

s5: extracting a characteristic track from each class cluster;

2. The ship track clustering method based on AIS data, as set forth in claim 1, wherein: in step S1, the anomaly data includes: the preprocessing comprises longitude and latitude error correction, repeated point cleaning and drift point filtering of the abnormal data.

3. The ship track clustering method based on AIS data, as set forth in claim 1, wherein: in step S2, the segmentation process includes: and calculating the maximum positioning time in the neighborhood of all the positions within the specified distance according to the positions and the positioning time of the ship, if the two times exceed a time threshold, considering that the ship stays at the position, segmenting the track according to the stay points, and extracting the navigation track.

4. The ship track clustering method based on AIS data, as set forth in claim 1, wherein: in step S3, the specific process of obtaining the plurality of clusters is as follows:

s33: taking an unoccupied point P in the target data set, judging whether the number of the points in the scanning radius of the point P is greater than or equal to Pmin, if not, marking the point as a noise point, and repeating the step S33; if yes, judging whether the point P belongs to a certain cluster C, and if not, establishing a new cluster;

5. The ship track clustering method based on AIS data, as set forth in claim 1, wherein: in step S4, the specific procedure for calculating the similarity distance by using the improved DTW algorithm is as follows:

6. The ship track clustering method based on AIS data, as set forth in claim 1, wherein: in step S5, for the track similarity distance calculated in step S4, a track with the smallest sum of the track distances from other tracks is found, and the track is the characteristic track.

7. The ship track clustering method based on AIS data, as set forth in claim 1, wherein: in step S6, the similarity distance between all the characteristic tracks is calculated by utilizing the characteristic tracks extracted in step S5, and a final clustering result is obtained.

8. A memory device, characterized by: the storage device stores instructions and data for implementing the AIS data based ship track clustering method according to any one of claims 1 to 7.

9. The utility model provides a boats and ships orbit cluster equipment based on AIS data which characterized in that: comprising the following steps: a processor and a storage device; the processor loads and executes the instructions and data in the storage device for implementing the AIS data based ship track clustering method of any one of claims 1 to 7.