CN117972008A - AIS data extraction method, system and readable storage medium based on spatial index - Google Patents

AIS data extraction method, system and readable storage medium based on spatial index Download PDF

Info

Publication number
CN117972008A
CN117972008A CN202410370768.1A CN202410370768A CN117972008A CN 117972008 A CN117972008 A CN 117972008A CN 202410370768 A CN202410370768 A CN 202410370768A CN 117972008 A CN117972008 A CN 117972008A
Authority
CN
China
Prior art keywords
data
ais
layer
nodes
longitude
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202410370768.1A
Other languages
Chinese (zh)
Other versions
CN117972008B (en
Inventor
王楠
盛尊阔
韩斌
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Elane Inc
Original Assignee
Elane Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Elane Inc filed Critical Elane Inc
Priority to CN202410370768.1A priority Critical patent/CN117972008B/en
Priority claimed from CN202410370768.1A external-priority patent/CN117972008B/en
Publication of CN117972008A publication Critical patent/CN117972008A/en
Application granted granted Critical
Publication of CN117972008B publication Critical patent/CN117972008B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Landscapes

  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention provides an AIS data extraction method, an AIS data extraction system and a readable storage medium based on a spatial index, and relates to the technical field of navigation and geographic information systems. The AIS data extraction method based on the spatial index comprises the following steps: constructing an AIS database, wherein each AIS data comprises longitude and latitude data; constructing a multi-layer data structure for the AIS database, wherein the multi-layer data structure comprises a plurality of nodes, the nodes are distributed in a multi-layer mode, and any node comprises a corresponding longitude and latitude range; acquiring a first longitude and latitude interval; traversing the multi-layer nodes layer by layer downwards according to the first longitude and latitude interval until a target node matched with the first longitude and latitude interval is found out; and extracting AIS data corresponding to the target node as target data.

Description

AIS data extraction method, system and readable storage medium based on spatial index
Technical Field
The invention relates to the technical field of navigation and geographic information systems, in particular to an AIS data extraction method, an AIS data extraction system and a readable storage medium based on spatial indexes.
Background
With increasing heavy sea traffic, AIS data has shown explosive growth. The traditional data extraction method has low efficiency when processing huge data, and cannot meet the requirements of practical application. The prior related patent technology has less research in the field of shipping big data AIS, for the existing method, for extracting AIS data with a time span of 1 month in a certain area, such as a grid area R1 with the angle of 1 degree multiplied by 1 degree (about 110km multiplied by 110 km), the method needs to go through to extract all AIS track points of the ship for 1 month after the area passes through, then takes the longitude and latitude of all track points and the longitude and latitude of a given area to judge, if the longitude and latitude of the track point is reserved in the longitude and latitude of the given area, the longitude and latitude of the track point is not discarded in the longitude and latitude of the given area, the extraction efficiency is low, and multiple matching and judgment are needed.
Therefore, it is important to develop an AIS data extraction method that is fast and efficient, and can reduce unnecessary data access, thereby improving data extraction efficiency.
Disclosure of Invention
In order to solve the technical problems, the application provides an AIS data extraction method for rapidly positioning a data area of interest based on a spatial index technology, reducing unnecessary data access and improving data extraction efficiency.
The first aspect of the invention provides an AIS data extraction method based on a spatial index.
A second aspect of the present invention proposes an AIS data extraction system based on spatial index.
A third aspect of the present invention proposes another spatial index based AIS data extraction system.
A fourth aspect of the invention proposes a readable storage medium.
In view of this, the first aspect of the present invention proposes an AIS data extraction method based on spatial index, comprising: constructing an AIS database, wherein each AIS data comprises longitude and latitude data; constructing a multi-layer data structure for the AIS database, wherein the multi-layer data structure comprises a plurality of nodes, the nodes are distributed in a multi-layer mode, and any node comprises a corresponding longitude and latitude range; acquiring a first longitude and latitude interval; traversing the multi-layer nodes layer by layer downwards according to the first longitude and latitude interval until a target node matched with the first longitude and latitude interval is found out; and extracting AIS data corresponding to the target node as target data.
In the technical scheme, the AIS data extraction method based on the spatial index comprises the following steps: constructing an AIS database, wherein each AIS data comprises longitude and latitude data; constructing a multi-layer data structure for the AIS database, wherein the multi-layer data structure comprises a plurality of nodes, the nodes are distributed in a multi-layer mode, and any node comprises a corresponding longitude and latitude range; acquiring a first longitude and latitude interval; traversing the multi-layer nodes layer by layer downwards according to the first longitude and latitude interval until a target node matched with the first longitude and latitude interval is found out; and extracting AIS data corresponding to the target node as target data. By establishing the multi-layer data structure, only one level of nodes need to be traversed each time, when one level is traversed, the sub-nodes of the nodes meeting the requirements can be traversed, and the sub-nodes of the excluded nodes do not need to be traversed, so that a large number of search files can be reduced.
Optionally, the step of building a multi-layer data structure for the AIS database comprises: dividing a marine area in a preset range into a plurality of first areas according to the limit of the continents and the limit of the seas, wherein the plurality of nodes comprise a plurality of first layer nodes, and the longitude and latitude range corresponding to each first layer node corresponds to one first area; dividing the first area into second areas according to target limit lines and/or target channels, wherein the plurality of nodes further comprise second-layer nodes, and the longitude and latitude range corresponding to each second-layer node corresponds to one second area; dividing the second region into a third region according to the data amount of AIS data, wherein the nodes further comprise third layer nodes, and the longitude and latitude range corresponding to each third layer node corresponds to one third region.
In this technical solution, the step of constructing a multi-layer data structure for the AIS database comprises: dividing a marine area in a preset range into a plurality of first areas according to the limit of the continents and the limit of the seas, wherein the plurality of nodes comprise a plurality of first layer nodes, and the longitude and latitude range corresponding to each first layer node corresponds to one first area; dividing the first area into second areas according to target limit lines and/or target channels, wherein the plurality of nodes further comprise second-layer nodes, and the longitude and latitude range corresponding to each second-layer node corresponds to one second area; dividing the second region into a third region according to the data amount of AIS data, wherein the nodes further comprise third layer nodes, and the longitude and latitude range corresponding to each third layer node corresponds to one third region. The first regions are divided in terms of continents and the ocean, and all of the first regions are referred to as a first layer, which may also be referred to as a continental hierarchy. The second areas are divided according to longitude and latitude, all the second areas are called second layers and can be called area levels, the first areas are composed of a plurality of second layers, or the second areas divide the first areas into two halves, and the second areas generally have similar longitude and latitude ranges. The third areas are divided according to the traffic, all the third areas are called third layers, which can also be called traffic levels or detail levels, the second area is composed of a plurality of third areas, and the AIS data volume in each third area has a similar size. This is because in the actual space, the navigation of the ship has a certain density, and some sea areas may not have a ship passing for a long time, and some sea areas may have a large ship flow, and if the sea areas are still divided according to longitude and latitude, there may be no data or little data in some third areas, and some third areas have large data, so that the data is not easy to be discharged when the data is retrieved.
Optionally, the step of building a multi-layer data structure for the AIS database further comprises: dividing the third area into fourth areas according to ports or channels, wherein the plurality of nodes further comprise fourth-layer nodes, and the longitude and latitude range corresponding to each fourth-layer node corresponds to one fourth area.
In this technical solution, the step of constructing a multi-layer data structure for the AIS database further comprises: dividing the third area into fourth areas according to ports or channels, wherein the plurality of nodes further comprise fourth-layer nodes, and the longitude and latitude range corresponding to each fourth-layer node corresponds to one fourth area. Specifically, according to the scheme disclosed by the application, the fourth area can be arranged on the basis of the third area according to the requirements, the fourth area is a subarea of the third area, and the subarea can be independently arranged at a port or a sea port with dense channels, so that the data range can be further reduced, and the searching detail degree is improved.
Optionally, any node includes a hierarchy number and a sequence number, the hierarchy numbers of nodes of different hierarchies are different, and the sequence numbers of nodes at the same hierarchy are different.
In the technical scheme, any node comprises a hierarchy number and a sequence number, the hierarchy numbers of nodes in different hierarchies are different, and the sequence numbers of nodes in the same hierarchy are different. When searching, the corresponding region can be found by inputting different region numbers, for example, the number of the first region is the same as the number of the second region, when the first region number is searched, the second region number is searched, and the searched data volume is increased. Therefore, the number of the first area, the number of the second area and the number of the third area are provided with unique numbering modes, so that the data volume of the search can be reduced.
Optionally, the step of traversing the multi-layer node layer by layer downwards according to the first longitude and latitude interval includes: when traversing the nodes of each same layer, judging whether the first longitude and latitude interval is intersected with the longitude and latitude range of the node currently traversed, and if so, traversing all the nodes of the next layer of the nodes intersected with the first longitude and latitude interval.
In the technical scheme, the step of traversing the multi-layer node layer by layer downwards according to the first longitude and latitude interval comprises the following steps: when traversing the nodes of each same layer, judging whether the first longitude and latitude interval is intersected with the longitude and latitude range of the node currently traversed, and if so, traversing all the nodes of the next layer of the nodes intersected with the first longitude and latitude interval. Specifically, in the selected first area, further searching is performed, since the first area includes a plurality of second areas, by searching the second areas, a lot of irrelevant data can be removed, and finally searching the selected third area in the second area to obtain a required third area, and by searching the areas in three steps, a lot of irrelevant data can be removed in advance, and because the area searching process only needs to distinguish the information of the areas, but not each AIS data in the areas is searched, the data amount of inquiry is greatly reduced. Meanwhile, different areas can be arranged in different servers or systems, so that the assistance of data can be dispersed, and the expandability and fault tolerance of the system are improved. And the AIS data is updated only by updating the data in the third area, and the data in the first area and the second area are not updated, so that the data updating process is greatly simplified, and the updating operation is reduced.
Optionally, the AIS data further comprises time data; the AIS data extraction method further comprises the following steps: and acquiring a time interval of the demand, traversing time data of AIS data corresponding to the target node, and extracting AIS data conforming to the time interval as target data.
In the technical scheme, the AIS data also comprises time data; the AIS data extraction method further comprises the following steps: and acquiring a time interval of the demand, traversing time data of AIS data corresponding to the target node, and extracting AIS data conforming to the time interval as target data. In the actual use process, the AIS data can be searched according to the time data of the AIS data, and the AIS data in the target node determined after the space index is completed can be further searched by setting the time interval of the requirement in the search requirement, so that the AIS data meeting the space requirement and the time requirement is obtained. The specific use condition may be that a user needs to retrieve the AIS data of a certain port in a certain time period, the target node may be retrieved according to the longitude and latitude data of the port, then the time is retrieved, and finally the AIS data of the certain port in a certain time period is obtained. The search can be performed in such a manner that the search amount can be reduced, and the time search for all the AIS data is not required, and only the AIS data in the designated area can be searched.
Optionally, the AIS data further comprises a vessel type; the AIS data extraction method further comprises the following steps: acquiring ship type requirements, traversing the ship types of AIS data corresponding to the target nodes, and extracting AIS data meeting the ship type requirements as target data.
In the technical scheme, the AIS data also comprises a ship type; the AIS data extraction method further comprises the following steps: acquiring ship type requirements, traversing the ship types of AIS data corresponding to the target nodes, and extracting AIS data meeting the ship type requirements as target data. In the actual use process, the AIS data can be searched according to the ship type of the AIS data, and the AIS data in the target node determined after the space index is completed can be further searched by setting the required ship type in the search requirement, so that the AIS data meeting the space requirement and the ship type is obtained. The specific use condition may be that a user needs to retrieve AIS data of a certain port in a certain type of ship, the target node can be retrieved according to longitude and latitude data of the port, then the ship type is retrieved, and finally AIS data of the certain port in the certain type of ship is obtained. The search can be performed in such a manner that the search amount can be reduced, and the ship type search for all the AIS data is not required, and only the AIS data in the designated area can be searched.
A second aspect of the present invention provides a spatial index based AIS data extraction system comprising: the AIS database construction module is used for constructing an AIS database, and each AIS data comprises longitude and latitude data; constructing a multi-layer data structure for the AIS database, wherein the multi-layer data structure comprises a plurality of nodes, the nodes are distributed in a multi-layer mode, and any node comprises a corresponding longitude and latitude range; the query module is used for acquiring a first longitude and latitude interval; the retrieval module is used for traversing the multi-layer nodes layer by layer downwards according to the first longitude and latitude interval until a target node matched with the first longitude and latitude interval is found out; and the extraction module is used for extracting AIS data corresponding to the target node as target data.
A third aspect of the present invention provides a spatial index based AIS data extraction system comprising a memory and a processor, the memory storing a program or instruction executable on the processor, the program or instruction when executed by the processor implementing the steps of the spatial index based AIS data extraction method provided in the first aspect of the present invention.
A fourth aspect of the present invention provides a readable storage medium having stored thereon a program and/or instructions which, when executed by a processor, implement the steps of the spatial index based AIS data extraction method provided in the first aspect.
In the prior art, database indexing techniques, such as B-tree and hash indexing, are not efficient for spatial data because they do not take full advantage of the characteristics of spatial data. In the application, the target area is divided into areas with fixed sizes by the child node grids of different levels, each area has a definite longitude and latitude range, and the grids divided according to the levels are particularly suitable for range query. When space inquiry is carried out, which areas are intersected with the inquiry areas can be rapidly determined, so that data retrieval is carried out only in the areas, the search space is greatly reduced, meanwhile, as grid division is uniform, the data volume in each grid is relatively uniformly distributed, and the problem that data in some areas are too dense and data in other areas are sparse is avoided. This helps to reduce redundant data during the query process and improve the accuracy and efficiency of the query. The data is divided into multiple independent parts by meshing, each of which can be handled independently by a different system or server. The partitioning strategy is helpful for distributing data load and improving the expandability and fault tolerance of the system.
Additional aspects and advantages of the invention will be set forth in part in the description which follows, or may be learned by practice of the invention.
Drawings
The foregoing and/or additional aspects and advantages of the invention will become apparent and may be better understood from the following description of embodiments taken in conjunction with the accompanying drawings in which:
FIG. 1 is one of the flow diagrams of a spatial index based AIS data extraction method according to one embodiment of the invention;
FIG. 2 is one of the block diagrams of a spatial index based AIS data extraction system according to an embodiment of the present invention;
FIG. 3 is a second block diagram of a spatial index based AIS data extraction system according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of the division of a first region and a second region in an embodiment in accordance with the invention;
FIG. 5 is a schematic diagram of the division of a third region in an embodiment in accordance with the invention;
Fig. 6 is a second flow chart of a spatial index based AIS data extraction method according to one embodiment of the invention.
Wherein, the correspondence between the reference numerals and the component names of fig. 4 and 5 is:
1a first region, 2a second region, 3a third region.
Detailed Description
In order that the above objects, features and advantages of the application will be more readily understood, a more particular description of the application will be rendered by reference to specific embodiments thereof which are illustrated in the appended drawings. It should be noted that, without conflict, the embodiments of the present application and features in the embodiments may be combined with each other.
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention, but the present invention may be practiced in other ways than those described herein, and therefore the scope of the present invention is not limited to the specific embodiments disclosed below.
The spatial index based AIS data extraction method in some embodiments of the invention is described below with reference to fig. 1.
As shown in fig. 1, the AIS data extraction method based on the spatial index includes:
s101: constructing an AIS database, wherein each AIS data comprises longitude and latitude data;
S103: constructing a multi-layer data structure for the AIS database, wherein the multi-layer data structure comprises a plurality of nodes, the nodes are distributed in a multi-layer mode, and any node comprises a corresponding longitude and latitude range;
s105: acquiring a first longitude and latitude interval;
S107: traversing the multi-layer nodes layer by layer downwards according to the first longitude and latitude interval until a target node matched with the first longitude and latitude interval is found out;
s109: and extracting AIS data corresponding to the target node as target data.
In the technical scheme, the AIS data extraction method based on the spatial index comprises the following steps: constructing an AIS database, wherein each AIS data comprises longitude and latitude data; constructing a multi-layer data structure for the AIS database, wherein the multi-layer data structure comprises a plurality of nodes, the nodes are distributed in a multi-layer mode, and any node comprises a corresponding longitude and latitude range; acquiring a first longitude and latitude interval; traversing the multi-layer nodes layer by layer downwards according to the first longitude and latitude interval until a target node matched with the first longitude and latitude interval is found out; and extracting AIS data corresponding to the target node as target data. By establishing the multi-layer data structure, only one level of nodes need to be traversed each time, when one level is traversed, the sub-nodes of the nodes meeting the requirements can be traversed, and the sub-nodes of the excluded nodes do not need to be traversed, so that a large number of search files can be reduced.
Optionally, the step of building a multi-layer data structure for the AIS database comprises: dividing a marine area in a preset range into a plurality of first areas according to the limit of the continents and the limit of the seas, wherein the plurality of nodes comprise a plurality of first layer nodes, and the longitude and latitude range corresponding to each first layer node corresponds to one first area; dividing the first area into second areas according to target limit lines and/or target channels, wherein the plurality of nodes further comprise second-layer nodes, and the longitude and latitude range corresponding to each second-layer node corresponds to one second area; dividing the second region into a third region according to the data amount of AIS data, wherein the nodes further comprise third layer nodes, and the longitude and latitude range corresponding to each third layer node corresponds to one third region.
In this technical solution, the step of constructing a multi-layer data structure for the AIS database comprises: dividing a marine area in a preset range into a plurality of first areas according to the limit of the continents and the limit of the seas, wherein the plurality of nodes comprise a plurality of first layer nodes, and the longitude and latitude range corresponding to each first layer node corresponds to one first area; dividing the first area into second areas according to target limit lines and/or target channels, wherein the plurality of nodes further comprise second-layer nodes, and the longitude and latitude range corresponding to each second-layer node corresponds to one second area; dividing the second region into a third region according to the data amount of AIS data, wherein the nodes further comprise third layer nodes, and the longitude and latitude range corresponding to each third layer node corresponds to one third region. The first regions are divided in terms of continents and the ocean, and all of the first regions are referred to as a first layer, which may also be referred to as a continental hierarchy. The second areas are divided according to longitude and latitude, all the second areas are called second layers and can be called area levels, the first areas are composed of a plurality of second layers, or the second areas divide the first areas into two halves, and the second areas generally have similar longitude and latitude ranges. The third areas are divided according to the traffic, all the third areas are called third layers, which can also be called traffic levels or detail levels, the second area is composed of a plurality of third areas, and the AIS data volume in each third area has a similar size. This is because in the actual space, the navigation of the ship has a certain density, and some sea areas may not have a ship passing for a long time, and some sea areas may have a large ship flow, and if the sea areas are still divided according to longitude and latitude, there may be no data or little data in some third areas, and some third areas have large data, so that the data is not easy to be discharged when the data is retrieved.
Optionally, the step of building a multi-layer data structure for the AIS database further comprises: dividing the third area into fourth areas according to ports or channels, wherein the plurality of nodes further comprise fourth-layer nodes, and the longitude and latitude range corresponding to each fourth-layer node corresponds to one fourth area.
In this technical solution, the step of constructing a multi-layer data structure for the AIS database further comprises: dividing the third area into fourth areas according to ports or channels, wherein the plurality of nodes further comprise fourth-layer nodes, and the longitude and latitude range corresponding to each fourth-layer node corresponds to one fourth area. Specifically, according to the scheme disclosed by the application, the fourth area can be arranged on the basis of the third area according to the requirements, the fourth area is a subarea of the third area, and the subarea can be independently arranged at a port or a sea port with dense channels, so that the data range can be further reduced, and the searching detail degree is improved.
Optionally, any node includes a hierarchy number and a sequence number, the hierarchy numbers of nodes of different hierarchies are different, and the sequence numbers of nodes at the same hierarchy are different.
In the technical scheme, any node comprises a hierarchy number and a sequence number, the hierarchy numbers of nodes in different hierarchies are different, and the sequence numbers of nodes in the same hierarchy are different. When searching, the corresponding region can be found by inputting different region numbers, for example, the number of the first region is the same as the number of the second region, when the first region number is searched, the second region number is searched, and the searched data volume is increased. Therefore, the number of the first area, the number of the second area and the number of the third area are provided with unique numbering modes, so that the data volume of the search can be reduced.
Optionally, the step of traversing the multi-layer node layer by layer downwards according to the first longitude and latitude interval includes: when traversing the nodes of each same layer, judging whether the first longitude and latitude interval is intersected with the longitude and latitude range of the node currently traversed, and if so, traversing all the nodes of the next layer of the nodes intersected with the first longitude and latitude interval.
In the technical scheme, the step of traversing the multi-layer node layer by layer downwards according to the first longitude and latitude interval comprises the following steps: when traversing the nodes of each same layer, judging whether the first longitude and latitude interval is intersected with the longitude and latitude range of the node currently traversed, and if so, traversing all the nodes of the next layer of the nodes intersected with the first longitude and latitude interval. Specifically, in the selected first area, further searching is performed, since the first area includes a plurality of second areas, by searching the second areas, a lot of irrelevant data can be removed, and finally searching the selected third area in the second area to obtain a required third area, and by searching the areas in three steps, a lot of irrelevant data can be removed in advance, and because the area searching process only needs to distinguish the information of the areas, but not each AIS data in the areas is searched, the data amount of inquiry is greatly reduced. Meanwhile, different areas can be arranged in different servers or systems, so that the assistance of data can be dispersed, and the expandability and fault tolerance of the system are improved. And the AIS data is updated only by updating the data in the third area, and the data in the first area and the second area are not updated, so that the data updating process is greatly simplified, and the updating operation is reduced.
Optionally, the AIS data further comprises time data; the AIS data extraction method further comprises the following steps: and acquiring a time interval of the demand, traversing time data of AIS data corresponding to the target node, and extracting AIS data conforming to the time interval as target data.
In the technical scheme, the AIS data also comprises time data; the AIS data extraction method further comprises the following steps: and acquiring a time interval of the demand, traversing time data of AIS data corresponding to the target node, and extracting AIS data conforming to the time interval as target data. In the actual use process, the AIS data can be searched according to the time data of the AIS data, and the AIS data in the target node determined after the space index is completed can be further searched by setting the time interval of the requirement in the search requirement, so that the AIS data meeting the space requirement and the time requirement is obtained. The specific use condition may be that a user needs to retrieve the AIS data of a certain port in a certain time period, the target node may be retrieved according to the longitude and latitude data of the port, then the time is retrieved, and finally the AIS data of the certain port in a certain time period is obtained. The search can be performed in such a manner that the search amount can be reduced, and the time search for all the AIS data is not required, and only the AIS data in the designated area can be searched.
Optionally, the AIS data further comprises a vessel type; the AIS data extraction method further comprises the following steps: acquiring ship type requirements, traversing the ship types of AIS data corresponding to the target nodes, and extracting AIS data meeting the ship type requirements as target data.
In the technical scheme, the AIS data also comprises a ship type; the AIS data extraction method further comprises the following steps: acquiring ship type requirements, traversing the ship types of AIS data corresponding to the target nodes, and extracting AIS data meeting the ship type requirements as target data. In the actual use process, the AIS data can be searched according to the ship type of the AIS data, and the AIS data in the target node determined after the space index is completed can be further searched by setting the required ship type in the search requirement, so that the AIS data meeting the space requirement and the ship type is obtained. The specific use condition may be that a user needs to retrieve AIS data of a certain port in a certain type of ship, the target node can be retrieved according to longitude and latitude data of the port, then the ship type is retrieved, and finally AIS data of the certain port in the certain type of ship is obtained. The search can be performed in such a manner that the search amount can be reduced, and the ship type search for all the AIS data is not required, and only the AIS data in the designated area can be searched.
Alternatively, as shown in fig. 4 and 5, the first areas 1 are divided according to continents and the ocean, and all the first areas 1 are called a first layer, which may also be called a continental level. The second areas 2 are divided according to longitude and latitude, all the second areas 2 are called second layers and can be called area levels, the first area 1 is composed of a plurality of second layers, or the second areas 2 divide the first area 1 into two halves, and the second areas 2 generally have similar longitude and latitude ranges. The third areas 3 are divided according to the traffic, all the third areas 3 are called third layers, which may also be called traffic levels or detail levels, the second area 2 is formed by a plurality of third areas 3, and the AIS data amount in each third area 3 has a similar size. This is because in the actual space, the navigation of the ship has a certain density, and some sea areas may not pass through for a long time, but some sea areas may have a large ship flow, and if the ship is still divided according to longitude and latitude, there may be no data or little data in some third areas 3, and some third areas 3 have large data, so that the data is not easily discharged when the data is retrieved.
In this embodiment, when a user specifies a particular region that needs to be extracted, the spatial index database is quickly located to the corresponding node. Then, recursively traversing the nodes according to the area range specified by the user, checking whether the space range of the node is intersected with the query area on each node, and if not, ignoring the node and all child nodes thereof; if the nodes are intersected, continuing to traverse the child nodes of the node, and screening AIS data meeting the conditions. In the screening process, the data range can be further narrowed by setting filtering conditions (such as time range, ship type and the like).
Optionally, the spatial range of the data to be extracted includes: latitude and longitude ranges of data to be extracted; the first layer node comprises a longitude and latitude range defined by a first area 1; the second layer node comprises a longitude and latitude range defined by a second area 2; the third level node comprises a latitude and longitude range defined by the third area 3. That is, in the actual searching process, according to the spatial range of the data to be extracted, the spatial range is compared with the longitude and latitude range of the node, instead of the longitude and latitude range of the AIS data, so that the data amount can be greatly reduced, the searching space can be reduced, the searching precision and the searching efficiency can be improved, the method is more suitable for range searching, the corresponding third area 3 can be quickly found through the index, and then the data corresponding to the third area 3 can be extracted.
Optionally, the number of the first region 1, the number of the second region 2 and the number of the third region 3 all have a unique numbering scheme.
In this embodiment, the numbers of the first area 1, the second area 2 and the third area 3 all have unique numbers, and when searching, the corresponding area can be found by inputting different area numbers, for example, the numbers of the first area 1 and the second area 2 are the same, when searching the numbers of the first area 1, the numbers of the second area 2 can be searched, which increases the searched data volume. Therefore, the number of the first area, the number of the second area and the number of the third area are provided with unique numbering modes, so that the data volume of the search can be reduced.
The AIS data extraction method based on the spatial index provided by the invention comprises the following steps: acquiring AIS data; as shown in fig. 4 and 5, the target space is divided into a plurality of first areas 1, the plurality of first areas 1 form a first layer, the first areas 1 are numbered, any one of the first areas 1 is divided into a plurality of second areas 2, the plurality of second areas 2 form a second layer, the second areas 2 are numbered, any one of the second areas 2 is divided into a plurality of third areas 3, the plurality of third areas 3 form a third layer, the third areas 3 are numbered, and a node is established according to the number of the first areas 1 and/or the number of the second areas 2 and/or the number of the third areas 3; based on AIS data and nodes, establishing a spatial index database of the AIS data; acquiring a spatial range of data to be extracted; traversing downwards from the nodes of the first areas in sequence based on the space scope, wherein when traversing each node, judging whether the space scope is intersected with the currently traversed node, and if so, traversing all the nodes of the next level of the nodes intersected with the space scope until the target node corresponding to the space scope is positioned; and extracting AIS data corresponding to the target node. The target space is divided into a plurality of first areas 1 according to longitude and latitude, the range of the first areas 1 is larger, the search of the first areas 1 corresponds to preliminary search, the search range is reduced in advance, so that a large amount of irrelevant data is removed, further, in the selected first areas 1, further search is carried out, because the first areas 1 comprise a plurality of second areas 2, a large amount of irrelevant data can be removed through searching the second areas 2, finally, the third areas 3 in the selected second areas 2 are searched to obtain the required third areas 3, a large amount of irrelevant data can be removed in advance through searching the areas in three steps, and because the search process of the areas only needs to distinguish the information of the areas, but not each AIS data in the areas is searched, so that the data amount of query is greatly reduced. Meanwhile, different areas can be arranged in different servers or systems, so that the assistance of data can be dispersed, and the expandability and fault tolerance of the system are improved. And the AIS data is updated, only the data in the third area 3 is required to be updated, and the data in the first area 1 and the second area 2 are not required to be updated, so that the data updating process is greatly simplified, and the updating operation difficulty is reduced.
Further, the spatial index-based AIS data extraction method disclosed in the present application, as shown in fig. 6, includes:
s1: the data preparation and preprocessing, and the collection and arrangement of original AIS message data, including longitude and latitude, time stamp, ship type and other information of the ship. And preprocessing ship data, such as removing repeated data, removing error and incomplete data, noise point and outlier processing and the like.
S2: and (3) establishing a spatial index, establishing a spatial index database, and establishing an index relation with the data processed in the step (S1). And (3) dividing longitude and latitude of the global network, and establishing a spatial index database with the AIS data processed in the step (S1) according to longitude and latitude information. The spatial index database adopts a layered data structure, and divides the global sea area into a plurality of layers according to the longitude and latitude range, wherein each layer comprises a plurality of areas. As shown in fig. 4, further, the detailed hierarchy is divided as follows:
1. Global hierarchy (Level 0):
The global sea area is regarded as a single area and is not further subdivided. The global grid number is G0.
2. Large Liu Cengji (Level 1):
the global sea area is divided into ocean areas around a plurality of continents according to the boundaries of the continents and the ocean. The marine area around each continent becomes one area, the first area 1. The grid number of each continent or ocean may be numbered based on their location in the global grid, e.g., A0, A1, B0, B1, etc., where a and B represent different continents or oceans.
3. Region hierarchy (Level 2):
The marine area around each continent is further divided into smaller areas, i.e. second areas 2, for example according to the natural demarcation of the ocean or sea area. Each region has a similar latitude and longitude range, and is divided. Under each continent or ocean, the grid number of the more finely divided sea area or geographic region may be numbered based on their location under the parent node, e.g., A0, A0A1, A1B0, B0c0, etc., as shown in fig. 4.
4. Detail Level (Level 3+):
The area may be further subdivided into finer levels, i.e. the third area 3, as desired, for example according to the territory of a particular country or important channel, and the area of high traffic may be more finely divided and the area of low traffic may be relatively coarse. Each region continues to be partitioned in terms of latitude and longitude ranges until the desired level of detail is reached. For more detailed grids, such as channels, ports, etc., it may continue to be numbered in a hierarchical fashion, such as A0A0i0, A0A0i1, etc., as shown in FIG. 5.
Each area corresponds to a node, and summary information (such as data amount, time range and the like) of AIS data in the area is stored in the node.
S3: and (3) data screening, namely rapidly positioning the space index database established based on the step S2 to index data, and screening the index data. When the user designates a specific area to be extracted, the system quickly locates to the corresponding node based on the spatial index database established in S2. Then, recursively traversing the nodes according to the area range specified by the user, checking whether the space range of the node is intersected with the query area on each node, and if not, ignoring the node and all child nodes thereof; if the nodes are intersected, continuing to traverse the child nodes of the node, and screening AIS data meeting the conditions. In the screening process, the data range can be further narrowed by setting filtering conditions (such as time range, ship type and the like).
S4: and outputting a result, processing the screened AIS data and outputting the processed AIS data. The screened AIS data records are subjected to necessary post-processing such as sorting, screening, aggregation and the like, and the results are output in a required format, such as text files, database tables, graphical interfaces and the like.
S5: and (5) periodically maintaining and updating the index, and periodically maintaining to ensure the compactness and the query efficiency of the index. Periodic maintenance and updating of the index is critical to maintaining its performance. This includes rebalancing the index structure, deleting invalid or outdated data records, merging adjacent nodes, etc., and by periodic maintenance, compactness of the index and query efficiency can be ensured according to the optimal hierarchy and sub-node distribution required by the actual service.
The scheme has the following advantages:
1) Query efficiency and extraction efficiency are improved:
and (3) quick positioning: the different levels of child node grids divide the world into fixed-size areas, and each area has a definite longitude and latitude range. When a spatial query is made, it is possible to quickly determine which regions intersect the query region, so that data retrieval is performed only in these regions, greatly reducing the search space.
Reducing redundant data: because the area division is uniform, the data volume in each area is relatively uniformly distributed, and the problem that data in some areas are too dense and data in other areas are sparse is avoided. This helps to reduce redundant data during the query process and improve the accuracy and efficiency of the query.
Support range query: the method of dividing the areas according to the hierarchy is particularly suitable for range query, such as querying all ships in a certain sea area. By indexing the regions, the regions containing the query region can be quickly found and then the desired data extracted from those regions.
2) Easy data management and maintenance:
data partitioning: gridding divides global data into multiple independent parts, each of which can be handled independently by a different system or server. The partitioning strategy is helpful for distributing data load and improving the expandability and fault tolerance of the system.
Simplifying data updating: when the spatial data changes, such as updating of the ship position, only the data in the corresponding area need to be updated, and the whole data set does not need to be changed. This greatly simplifies the process of data maintenance and reduces the complexity of the update operation.
Data aggregation: the meshed data structure enables aggregation and analysis of data to be simple and efficient. For example, information such as the number of ships, traffic flow and the like in each area can be counted, and powerful support is provided for decision making.
3) System flexibility and scalability are enhanced:
flexibility: the division mode of the areas enables the system to flexibly adapt to different application requirements. The size and granularity of the regions can be adjusted according to different query precision and performance requirements to meet specific application requirements.
Scalability: as data grows and query requirements increase, the performance of the system may be improved by increasing the resolution of the region (i.e., decreasing the region size). This scalability enables the system to cope with ever-increasing data volumes and query loads.
In summary, the scheme of establishing the spatial index by adopting the sub-node areas under different levels has remarkable advantages in aspects of query efficiency, data extraction, data management and maintenance, system flexibility, expandability and the like, and provides a more efficient and reliable method for spatial data processing in the navigation field.
The scheme provides a space index construction method based on child node areas of different levels. Compared with the traditional space index method, the method adopts different levels of sub-nodes to divide the global space into regular areas, and each area has similar longitude and latitude ranges. The regular grid structure not only simplifies the organization and storage of spatial data, but also optimizes the efficiency of spatial querying. By indexing the region, the region containing the target object can be quickly located, thereby avoiding inefficient operation of the entire scan. The innovative space index construction method greatly improves the query and processing efficiency of space data in the navigation field.
The sub-nodes of different levels are combined with the spatial index, so that the integrated application of the sub-nodes and the spatial index is realized. The traditional equal longitude and latitude grid is mainly used for dividing and organizing spatial data, and the scheme further expands the spatial data into a basis of spatial index. By establishing a node for each region, the space object is associated with the corresponding region, so that the rapid query and positioning of the space data are realized. The integrated application not only improves the query efficiency of the space data, but also simplifies the management and maintenance process of the data. Meanwhile, the integrated application of nodes of different levels and the spatial index provides powerful support for other spatial data analysis tasks in the navigation field.
A second aspect of the present invention provides a spatial index based AIS data extraction system 200, as shown in fig. 2, comprising: an AIS database construction module 201, configured to construct an AIS database, where each AIS data includes latitude and longitude data; constructing a multi-layer data structure for the AIS database, wherein the multi-layer data structure comprises a plurality of nodes, the nodes are distributed in a multi-layer mode, and any node comprises a corresponding longitude and latitude range; a query module 203, configured to obtain a first longitude and latitude interval; the retrieval module 205 is configured to traverse the multi-layer nodes layer by layer downwards according to the first longitude and latitude interval until a target node matching the first longitude and latitude interval is found; the extracting module 207 is configured to extract AIS data corresponding to the target node as target data. The spatial index-based AIS data extraction system 200 is configured to implement the steps of the spatial index-based AIS data extraction method provided in the first aspect of the present invention, so that the spatial index-based AIS data extraction system 200 has all technical effects of the spatial index-based AIS data extraction method, which are not described herein.
A third aspect of the present invention provides a spatial index based AIS data extraction system 300, as shown in fig. 3, comprising a memory 302 and a processor 304, the memory storing programs or instructions executable on the processor, which when executed by the processor implement the steps of the spatial index based AIS data extraction method provided by the first aspect of the present invention.
The spatial index based AIS data extraction system 300 provided according to an embodiment of the present application includes a memory 302, a processor 304, and a program stored in the memory 302 and executable on the processor 304, which when executed by the processor 304 implements the steps defined by the spatial index based AIS data extraction method according to the first aspect of the present application. Meanwhile, since the spatial index-based AIS data extraction system 300 of the present application can implement the steps of the spatial index-based AIS data extraction method provided in the first aspect of the present application, the spatial index-based AIS data extraction system 300 provided in this embodiment has all the advantages of the spatial index-based AIS data extraction method described in the first aspect.
A fourth aspect of the present invention provides a readable storage medium having stored thereon a program and/or instructions which, when executed by a processor, implement the steps of the spatial index based AIS data extraction method provided in the first aspect of the present invention.
According to the readable storage medium provided by the embodiment of the present invention, since the steps of the spatial index based AIS data extraction method provided in the first aspect of the present invention can be implemented when the program and/or the instruction stored thereon are executed by the processor, all the beneficial technical effects of the spatial index based AIS data extraction method described in the first aspect are provided, and are not described herein.
In this specification, the term "plurality" means two or more, unless explicitly defined otherwise. The terms "mounted," "connected," "secured," and the like are to be construed broadly, and may be, for example, fixedly connected, detachably connected, or integrally connected; "coupled" may be directly coupled or indirectly coupled through intermediaries. The specific meaning of the above terms in the present invention can be understood by those of ordinary skill in the art according to the specific circumstances.
In the description of the present specification, the terms "one embodiment," "some embodiments," and the like, mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
The above is only a preferred embodiment of the present invention, and is not intended to limit the present invention, but various modifications and variations can be made to the present invention by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims (10)

1. An AIS data extraction method based on spatial index, which is characterized by comprising the following steps:
Constructing an AIS database, wherein each AIS data comprises longitude and latitude data;
Constructing a multi-layer data structure for an AIS database, wherein the multi-layer data structure comprises a plurality of nodes, the nodes are distributed in a multi-layer mode, and any node comprises a corresponding longitude and latitude range;
Acquiring a first longitude and latitude interval;
traversing a plurality of layers of nodes layer by layer downwards according to the first longitude and latitude interval until a target node matched with the first longitude and latitude interval is found out;
and extracting the AIS data corresponding to the target node as target data.
2. The spatial index based AIS data extraction method of claim 1 wherein said step of constructing a multi-tier data structure for the AIS database comprises:
Dividing a marine area in a preset range into a plurality of first areas according to the limit of continents and the limit of oceans, wherein the plurality of nodes comprise a plurality of first layer nodes, and the longitude and latitude range corresponding to each first layer node corresponds to one first area;
dividing the first area into a second area according to a target limit line and/or a target channel, wherein the nodes further comprise second-layer nodes, and the longitude and latitude range corresponding to each second-layer node corresponds to one second area;
Dividing the second area into a third area according to the data volume of AIS data, wherein the nodes further comprise third-layer nodes, and the longitude and latitude range corresponding to each third-layer node corresponds to one third area.
3. The spatial index based AIS data extraction method of claim 2 wherein said step of constructing a multi-tier data structure for the AIS database further comprises:
Dividing the third area into fourth areas according to ports or channels, wherein the nodes further comprise fourth-layer nodes, and the longitude and latitude range corresponding to each fourth-layer node corresponds to one fourth area.
4. A spatial index based AIS data extraction method according to any one of claims 1 to 3 wherein any one of said nodes comprises a hierarchy number and a sequence number, said hierarchy numbers of said nodes of different hierarchies being different, said sequence numbers of said nodes at the same hierarchy being different.
5. A spatial index based AIS data extraction method according to any one of claims 1 to 3 wherein said step of traversing a plurality of layers of said nodes layer by layer down according to said first latitude and longitude intervals comprises:
When traversing the nodes of each same layer, judging whether the first longitude and latitude interval is intersected with the longitude and latitude range of the node currently traversed, and if so, traversing all the nodes of the next layer of the nodes intersected with the first longitude and latitude interval.
6. A spatial index based AIS data extraction method according to any one of claims 1 to 3 wherein the AIS data further comprises time data;
The AIS data extraction method further comprises the following steps:
And acquiring a required time interval, traversing the time data of the AIS data corresponding to the target node, and extracting the AIS data which accords with the time interval as target data.
7. A spatial index based AIS data extraction method according to any one of claims 1 to 3 wherein the AIS data further comprises a vessel type;
The AIS data extraction method further comprises the following steps:
And acquiring a ship type requirement, traversing the ship type of the AIS data corresponding to the target node, and extracting the AIS data which meets the ship type requirement as target data.
8. An AIS data extraction system based on spatial index, comprising:
The AIS database construction module is used for constructing an AIS database, and each AIS data comprises longitude and latitude data; constructing a multi-layer data structure for an AIS database, wherein the multi-layer data structure comprises a plurality of nodes, the nodes are distributed in a multi-layer mode, and any node comprises a corresponding longitude and latitude range;
The query module is used for acquiring a first longitude and latitude interval;
the retrieval module is used for traversing the plurality of layers of nodes layer by layer downwards according to the first longitude and latitude interval until a target node matched with the first longitude and latitude interval is found out;
And the extraction module is used for extracting the AIS data corresponding to the target node as target data.
9. A spatial index based AIS data extraction system comprising a memory and a processor, the memory storing a program or instruction executable on the processor, the program or instruction when executed by the processor implementing the steps of the spatial index based AIS data extraction method according to any one of claims 1 to 7.
10. A readable storage medium, having stored thereon a program and/or instructions which when executed by a processor implement the steps of the spatial index based AIS data extraction method according to any one of claims 1 to 7.
CN202410370768.1A 2024-03-29 AIS data extraction method, system and readable storage medium based on spatial index Active CN117972008B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202410370768.1A CN117972008B (en) 2024-03-29 AIS data extraction method, system and readable storage medium based on spatial index

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202410370768.1A CN117972008B (en) 2024-03-29 AIS data extraction method, system and readable storage medium based on spatial index

Publications (2)

Publication Number Publication Date
CN117972008A true CN117972008A (en) 2024-05-03
CN117972008B CN117972008B (en) 2024-06-28

Family

ID=

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070250477A1 (en) * 2006-04-25 2007-10-25 Google Inc. Ranking and Clustering of Geo-Located Objects
CN103198151A (en) * 2013-04-24 2013-07-10 北京航空航天大学 Index inquiry system and method for running information of regional urban buses
CN112347217A (en) * 2020-11-09 2021-02-09 中国船舶重工集团公司第七二四研究所 Real-time plotting method based on massive data layering
CN113553682A (en) * 2021-07-12 2021-10-26 大连海事大学 Data-driven multi-level ship route network construction method
WO2022068417A1 (en) * 2020-09-29 2022-04-07 南京北斗创新应用科技研究院有限公司 Vessel loitering detection method based on ais data

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070250477A1 (en) * 2006-04-25 2007-10-25 Google Inc. Ranking and Clustering of Geo-Located Objects
CN103198151A (en) * 2013-04-24 2013-07-10 北京航空航天大学 Index inquiry system and method for running information of regional urban buses
WO2022068417A1 (en) * 2020-09-29 2022-04-07 南京北斗创新应用科技研究院有限公司 Vessel loitering detection method based on ais data
CN112347217A (en) * 2020-11-09 2021-02-09 中国船舶重工集团公司第七二四研究所 Real-time plotting method based on massive data layering
CN113553682A (en) * 2021-07-12 2021-10-26 大连海事大学 Data-driven multi-level ship route network construction method

Similar Documents

Publication Publication Date Title
CN109992636B (en) Space-time coding method, space-time index and query method and device
CN106960037B (en) A kind of distributed index the resources integration and share method across intranet and extranet
CN104199986B (en) Vector data space index method based on hbase and geohash
CN109635068A (en) Mass remote sensing data high-efficiency tissue and method for quickly retrieving under cloud computing environment
CN101324896B (en) Method for storing and searching vector data and management system thereof
CN110147377A (en) General polling algorithm based on secondary index under extensive spatial data environment
CN111782742A (en) Large-scale geographic space data oriented storage and retrieval method and system thereof
CN109492060A (en) A kind of map tile storage method based on MBTiles
CN111552694B (en) Self-adaptive geospatial grid indexing method
CN110175175A (en) Secondary index and range query algorithm between a kind of distributed space based on SPARK
CN108009265B (en) Spatial data indexing method in cloud computing environment
CN106796589A (en) The indexing means and system of spatial data object
CN113626448B (en) HBase distributed storage-based space vector data indexing and query method
CN112035586A (en) Spatial range query method based on extensible learning index
CN106484815B (en) A kind of automatic identification optimization method based on mass data class SQL retrieval scene
CN116775661A (en) Big space data storage and management method based on Beidou grid technology
CN113934686A (en) Distributed multi-level spatial index method for massive airborne laser point clouds
CN113407542B (en) Urban road network vehicle travel track retrieval method and system
CN116860905A (en) Space unit coding generation method of city information model
CN117972008B (en) AIS data extraction method, system and readable storage medium based on spatial index
CN117972008A (en) AIS data extraction method, system and readable storage medium based on spatial index
CN117009411A (en) Method, device and computer readable storage medium for meshing space storage and indexing based on point cloud data
CN109299106B (en) Data query method and device
CN108595588B (en) Scientific data storage association method
CN115858589A (en) Hive number bin dynamic intelligent partitioning method and system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant