CN113282582B - Efficient storage method and system for ship position data - Google Patents
Efficient storage method and system for ship position data Download PDFInfo
- Publication number
- CN113282582B CN113282582B CN202110560023.8A CN202110560023A CN113282582B CN 113282582 B CN113282582 B CN 113282582B CN 202110560023 A CN202110560023 A CN 202110560023A CN 113282582 B CN113282582 B CN 113282582B
- Authority
- CN
- China
- Prior art keywords
- position data
- ship position
- information
- data
- ship
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2228—Indexing structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2282—Tablespace storage structures; Management thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/061—Improving I/O performance
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0638—Organizing or formatting or addressing of data
- G06F3/064—Management of blocks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/0671—In-line storage system
- G06F3/0683—Plurality of storage devices
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Human Computer Interaction (AREA)
- Databases & Information Systems (AREA)
- Software Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides a high-efficiency storage method and a high-efficiency storage system for ship position data, wherein the method comprises the following steps: s1, acquiring ship position data, wherein the ship position data comprises ship information and AIS information; s2, identifying unique identification data in the ship position data, and dividing the ship position data into a plurality of block data sets according to the unique identification data; s3, respectively transmitting the block data sets corresponding to the unique identification data to different storage devices, and storing the data in the block data sets into different sectors of the storage devices according to time information in ship position data in each storage device; s4, generating a catalog index table according to the actual storage positions of all ship position data; s5, inquiring and updating the ship position data based on the catalog index table. The invention can effectively improve the storage, reading and writing efficiency of the ship position data and does not increase the complexity in use.
Description
Technical Field
The invention relates to the technical field of ship position data storage, in particular to a high-efficiency storage method and system for ship position data.
Background
In the maritime transportation industry, in order to facilitate management, a shipping company needs to master information such as real-time position, navigation state, navigation line and the like of a flag ship, and along with rapid development of network transmission technology and satellite positioning technology, a ship located on the sea can rapidly return corresponding data to the background of the shipping company and store the corresponding data through a database. For large shipping companies, the number of vessels under the flag management may reach thousands, when the vessel data is periodically updated, the data in the database can become huge after being accumulated for a certain time, when the traditional relational database list table is adopted to store the data, the reading and writing efficiency is lower, the list table file can become huge along with the time growth, and the efficiency of a user can be slow when the user searches the data.
Disclosure of Invention
Accordingly, it is an object of the present invention to provide a method and system for efficiently storing ship position data, which overcomes or at least partially solves the above-mentioned problems of the prior art.
The first aspect of the invention provides a high-efficiency storage method of ship position data, which comprises the following steps:
s1, acquiring ship position data, wherein the ship position data comprises ship information and AIS information;
s2, identifying unique identification data in the ship position data, and dividing the ship position data into a plurality of block data sets according to the unique identification data;
s3, respectively transmitting the block data sets corresponding to the unique identification data to different storage devices, and storing the data in the block data sets into different sectors of the storage devices according to time information in ship position data in each storage device;
s4, generating a catalog index table according to the actual storage positions of all ship position data;
s5, inquiring and updating the ship position data based on the catalog index table.
Further, in the step S5, updating the ship position data based on the directory index table specifically includes the steps of:
s501, identifying unique identification data in the ship position data when the latest ship position data is periodically acquired, and polling a storage address corresponding to the unique identification data in a directory index table according to the unique identification data;
s502, transmitting the latest ship position data to storage equipment corresponding to the storage address in the query result.
Further, in the step 502, the transmitting the latest ship position data to the storage device corresponding to the storage address in the query result specifically includes:
s5021, writing the latest ship position data into a cache area of the storage device;
and S5022, when new ship position data is acquired in the next period, the ship position data in the current cache region of the storage device to be written into is written into the corresponding sector of the storage device according to the time information, and the newly acquired ship position data is written into the cache region of the storage device.
Further, the steps between the step S4 and the step S5 further include:
s41, acquiring historical AIS information, acquiring corresponding historical route information based on the historical AIS information, and extracting historical sailing state information;
s42, analyzing and extracting blocking information affecting the historical sailing state information in the historical route information;
s43, constructing a classifier according to the obstruction information and the historical sailing state information, and generating a training model;
s44, taking part of historical route information as a training sample, and inputting the training sample into a training model for iterative training.
Further, before step S5, the method further includes the steps of:
s45, acquiring AIS information of all ship position data stored in the storage equipment according to the unique identification information of the ship position data, and acquiring current route information of the ship based on the AIS information;
s46, analyzing and extracting the obstruction information in the current route information of the ship;
s47, inputting the obstruction information into a training model to obtain the prediction information of the navigation state of the corresponding ship in a period of time in the future;
s48, screening out ships with predicted berthing states in a future period of time, and temporarily shielding index items of corresponding ship position data in the directory index table based on the unique identification codes of the screened ships.
Further, in the step S501, when the memory address corresponding to the unique identification data in the directory index table is polled according to the unique identification data, the temporarily masked index entry is skipped.
Further, when judging that the temporarily masked index item meets the preset condition, unmasking the index item, and judging whether the preset condition is met or not specifically includes: and judging whether the current real-time is out of limit or not by limiting the duration time of the predicted berthing state of the ship corresponding to the index item.
A second aspect of the present invention provides an efficient storage system for ship position data, the system comprising:
the ship position data comprises ship information and AIS information;
the identifying module is used for identifying the unique identifying data in the ship position data and dividing the ship position data into a plurality of block data sets according to the unique identifying data;
the storage module is used for respectively transmitting the block data sets corresponding to the unique identification data to different storage devices according to the unique identification data, and storing the data in the block data sets into different sectors of the storage devices according to time information in ship position data in each storage device;
the index module is used for generating a catalog index table according to the actual storage positions of all ship position data;
and the inquiring and updating module is used for inquiring and updating the ship position data based on the catalog index table.
Compared with the prior art, the invention has the beneficial effects that:
according to the efficient storage method for the ship position data, the ship position data of different ships are divided into a plurality of block data sets based on the unique identification data, the block data sets corresponding to the unique identification data are respectively transmitted to different storage devices according to the unique identification data, the data in the block data sets are stored in different sectors of the storage devices according to time information, IO read-write performance can be improved when the data are read, and meanwhile, a catalog index table is generated according to the actual storage positions of all the ship position data, so that the data retrieval and updating efficiency can be improved without increasing complexity in use, namely the storage and read-write efficiency of the ship position data can be effectively improved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only preferred embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a schematic overall flow chart of a method for efficient storage of ship position data according to an embodiment of the present invention.
Fig. 2 is a schematic diagram of an overall structure of a high-efficiency storage system for ship position data according to an embodiment of the present invention.
In the figure, a first acquisition module, a 2 identification module, a 3 storage module, a 4 index module and a 5 query and update module are shown.
Detailed Description
The principles and features of the present invention are described below with reference to the drawings, the illustrated embodiments are provided for the purpose of illustrating the invention and are not to be construed as limiting the scope of the invention.
Referring to fig. 1, an embodiment of the present invention provides a method for efficiently storing ship position data, the method including the steps of:
s1, acquiring ship position data, wherein the ship position data comprises ship information and AIS information.
Illustratively, the ship information may include data of ship name, call sign, MMSI, ship type, ship length, width, height, load, construction time, ship harbor, number of ship mouths, upper limit of draft, etc.; the AIS information can comprise data such as ship position longitude and latitude, sailing state, sailing speed, course, real-time draft, starting port, destination port, pre-resisting time, information updating time and the like.
S2, identifying unique identification data in the ship position data, and dividing the ship position data into a plurality of block data sets according to the unique identification data.
The unique identification data is used to uniquely identify a ship, which may be a string of characters, such as call signs, MMSI, etc., or other forms of identifiable content, for example. The dividing of the ship position data into a plurality of block data sets according to the unique identification data is specifically as follows: and dividing the data in the ship position data into a plurality of block data sets according to the unique identification data corresponding to the ship to which the data belongs, wherein the data in each block data set corresponds to the same ship.
And S3, respectively transmitting the block data sets corresponding to the unique identification data to different storage devices, and storing the data in the block data sets into different sectors of the storage devices according to time information in the ship position data in each storage device.
The storage device may be a block device such as a mechanical hard disk, a solid state hard disk, a usb disk, or an SD card, for example. The time information is the updating time of the ship position data, and the data with different updating times are respectively stored in different sectors of the storage device.
S4, generating a catalog index table according to the actual storage positions of all the ship position data.
S5, inquiring and updating the ship position data based on the catalog index table.
Compared with a traditional single-table storage mode with low reading and writing efficiency of a relational database, the high-efficiency storage method for ship position data provided by the embodiment stores the ship position data of different ships into different sectors of different storage devices according to unique identification data and time information, so that IO reading and writing performance is improved, the problems of overlarge single file and low retrieval efficiency are avoided in physical storage due to the fact that the ship position data are logically identical in table and directory index is added for optimization, complexity in use is not increased in the fact that the ship position data are logically identical in table, and storage and retrieval efficiency of the ship position data with very high data volume increasing speed in quick update can be effectively improved.
As an optional implementation manner of this embodiment, in step S5, updating the ship position data based on the directory index table specifically includes the steps of:
s501, identifying unique identification data in the ship position data every time the latest ship position data is periodically acquired, and polling a storage address corresponding to the unique identification data in the directory index table according to the unique identification data.
For example, the ship position data may be periodically acquired from each ship through a network or a radio, and the acquisition period of the ship position data may be set according to actual needs, for example, the ship position data is acquired once for 15 seconds. When the latest ship position data is acquired, the catalog index table is polled according to the unique identification data in the ship position data, so that the specific storage equipment in which the ship position data of the ship corresponding to the unique identification data is stored is determined.
S502, transmitting the latest ship position data to storage equipment corresponding to the storage address in the query result.
As a further optional implementation manner, in step 502, the transmitting the latest ship position data to the storage device corresponding to the storage address in the query result specifically includes:
and S5021, writing the latest ship position data into a cache area of the storage device.
And S5022, when new ship position data is acquired in the next period, the ship position data in the current cache region of the storage device to be written into is written into the corresponding sector of the storage device according to the time information, and the newly acquired ship position data is written into the cache region of the storage device.
According to the high-efficiency storage method for the ship position data, on one hand, the latest ship position data are updated to the cache region of the storage device, so that new data with relatively more queried probability and times can be quickly read and written, on the other hand, when the data are periodically updated, the data stored in the cache region can be transferred into the sector of the storage device for storage, and the latest data are written into the cache region of the storage device, so that the latest ship position data of corresponding ships are stored in the cache region of all the storage devices, and the IO read-write performance of the ship position data is further improved.
As an optional implementation manner of this embodiment, the step S4 and the step S5 further include the steps of:
s41, acquiring historical AIS information, acquiring corresponding historical route information based on the historical AIS information, and extracting historical sailing state information.
The obtaining of the corresponding historical route information based on the historical AIS information may specifically be obtaining the common route information between two places through the information of the initial port and the destination port in the historical AIS information, or may also be generating the corresponding historical route information according to the longitude and latitude information of the continuous position in a period of historical time. The historical AIS information should contain a plurality of pieces of historical AIS information of different ships, so that a sample has a certain scale, and the historical route information corresponding to the historical AIS information should cover a main route commonly used in the global scope.
S42, analyzing and extracting the blocking information affecting the historical sailing state information in the historical sailing route information.
Illustratively, the blocking information is specific to related information capable of significantly affecting the normal sailing state of the ship, such as the state of the ship itself, including equipment operation conditions, fuel reserves and the like; or weather conditions in the forward voyage segment; or route control information such as canal navigation control information, etc.
S43, constructing a classifier according to the obstruction information and the historical sailing state information, and generating a training model.
S44, taking part of historical route information as a training sample, and inputting the training sample into a training model for iterative training.
The training model established in this embodiment aims at predicting the ship navigation state information in a future period according to the blocking information, so that a classifier needs to be constructed according to the historical blocking information and the corresponding ship historical navigation state information, so that the training model can predict the ship navigation state most likely to be caused by the training model according to the blocking information. In the iterative training process, the prediction result output by the training model is compared with the actual historical sailing state information, and is adjusted according to the comparison result, so that the prediction result of the training model can be in line with the sailing state change of the ship caused by the influence of various obstruction information in the actual situation as much as possible.
As a further alternative embodiment, the method further comprises the step of, before step S5:
s45, acquiring AIS information of the ship according to the unique identification information of all ship position data stored in the storage equipment, and acquiring current route information of the ship based on the AIS information.
S46, analyzing and extracting the obstruction information in the current route information of the ship.
S47, inputting the obstruction information into the training model to obtain the predicted sailing state information of the corresponding ship in a period of time in the future.
S48, screening out ships with predicted berthing states in a future period of time, and temporarily shielding index items of corresponding ship position data in the directory index table based on the unique identification codes of the screened ships.
The method comprises the steps of inquiring the storage position of the ship position data of a corresponding ship through a catalog index table when the ship is in a berthing state, wherein the ship position data of the corresponding ship is basically unchanged when the ship is in the berthing state, and comparing whether the newly received ship position data belongs to the berthed ship or not when the catalog index table is polled, so that the efficiency of polling the catalog index table is still to be further improved.
Specifically, in step S501, when the memory address corresponding to the unique identification data in the directory index table is polled according to the unique identification data, the temporarily masked index entry is skipped, thereby achieving the effect of improving the speed of polling the directory index table.
In addition, when the index item which is temporarily masked is judged to meet the preset condition, the masking of the index item is released. Judging whether the preset condition is met or not specifically comprises the following steps: and judging whether the current real-time is out of limit or not by limiting the duration time of the predicted berthing state of the ship corresponding to the index item.
In some embodiments, if the index item of the ship position data corresponding to the newly acquired ship position data is not polled in the directory index table according to the newly acquired ship position data under the condition of skipping the temporarily shielded index item, the shielding of the temporarily shielded index item is released, the unmasked index items are polled for the second time, whether the index item of the ship corresponding to the new ship position data exists or not is searched, if not, the newly acquired ship position data is stored in the new storage device, and the index item of the directory index table is updated according to the storage position of the newly acquired ship position data. So that the ship position data of the vessels predicted to be berthed can be updated when the ship position data of the vessels suddenly is updated.
Based on the same inventive concept as the previous embodiments, another embodiment of the present invention provides an efficient storage system of ship position data.
Referring to fig. 2, the system includes:
the first acquiring module 1 is configured to acquire ship position data, where the ship position data includes ship information and AIS information.
And the identification module 2 is used for identifying the unique identification data in the ship position data and dividing the ship position data into a plurality of block data sets according to the unique identification data.
And the storage module 3 is used for respectively transmitting the corresponding block data sets to different storage devices according to the unique identification data, and storing the data in the block data sets into different sectors of the storage devices according to the time information in the ship position data in each storage device.
And the index module 4 is used for generating a catalog index table according to the actual storage positions of all the ship position data.
And the inquiring and updating module 5 is used for inquiring and updating the ship position data based on the directory index table.
Optionally, the querying and updating module 5 specifically includes:
the polling sub-module is used for identifying unique identification data in the ship position data when the latest ship position data is periodically acquired, and polling a storage address corresponding to the unique identification data in the directory index table according to the unique identification data;
and the storage sub-module is used for transmitting the latest ship position data to the storage equipment corresponding to the storage address in the query result.
Optionally, the storage submodule is specifically configured to write the latest ship position data into the cache area of the storage device, and when new ship position data is acquired in the next period, write the ship position data in the current cache area of the storage device to be written into the corresponding sector of the storage device according to the time information, and write the newly acquired ship position data into the cache area of the storage device.
Optionally, the system further comprises:
the second acquisition module is used for acquiring the historical AIS information, acquiring corresponding historical route information based on the historical AIS information and extracting historical sailing state information;
the first analysis module is used for analyzing and extracting blocking information affecting the historical sailing state information in the historical sailing line information;
the model generation module is used for constructing a classifier according to the obstruction information and the historical sailing state information and generating a training model;
and the iterative training module is used for taking part of historical route information as a training sample and inputting the training sample into the training model for iterative training.
The third acquisition module is used for acquiring AIS information of all ship position data stored in the storage equipment according to the unique identification information of the ship position data, and acquiring current route information of the ship based on the AIS information;
the second analysis module is used for analyzing and extracting the obstruction information in the current route information of the ship;
the prediction module is used for inputting the obstruction information into the training model to obtain the prediction information of the navigation state of the corresponding ship in a period of time in the future;
the screening module is used for screening out ships with predicted berthing states in a future period of time, and temporarily screening index items of corresponding ship position data in the directory index table based on the screened unique identification codes of the ships.
Optionally, the polling submodule is specifically further configured to skip the temporarily masked index item when polling the storage address corresponding to the unique identification data in the directory index table according to the unique identification data.
Optionally, the system further includes a mask release module, configured to release masking of the index item when it is determined that the temporarily masked index item meets a preset condition, and determining whether the preset condition is met is specifically: and judging whether the current real-time is out of limit or not by limiting the duration time of the predicted berthing state of the ship corresponding to the index item.
Optionally, the system further includes a secondary polling module, where the secondary polling module is configured to, when the index item of the ship position data corresponding to the newly acquired ship position data is not polled in the directory index table according to the newly acquired ship position data in the case of skipping the temporarily shielded index item, unblock the temporarily shielded index item, and perform secondary polling on the unblocked index items, find whether there is an index item of the ship corresponding to the new ship position data, and if not, store the newly acquired ship position data in the new storage device, and update the index item of the directory index table according to the storage location thereof.
The above system embodiments are used for executing the foregoing method embodiments, and the working principles and beneficial effects thereof may refer to the foregoing method embodiments and are not described herein in detail.
The foregoing description of the preferred embodiments of the invention is not intended to limit the invention to the precise form disclosed, and any such modifications, equivalents, and alternatives falling within the spirit and scope of the invention are intended to be included within the scope of the invention.
Claims (6)
1. A method for efficient storage of ship position data, the method comprising the steps of:
s1, acquiring ship position data, wherein the ship position data comprises ship information and AIS information;
s2, identifying unique identification data in the ship position data, and dividing the ship position data into a plurality of block data sets according to the unique identification data;
s3, respectively transmitting the block data sets corresponding to the unique identification data to different storage devices, and storing the data in the block data sets into different sectors of the storage devices according to time information in ship position data in each storage device;
s4, generating a catalog index table according to the actual storage positions of all ship position data;
s5, inquiring and updating ship position data based on the catalog index table;
in the step S5, updating the ship position data based on the directory index table specifically includes the steps of:
s501, identifying unique identification data in the ship position data when the latest ship position data is periodically acquired, and polling a storage address corresponding to the unique identification data in a directory index table according to the unique identification data;
s502, transmitting the latest ship position data to storage equipment corresponding to a storage address in the query result;
the steps between the step S4 and the step S5 further comprise the steps of:
s41, acquiring historical AIS information, acquiring corresponding historical route information based on the historical AIS information, and extracting historical sailing state information;
s42, analyzing and extracting blocking information affecting the historical sailing state information in the historical route information;
s43, constructing a classifier according to the obstruction information and the historical sailing state information, and generating a training model;
s44, taking part of historical route information as a training sample, and inputting the training sample into a training model for iterative training.
2. The efficient storage method of ship position data according to claim 1, wherein in step 502, the latest ship position data is transmitted to the storage device corresponding to the storage address in the query result, and specifically includes:
s5021, writing the latest ship position data into a cache area of the storage device;
and S5022, when new ship position data is acquired in the next period, the ship position data in the current cache region of the storage device to be written into is written into the corresponding sector of the storage device according to the time information, and the newly acquired ship position data is written into the cache region of the storage device.
3. The efficient storage method of ship position data according to claim 1, further comprising the step of, before step S5:
s45, acquiring AIS information of all ship position data stored in the storage equipment according to the unique identification information of the ship position data, and acquiring current route information of the ship based on the AIS information;
s46, analyzing and extracting the obstruction information in the current route information of the ship;
s47, inputting the obstruction information into a training model to obtain the prediction information of the navigation state of the corresponding ship in a period of time in the future;
s48, screening out ships with predicted berthing states in a future period of time, and temporarily shielding index items of corresponding ship position data in the directory index table based on the unique identification codes of the screened ships.
4. A method for efficiently storing ship position data according to claim 3, wherein in the step S501, when the memory address corresponding to the unique identification data in the directory index table is polled according to the unique identification data, the temporarily masked index entry is skipped.
5. The efficient storage method of ship position data according to claim 3, wherein when the temporarily masked index item is judged to satisfy a preset condition, the masking of the index item is released, and whether the preset condition is satisfied is judged to be specifically: and judging whether the current real-time is out of limit or not by limiting the duration time of the predicted berthing state of the ship corresponding to the index item.
6. An efficient storage system for ship position data, said system comprising:
the ship position data comprises ship information and AIS information;
the identifying module is used for identifying the unique identifying data in the ship position data and dividing the ship position data into a plurality of block data sets according to the unique identifying data;
the storage module is used for respectively transmitting the block data sets corresponding to the unique identification data to different storage devices according to the unique identification data, and storing the data in the block data sets into different sectors of the storage devices according to time information in ship position data in each storage device;
the index module is used for generating a catalog index table according to the actual storage positions of all ship position data;
the inquiring and updating module is used for inquiring and updating the ship position data based on the catalog index table;
the query and update module 5 specifically includes:
the polling sub-module is used for identifying unique identification data in the ship position data when the latest ship position data is periodically acquired, and polling a storage address corresponding to the unique identification data in the directory index table according to the unique identification data;
the storage sub-module is used for transmitting the latest ship position data to the storage equipment corresponding to the storage address in the query result;
the system further comprises:
the second acquisition module is used for acquiring the historical AIS information, acquiring corresponding historical route information based on the historical AIS information and extracting historical sailing state information;
the first analysis module is used for analyzing and extracting blocking information affecting the historical sailing state information in the historical sailing line information;
the model generation module is used for constructing a classifier according to the obstruction information and the historical sailing state information and generating a training model;
and the iterative training module is used for taking part of historical route information as a training sample and inputting the training sample into the training model for iterative training.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110560023.8A CN113282582B (en) | 2021-05-21 | 2021-05-21 | Efficient storage method and system for ship position data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110560023.8A CN113282582B (en) | 2021-05-21 | 2021-05-21 | Efficient storage method and system for ship position data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113282582A CN113282582A (en) | 2021-08-20 |
| CN113282582B true CN113282582B (en) | 2023-06-20 |
Family
ID=77280712
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110560023.8A Active CN113282582B (en) | 2021-05-21 | 2021-05-21 | Efficient storage method and system for ship position data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113282582B (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6381605B1 (en) * | 1999-05-29 | 2002-04-30 | Oracle Corporation | Heirarchical indexing of multi-attribute data by sorting, dividing and storing subsets |
| CN101644996A (en) * | 2009-09-25 | 2010-02-10 | 杭州华三通信技术有限公司 | Storage method of index data and storage control device |
| CN103714010A (en) * | 2013-12-24 | 2014-04-09 | 华为技术有限公司 | Storage device write-in method and storage device |
| CN104008212A (en) * | 2014-06-23 | 2014-08-27 | 中国科学院重庆绿色智能技术研究院 | Method for storing IOT time series data related to geographical location information |
| CN106462491A (en) * | 2014-12-31 | 2017-02-22 | 华为技术有限公司 | Method for managing storage data, storage manager and storage system |
| CN106980471A (en) * | 2017-03-27 | 2017-07-25 | 联想(北京)有限公司 | A kind of method and device for the hard disk write performance for improving smart machine |
| CN108281043A (en) * | 2018-03-23 | 2018-07-13 | 上海海事大学 | A kind of ship collision Warning System and method for early warning |
| CN108304144A (en) * | 2018-01-16 | 2018-07-20 | 广州视源电子科技股份有限公司 | Data write-in, read method and system, data read-write system |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104731864B (en) * | 2015-02-26 | 2018-05-29 | 国家计算机网络与信息安全管理中心 | A kind of date storage method of magnanimity unstructured data |
| CN106218831B (en) * | 2016-07-22 | 2018-09-25 | 大连海事大学 | A kind of method and system obtaining Ship Controling behavior based on watercraft AIS track data |
| CN107193869A (en) * | 2017-04-10 | 2017-09-22 | 深圳市智物联网络有限公司 | The method and device of data access in industrial Internet of Things cloud platform |
| US10909121B2 (en) * | 2019-02-22 | 2021-02-02 | Snowflake Inc. | Storing multi-level metadata in database systems |
| CN111399762B (en) * | 2019-11-27 | 2021-06-18 | 杭州海康威视系统技术有限公司 | Data storage method, device and storage system |
-
2021
- 2021-05-21 CN CN202110560023.8A patent/CN113282582B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6381605B1 (en) * | 1999-05-29 | 2002-04-30 | Oracle Corporation | Heirarchical indexing of multi-attribute data by sorting, dividing and storing subsets |
| CN101644996A (en) * | 2009-09-25 | 2010-02-10 | 杭州华三通信技术有限公司 | Storage method of index data and storage control device |
| CN103714010A (en) * | 2013-12-24 | 2014-04-09 | 华为技术有限公司 | Storage device write-in method and storage device |
| CN104008212A (en) * | 2014-06-23 | 2014-08-27 | 中国科学院重庆绿色智能技术研究院 | Method for storing IOT time series data related to geographical location information |
| CN106462491A (en) * | 2014-12-31 | 2017-02-22 | 华为技术有限公司 | Method for managing storage data, storage manager and storage system |
| CN106980471A (en) * | 2017-03-27 | 2017-07-25 | 联想(北京)有限公司 | A kind of method and device for the hard disk write performance for improving smart machine |
| CN108304144A (en) * | 2018-01-16 | 2018-07-20 | 广州视源电子科技股份有限公司 | Data write-in, read method and system, data read-write system |
| CN108281043A (en) * | 2018-03-23 | 2018-07-13 | 上海海事大学 | A kind of ship collision Warning System and method for early warning |
Non-Patent Citations (2)
| Title |
|---|
| Object segmentation and labeling by learning from examples;Yaowu Xu 等;《 IEEE Transactions on Image Processing》;第12卷(第6期);627- 638 * |
| 高可靠数据存储技术研究;邬天恺;《中国优秀硕士学位论文全文数据库 信息科技辑》(第06期);I137-46 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113282582A (en) | 2021-08-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9697250B1 (en) | Systems and methods for high-speed searching and filtering of large datasets | |
| CN109902100A (en) | Report form inquiring method, device and storage medium | |
| CN101826107B (en) | Hash data processing method and device | |
| JP2010128806A (en) | Information analyzing device | |
| US20200334246A1 (en) | Information processing device, combination condition generation method, and combination condition generation program | |
| CN111914539A (en) | Channel announcement information extraction method and system based on BilSTM-CRF model | |
| EP0744702A1 (en) | Information searching apparatus for searching text to retrieve character streams agreeing with a key word | |
| CN102567421A (en) | Document retrieval method and device | |
| US6973446B2 (en) | Knowledge finding method | |
| CN102541925A (en) | Method and device for rapidly storing and retrieving detailed tickets | |
| CN108932271A (en) | A kind of file management method and device | |
| CN113282582B (en) | Efficient storage method and system for ship position data | |
| CN114185903A (en) | Ship information searching method | |
| Li et al. | Geospatial data mining on the web: Discovering locations of emergency service facilities | |
| CN107247716B (en) | Method and device for increasing electronic eye information, navigation chip and server | |
| KR20170104209A (en) | Apparatus and method for trajectory processing of vessel | |
| CN108052629A (en) | A kind of quick extra large land determination methods based on high accuracy DEM data | |
| KR102536105B1 (en) | Event collection management system and method for maritime traffic control | |
| Qi et al. | Ship trajectory data compression algorithms for Automatic Identification System: Comparison and analysis | |
| CN111177999B (en) | Text box-based text display method and device and computer equipment | |
| Huang et al. | Processing continuous K-nearest skyline query with uncertainty in spatio-temporal databases | |
| CN114827132B (en) | Ship traffic file transmission control method, system, device and storage medium | |
| CN116993245B (en) | Regional waterway traffic measuring and calculating method and device, electronic equipment and storage medium | |
| CN100461806C (en) | Voice value-added service data information processing method | |
| CN110826488A (en) | Image identification method and device for electronic document and storage equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |