CN112612777B - Ocean data management and visualization system and method based on MySQL database - Google Patents
Ocean data management and visualization system and method based on MySQL database Download PDFInfo
- Publication number
- CN112612777B CN112612777B CN202011547626.6A CN202011547626A CN112612777B CN 112612777 B CN112612777 B CN 112612777B CN 202011547626 A CN202011547626 A CN 202011547626A CN 112612777 B CN112612777 B CN 112612777B
- Authority
- CN
- China
- Prior art keywords
- data
- database
- ocean
- information
- index
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 30
- 238000013523 data management Methods 0.000 title claims abstract description 13
- 238000013079 data visualisation Methods 0.000 title claims abstract description 11
- 238000003860 storage Methods 0.000 claims abstract description 26
- 230000000007 visual effect Effects 0.000 claims description 51
- 230000003993 interaction Effects 0.000 claims description 30
- 238000002474 experimental method Methods 0.000 claims description 29
- 230000008569 process Effects 0.000 claims description 18
- 230000005540 biological transmission Effects 0.000 claims description 12
- 238000012800 visualization Methods 0.000 claims description 12
- 238000010586 diagram Methods 0.000 claims description 7
- 238000005070 sampling Methods 0.000 claims description 6
- 241001137251 Corvidae Species 0.000 claims description 4
- 238000004587 chromatography analysis Methods 0.000 claims description 4
- 230000008878 coupling Effects 0.000 claims description 4
- 238000010168 coupling process Methods 0.000 claims description 4
- 238000005859 coupling reaction Methods 0.000 claims description 4
- 238000001514 detection method Methods 0.000 claims description 4
- 238000003384 imaging method Methods 0.000 claims description 4
- 230000008676 import Effects 0.000 claims description 4
- 235000015108 pies Nutrition 0.000 claims description 4
- 230000003068 static effect Effects 0.000 claims description 4
- 230000008859 change Effects 0.000 claims description 3
- 238000012952 Resampling Methods 0.000 claims description 2
- 238000005266 casting Methods 0.000 claims description 2
- 238000012546 transfer Methods 0.000 claims description 2
- 230000009466 transformation Effects 0.000 claims description 2
- 238000007794 visualization technique Methods 0.000 claims description 2
- 230000015572 biosynthetic process Effects 0.000 claims 1
- 238000003786 synthesis reaction Methods 0.000 claims 1
- 238000013500 data storage Methods 0.000 abstract description 3
- 230000007547 defect Effects 0.000 abstract description 2
- 230000006870 function Effects 0.000 description 9
- 238000007726 management method Methods 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 3
- 230000002085 persistent effect Effects 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 230000002452 interceptive effect Effects 0.000 description 2
- 230000011218 segmentation Effects 0.000 description 2
- 238000000926 separation method Methods 0.000 description 2
- 238000012876 topography Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 230000002194 synthesizing effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/13—File access structures, e.g. distributed indices
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/14—Details of searching files based on file metadata
- G06F16/148—File search processing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2282—Tablespace storage structures; Management thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/26—Visual data mining; Browsing structured data
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
- Y02A90/10—Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Library & Information Science (AREA)
- Processing Or Creating Images (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application discloses a marine data management and visualization system and method based on a MySQL database, and provides a marine data self-adaptive storage method and a marine data visualization method based on MySQL aiming at the characteristics of marine data and the defects of the prior art in marine data storage, management and display.
Description
Technical Field
The application belongs to the field of ocean data management and visualization, and particularly relates to an ocean data management and visualization system and method based on a MySQL database.
Background
In marine experimental science, the management of experimental data goes through the stages from manual management, file management to database management. The ocean data comprises environmental data and experimental acquisition data, and has the characteristics of multiple sources, large scale, space-time dependence, homology and isomerism and the like. In a common database storage mode, the current MongoDB-based non-relational database can improve the read-write speed and enhance the expansibility in web development in the storage of single or less types of large-scale ocean data; however, for more kinds of ocean data, the non-relational database is not suitable for persistent storage of massive data and more complex query, so the relational database represented by MySQL still should be selected for the needs of persistent storage and ocean data storage suitable for cross-type experimental analysis retrieval data.
Aiming at the characteristics of the ocean data, an ocean data organization scheme based on MySQL is needed, and the key and difficult points are as follows: the read-write efficiency is improved on the basis of persistent storage; an integrated data format storage scheme is established, so that the speed reduction caused by the joint search of multiple fields in the process of search and visualization is avoided; and performing self-adaptive vertical segmentation and horizontal segmentation according to the type, the size and the basic information of the imported ocean data file.
The visualization technology is used for visually presenting the ocean data, and the common visualization technology mainly comprises the steps of representing the quantity, the duty ratio or the change trend of the data through a histogram, a pie chart, a line graph and the like; and modeling and simulating the reality data through a model or a live-action. Based on the strong space-time dependence of the ocean data, the common line graph and other modes can provide an analysis flow of part of experimental data, but the spatial characteristics of the ocean data and the time-variation-dependent procedural property cannot be reflected.
On the basis, the current ocean GIS software has single visual setting and lacks a joint display function; the multi-view display function is used for modeling and simulating the model or the real scene, and lacks the multi-view function when reflecting the data characteristics; there is a lack of dynamic functionality reflecting its procedural nature.
Disclosure of Invention
Aiming at the defects and practical needs in the prior art, the application provides a ocean data management and visualization system and method based on a MySQL database.
The application firstly provides a marine data management and visualization system based on a MySQL database, which comprises the following components:
a marine data index library for storing marine data index information;
the ocean database is used for carrying out three-dimensional coding on ocean data and storing the ocean data in the corresponding database and data table;
a data interface comprising a local interface for obtaining a local data file and a remote interface for data transmission with a server;
the search module comprises an index query unit and a marine data query unit; the index inquiry unit is used for searching whether corresponding index information exists in the ocean data index database according to index information to be inquired, and the ocean data inquiry unit is used for searching corresponding data content in the ocean database according to inquiry information input by a user;
the data importing module acquires a local data file or a server data file received by the data interface; analyzing the data to obtain index information corresponding to the data file, calling a search module to search an ocean data index database, and storing the data to the ocean database according to an index search result or storing the data file by newly building a data table in the ocean database; after the data import module stores the data file, the ocean data index library updates the index entry;
the visual module comprises a visual interaction unit, and the visual interaction unit calls the retrieval module according to retrieval conditions and display types input by a user to obtain data content meeting the conditions and performs visual display of the data through the visual interaction unit;
and the data transmission module is used for acquiring a remote data file through a remote interface or transmitting a local data file outwards.
Preferably, the ocean database is divided into a plurality of regional sub-databases according to the geographic position of the data, each regional sub-database is divided into a plurality of annual sub-databases according to the year of the data, each annual sub-database is divided into a plurality of sub-databases according to the number, each sub-database is provided with an information table, and each sub-database comprises a plurality of data tables.
The application also provides a marine data management and visualization method of the system, which can realize the storage, visualization and data transmission of marine data;
the storing of the ocean data comprises the following steps:
1.1 A data interface receives the imported local data file or remote data file, a data importing module analyzes the obtained data file to obtain index information corresponding to the data file,
1.2 Searching the ocean data index database according to the index information by the searching module, and if the corresponding index information is searched, entering the step 1.4); if no corresponding index information is searched, the step 1.3 is entered;
1.3 Creating a sub-database in the ocean database according to the index information, and creating an information table in the sub-database, wherein the information table is used for recording a data table contained in the sub-database and stored data information; and go to step 1.4)
1.4 Inquiring information tables in the sub-database, and if the existing data tables in the sub-database can be matched with the data types and time information of the data files to be stored, storing the data contents into the corresponding data tables; if the data cannot be matched, a new data table is created to store the data content;
1.5 According to the information determined by the steps, combining the data type code, the area code, the year, month and day and the line number of each line of data to generate a data code, wherein the data code is unique data code for each line of data, and the data code is used as a main key index in a data table and can uniquely determine the position where the data is stored;
1.6 According to the storage condition of the data, updating index entries by the ocean data index base;
the visualization of the marine data comprises the steps of:
2.1 The visual interaction unit obtains the search condition of the user, and invokes the search module to firstly query the ocean data index library to obtain the database information meeting the condition;
2.2 The visual interaction unit displays the database information meeting the conditions in the electronic chart;
2.3 When the user inputs the refined search condition, the visual interaction unit generates the coding range of the data file according to the refined search condition, and searches the data file conforming to the coding range from the database conforming to the condition of the step 2.1); the visual interaction unit acquires a data file and performs visual display according to the data type;
2.4 For the temporary data file to be displayed, the data interface acquires the temporary data file and the type information thereof, and the data interface directly performs visual display at the corresponding position in the electronic chart according to the index information and the data type information thereof.
The ocean data, namely ocean experimental process data, comprises: sound field modeling experiment, deep sea acoustic chromatography experiment, submarine acoustic imaging experiment, sound field data coupling assimilation experiment, deep sea mesoscale process detection experiment, PIES deep sea dynamics parameter inversion experiment input data and output data.
The marine experiment process data comprises a data head, wherein the data head comprises a parameter type, a length and data content length information, and the parameter type comprises: the sending and receiving addresses, data type, data content length, checksum characterized by IP address and port number.
Preferably, the transmission of the marine experimental process data and instructions is realized based on the TCP/IP protocol.
The beneficial effects of the application are as follows:
according to the structural characteristics of the ocean data, the database and the table are divided, and compared with a common storage mode, the data storage content is adaptively divided through time information, space information, type information and the like acquired from the data according to the structural characteristics of the ocean data; and carrying out three-dimensional coding on the information, and combining the information with the position stored in the data table to generate a three-dimensional coding scheme which is used as a unique identifier of the data in the whole system. Based on the method, the code can be generated according to the search condition in the search process, so that the joint search in multiple tables under multiple conditions, the possible complex search structure and the like are avoided, and the search efficiency is improved.
The time dimension dynamic display scheme and the two-dimensional and three-dimensional combined display scheme are provided, the connection between display data and an observation visual angle coordinate system is established, and a multi-dimensional observation visual angle is provided.
Drawings
FIG. 1 is a system block diagram of a MySQL database-based marine data management and visualization system of the present application.
FIG. 2 is a flow chart of the data file storage of the present application;
FIG. 3 is a schematic diagram of a visualization flow of marine data of the system of the present application;
FIG. 4 is a schematic diagram of an operation interface of the visual interactive unit of the present application;
FIG. 5 is a schematic diagram of a database operation panel according to the present application;
FIG. 6 is a schematic diagram of an operation panel of the history data playback function of the present application.
Detailed Description
The application is further described below with reference to the accompanying drawings.
As shown in fig. 1, the ocean data management and visualization system based on MySQL database of the present application mainly comprises: the system comprises a marine data index database, a marine database, a data interface, a retrieval module, a data import module, a visualization module and a data transmission module.
The system can quickly locate the position of the data file to be searched in the ocean database through the index information contained in the data file or the index information contained in the search information input by a user. In a specific embodiment of the present application, the index information of the ocean data index library includes longitude and latitude, area code, database name, data type and data quantity.
The ocean database is used for carrying out three-dimensional coding on ocean data and storing the ocean data in corresponding sub-databases and data tables; in one embodiment of the application, the geographic area of the data source is pre-partitioned and marked with a three-digit area code. The corresponding ocean database is divided into a plurality of regional sub-databases according to the geographic position of the data.
According to the characteristics of ocean data, the stored data of different years in the same region are distinguished, so that year + number is adopted for identification, the year is the experiment proceeding time, the number is sequentially increased from 1, and the number is divided according to the number. That is, in a specific embodiment of the present application, each regional sub-library is further divided into several annual sub-libraries according to the year of the data, and each annual sub-library is divided into several sub-libraries according to the number.
In one embodiment of the application, each sub-library has a table of information, and each sub-library contains a plurality of tables of data. The information table is used for recording information of the sub-database and the data table therein. The data table is used for storing data.
In a specific embodiment, the information recorded in the information table includes a data type of the data contained in the sub-database, a name of the data table, a number of data table structures of a current data type, a displayable type of the current data type, a current storage line number of the data table, a maximum storage line number of the data table, and whether a flag bit of the maximum line number has been reached.
In a specific embodiment, the data tables are used for recording data content, each data table is named and distinguished by a data type, a month date, a number and a table structure type, wherein the data type is used for recording the type of data stored in the data table, the month date is a specific time for data acquisition or acquisition, the number is Arabic number and is used for indicating the serial number of the data table in a sub-database, and the table structure type is used for displaying inter-table information when the same data is stored by a plurality of data tables.
The data interface comprises a local interface for acquiring a local data file and a remote interface for carrying out data transmission with a server.
The search module comprises an index query unit and a marine data query unit; the index query unit is used for searching whether corresponding index information exists in the ocean data index database according to index information to be queried, and the ocean data query unit retrieves specific data content in the corresponding ocean database according to the retrieval result of the index query unit.
The data importing module acquires a local data file or a server data file received by the data interface; analyzing the data to obtain index information corresponding to the data file, calling a search module to search an ocean data index database, and storing the data to the ocean database according to an index search result, or creating a database or a data table in the ocean database through a database and table dividing module to store the data file; after the data import module stores the data file, the ocean data index library updates the index entry.
The visual module comprises a visual interaction unit and an internal display logic unit, wherein the visual interaction unit calls the search module according to search conditions and display types input by a user to obtain data content meeting the conditions, the internal display logic unit selects a data display method according to display requirements of the user, and the visual interaction unit is used for visual display of the data.
And the data transmission module is used for acquiring a remote data file through a remote interface or transmitting a local data file outwards.
The system shown in the figure 1 can be used for realizing the storage, management, visualization and data transmission of ocean data;
the storing process of the ocean data of the system of the application, as shown in fig. 2, comprises the following steps:
1.1 A data interface receives the imported local data file or remote data file, a data importing module analyzes the obtained data file to obtain index information corresponding to the data file,
1.2 Searching the ocean data index database according to the index information by the searching module, and if the corresponding index information is searched, entering the step 1.4); if no corresponding index information is searched, the step 1.3 is entered;
1.3 Creating a sub-database in the ocean database according to the index information, and creating an information table in the sub-database, wherein the information table is used for recording a data table contained in the sub-database and stored data information; and go to step 1.4)
1.4 Inquiring information tables in the sub-database, and if the existing data tables in the sub-database can be matched with the data types and time information of the data files to be stored, storing the data contents into the corresponding data tables; if the data cannot be matched, a new data table is created to store the data content;
the step 1.4) comprises the following steps:
and (5) entering an information table in a sub-database of the ocean database, and searching the data types and the experimental month and day.
The information table structurally comprises a data type, a table name, the number of table structures of the current data type, a displayable type of the current data type, the current storage line number of the table, the maximum storage line number of the table and whether the maximum line number flag bit is reached.
In the information table, searching the name of the data table which is not fully stored and meets the conditions according to the data structure and the experimental time, judging whether the number of lines of the data table exceeds the maximum value of the number of lines of the data table by combining the number of lines of the current data and the number of lines in the current data table, if the number of lines of the current data table does not exceed the maximum value of the number of lines of the current data table, storing the data in the current data table, and updating the number of lines of the data stored in the current line in the information table; if yes, the next data table is searched continuously, if not, a new data table is created and a new item is added, and the maximum number of data lines in the new item is the same as that in the data table of the same type.
In this embodiment, the table name structure of the data table is "data type_month_number_table structure type", wherein the numbers increase sequentially from 1, and the table structure is used for building multiple data tables of the same data type.
1.5 According to the information determined in the above steps, combining the data type code (2-bit Arabic numerals), the region code (3-bit Arabic numerals), the year, month and day (6-bit Arabic numerals) and the number of rows (7-bit Arabic numerals) of each row of data to generate 18-bit data codes, wherein each data code has unique data codes, and the data codes are used as main key indexes in a data table and can uniquely determine the storage positions of the data;
1.6 According to the storage condition of the data, the ocean data index library updates the index entries.
After determining the current database name and the current data table name, storing based on a multithreading technology, and adding new item storage for the index information table stored in the index library by the main thread.
The retrieval process of the ocean data of the system comprises the following steps: when the ocean data is searched, the search of the ocean data in the database is firstly defined according to the use requirement of the database and the characteristics of the ocean data, the search comprises the acquisition time and the acquisition place of the data, the database where the ocean data is located and the name of the data table are determined in the index database according to the corresponding space-time information, and the database and the name of the data table enter the corresponding data table to be searched according to other parameter conditions and acquire the information.
As shown in fig. 3, the visualization of marine data of the system of the present application comprises the steps of:
2.1 The visual interaction unit obtains the search condition of the user, and invokes the search module to firstly query the ocean data index library to obtain the database information meeting the condition;
2.2 The visual interaction unit displays the database information meeting the conditions in the electronic chart;
2.3 When the user inputs the refined search condition, the visual interaction unit generates a three-dimensional coding range of the data file in the ocean database according to the refined search condition, and searches the data file conforming to the three-dimensional coding range from the database conforming to the condition of the step 2.1); the internal display logic unit selects a display method of data according to the display requirement of a user (for example, the display is usually carried out according to the data type), and obtains a data file through the visual interaction unit, and carries out visual display according to the data type;
2.4 For the temporary data file to be displayed, the data interface acquires the temporary data file and the type information thereof, and the data interface directly performs visual display at the corresponding position in the electronic chart according to the index information and the data type information thereof.
The visual interaction unit in the step 2.3) obtains a data file and performs visual display according to a data type, and the visual interaction unit specifically comprises:
establishing a relation between a data display coordinate and a space geographic position, displaying three-dimensional scalar field data in the ocean data through a ray casting algorithm, and performing multi-view and multi-plane real-time refreshing and display on the three-dimensional scalar field through user control adjustment; the three-dimensional scalar field data comprises experimental region temperature, salinity, isothermal layer depth, mixed layer depth and synthetic mixed layer depth.
The data types include: for scalar data of the ocean environment, such as ocean temperature, salinity, isothermal layer depth, mixed layer depth, synthetic mixed layer depth and the like, displaying according to a three-dimensional scalar field mode; for marine environment vector data, such as flow velocity data, vortex data and the like, the marine environment vector data is displayed in a static or dynamic two-dimensional vector field mode; and drawing a static or dynamic line graph, a spectrogram and the like according to the experimental requirements for the data of the common sensor.
The three-dimensional scalar field display mode is characterized in that parallel light passing through pixels of a display area passes through a body structure of the data to be displayed, resampling and synthesizing color and opacity are carried out in the parallel light, and the result is the color and opacity values of the pixels of the display area. The color distribution is characterized in that the non-ocean area data is determined to be black, the ocean data area is divided into nine sections according to the range, the range value of each section is automatically divided, the data quantity falling in each section floats near the average value, and the pixel value of each point linearly changes gradually along with the distance from the boundary of the section. The opacity value distribution is characterized in that for the nine divided color intervals, the interval opacity value with the smallest set data is the same and the highest, and the interval opacity value is sequentially and equidistantly reduced upwards.
The three-dimensional vector field display mode is characterized in that the scalar value size of the vector field is represented by a color cloud image mode, the vector direction of the arrow starting point position is represented by an arrow direction, and the scalar value size of the arrow starting point position is represented by an arrow length.
And carrying out coordinate transformation on the ocean data type displayed by the three-dimensional field according to the original data, so as to realize real-time multidimensional observation, selecting a position for one dimension, and carrying out section two-dimensional state display by interpolation.
For data types with the interval between two samples less than 30 minutes, linear interpolation is performed every five minutes according to the sampling time, and the display interval is concentrated in 1 minute and linearly reduced so as to simulate dynamic change between the two samples.
In one embodiment of the application, the marine data display function is based on a selected area topography drawn with published etopo data, the extent of which can be adjusted in terms of latitude and longitude on the right side of the panel. In the range of the display area, the existing experimental area position is marked by a square frame, the longitude and latitude of the bullet frame and basic information such as the sampled data type, sampling time and geographic position are displayed by clicking in the area, specific retrieval can be carried out according to the data retrieval scheme, and the bullet frame can be selectively displayed according to the specific data type.
FIGS. 4-6 are schematic diagrams of the operation interface of the visual interactive unit of the present application; as shown in fig. 4, the home panel of the visual interaction unit provides three main functions of database login, historical data playback and processing software, and a single click of a key can enter a corresponding operation panel.
The database operation panel is shown in fig. 5, the right panel inputs corresponding query conditions according to different data types, the left blank space displays the search result in a table form, the right panel can select a reset condition to re-query, or export a query result data file, or perform data display on the query result, and the selection display function displays according to specific data types.
The operation panel of the historical data playback function is shown in fig. 6, the ocean topography of the default position is displayed when the panel is started on the left side, and the display area range can be adjusted in the form of longitude and latitude on the right side panel. In the range of the display area, the existing experimental area position is marked by a square frame, the longitude and latitude of the bullet frame and basic information such as the sampled data type, sampling time and geographic position are displayed by clicking in the area, the specific retrieval can be performed by the operation panel of the rebound database, and the specific data type can be selectively displayed.
In one embodiment of the application, the data transfer is carried out by the transmission of the marine experimental process data and instructions via the TCP/IP protocol;
the marine experimental process data comprises: input data and output data of a sound field modeling experiment, a deep sea acoustic chromatography experiment, a submarine acoustic imaging experiment, a sound field data coupling assimilation experiment, a deep sea mesoscale process detection experiment and a PIES deep sea dynamics parameter inversion experiment;
the instruction comprises a software calling instruction, a data transmission instruction, a data receiving instruction and a data display instruction;
the marine experiment process data comprises a data head, wherein the data head comprises a parameter type, a length and data content length information, and the parameter type comprises: the sending and receiving addresses, data type, data content length, checksum characterized by IP address and port number.
In general, the application can realize the functions of storing, retrieving, visualizing and transmitting ocean data; after the ocean data file is imported, the ocean data file is stored in a database after self-adaptive database separation and table separation; searching and visualizing the data in the database through a visual interaction unit, and providing different display schemes according to different ocean data types; according to the TCP/IP protocol, the application transmits the instruction and the data (if necessary) to the server, thereby realizing the cross-server flow of the instruction and the data.
The foregoing examples illustrate only a few embodiments of the application and are described in detail herein without thereby limiting the scope of the application. It should be noted that it will be apparent to those skilled in the art that several variations and modifications can be made without departing from the spirit of the application, which are all within the scope of the application. Accordingly, the scope of protection of the present application is to be determined by the appended claims.
Claims (2)
1. A MySQL database-based marine data management and visualization system, comprising:
a marine data index library for storing marine data index information; the index information of the ocean data index library comprises longitude and latitude, area codes, database names, data types and data quantity;
the ocean database is used for carrying out three-dimensional coding on ocean data and storing the ocean data in the corresponding database and data table;
the ocean data, namely ocean experimental process data, comprises: input data and output data of a sound field modeling experiment, a deep sea acoustic chromatography experiment, a submarine acoustic imaging experiment, a sound field data coupling assimilation experiment, a deep sea mesoscale process detection experiment and a PIES deep sea dynamics parameter inversion experiment; the marine experiment process data comprises a data head, wherein the data head comprises a parameter type, a length and data content length information, and the parameter type comprises: a transmitting address and a receiving address characterized by an IP address and a port number, a data type, a data content length and a checksum; the ocean database is divided into a plurality of regional sub-databases according to the geographic position of the data, each regional sub-database is divided into a plurality of annual sub-databases according to the year of the data, each annual sub-database is divided into a plurality of sub-databases according to the number, each sub-database is provided with an information table, and each sub-database comprises a plurality of data tables; the information recorded by the information table comprises the data type, the name, the structure number, the displayable type, the current storage line number, the maximum storage line number and whether the maximum line number zone bit is reached or not; the data tables are used for recording data content, each data table is named and distinguished by a data type, a month date, a number and a table structure type, wherein the data type is used for recording the type of data stored in the data table, the month date is the specific time for data acquisition or acquisition, the number is Arabic number and is used for representing the serial number of the data table in a sub-database, and the table structure type is used for displaying the information among tables when the same data is stored by a plurality of data tables;
a data interface comprising a local interface for obtaining a local data file and a remote interface for data transmission with a server;
the search module comprises an index query unit and a marine data query unit; the index inquiry unit is used for searching whether corresponding index information exists in the ocean data index database according to index information to be inquired, and the ocean data inquiry unit is used for searching corresponding data content in the ocean database according to inquiry information input by a user;
the data importing module acquires a local data file or a server data file received by the data interface; analyzing the data to obtain index information corresponding to the data file, calling a search module to search an ocean data index database, and storing the data to the ocean database according to an index search result or storing the data file by newly building a data table in the ocean database; after the data import module stores the data file, the ocean data index library updates the index entry;
the visual module comprises a visual interaction unit, and the visual interaction unit calls the retrieval module according to retrieval conditions and display types input by a user to obtain data content meeting the conditions and performs visual display of the data through the visual interaction unit;
the storing of the ocean data comprises the following steps:
1.1 A data interface receives the imported local data file or remote data file, a data importing module analyzes the obtained data file to obtain index information corresponding to the data file,
1.2 Searching the ocean data index database according to the index information by the searching module, and if the corresponding index information is searched, entering the step 1.4); if no corresponding index information is searched, the step 1.3 is entered;
1.3 Creating a sub-database in the ocean database according to the index information, and creating an information table in the sub-database, wherein the information table is used for recording a data table contained in the sub-database and stored data information; and go to step 1.4)
1.4 Inquiring information tables in the sub-database, and if the existing data tables in the sub-database can be matched with the data types and time information of the data files to be stored, storing the data contents into the corresponding data tables; if the data cannot be matched, a new data table is created to store the data content;
the step 1.4) comprises the following steps:
entering an information table in a sub-database of the ocean database, and searching the data types and the experimental month and day;
in the information table, searching the name of the data table which is not fully stored and meets the conditions according to the data structure and the experimental time, judging whether the number of lines of the data table exceeds the maximum value of the number of lines of the data table by combining the number of lines of the current data and the number of lines in the current data table, if the number of lines of the current data table does not exceed the maximum value of the number of lines of the current data table, storing the data in the current data table, and updating the number of lines of the data stored in the current line in the information table; if yes, continuously inquiring whether the next data table meets the storage condition, if not, creating a new data table and adding a new item, wherein the maximum number of lines of data in the new item is the same as that of the data table of the same type;
the table name structure of the data table is a data type_month_day_number_table structure type, wherein the numbers are sequentially increased from 1, and the table structure is used for establishing a plurality of data tables of the same data type;
1.5 According to the information determined by the steps, combining the data type code, the area code, the year, month and day and the line number of each line of data to generate a data code, wherein the data code is unique data code for each line of data, and the data code is used as a main key index in a data table and can uniquely determine the position where the data is stored;
1.6 According to the storage condition of the data, updating index entries by the ocean data index base;
the visualization of marine data comprises the steps of:
2.1 The visual interaction unit obtains the search condition of the user, and invokes the search module to firstly query the ocean data index library to obtain the database information meeting the condition;
2.2 The visual interaction unit displays the database information meeting the conditions in the electronic chart;
2.3 When the user inputs the refined search condition, the visual interaction unit generates the coding range of the data file according to the refined search condition, and searches the data file conforming to the coding range from the database conforming to the condition of the step 2.1); the visual interaction unit acquires a data file and performs visual display according to the data type;
2.4 For the temporary data file to be displayed, the data interface acquires the temporary data file and the type information thereof, and the data interface directly performs visual display at the corresponding position in the electronic chart according to the index information and the data type information thereof;
and the data transmission module is used for acquiring a remote data file through a remote interface or transmitting a local data file outwards.
2. A marine data management and visualization method applied to the system of claim 1, wherein the method can realize storage, visualization and data transfer of marine data; the ocean data, namely ocean experimental process data, comprises: input data and output data of a sound field modeling experiment, a deep sea acoustic chromatography experiment, a submarine acoustic imaging experiment, a sound field data coupling assimilation experiment, a deep sea mesoscale process detection experiment and a PIES deep sea dynamics parameter inversion experiment; the marine experiment process data comprises a data head, wherein the data head comprises a parameter type, a length and data content length information, and the parameter type comprises: a transmitting address and a receiving address characterized by an IP address and a port number, a data type, a data content length and a checksum;
the storing of the ocean data comprises the following steps:
1.1 A data interface receives the imported local data file or remote data file, a data importing module analyzes the obtained data file to obtain index information corresponding to the data file,
1.2 Searching the ocean data index database according to the index information by the searching module, and if the corresponding index information is searched, entering the step 1.4); if no corresponding index information is searched, the step 1.3 is entered;
1.3 Creating a sub-database in the ocean database according to the index information, and creating an information table in the sub-database, wherein the information table is used for recording a data table contained in the sub-database and stored data information; and go to step 1.4)
1.4 Inquiring information tables in the sub-database, and if the existing data tables in the sub-database can be matched with the data types and time information of the data files to be stored, storing the data contents into the corresponding data tables; if the data cannot be matched, a new data table is created to store the data content;
the step 1.4) comprises the following steps:
entering an information table in a sub-database of the ocean database, and searching the data types and the experimental month and day;
in the information table, searching the name of the data table which is not fully stored and meets the conditions according to the data structure and the experimental time, judging whether the number of lines of the data table exceeds the maximum value of the number of lines of the data table by combining the number of lines of the current data and the number of lines in the current data table, if the number of lines of the current data table does not exceed the maximum value of the number of lines of the current data table, storing the data in the current data table, and updating the number of lines of the data stored in the current line in the information table; if yes, continuously inquiring whether the next data table meets the storage condition, if not, creating a new data table and adding a new item, wherein the maximum number of lines of data in the new item is the same as that of the data table of the same type;
the table name structure of the data table is a data type_month_day_number_table structure type, wherein the numbers are sequentially increased from 1, and the table structure is used for establishing a plurality of data tables of the same data type;
1.5 According to the information determined by the steps, combining the data type code, the area code, the year, month and day and the line number of each line of data to generate a data code, wherein the data code is unique data code for each line of data, and the data code is used as a main key index in a data table and can uniquely determine the position where the data is stored;
1.6 According to the storage condition of the data, updating index entries by the ocean data index base;
the visualization of the marine data comprises the steps of:
2.1 The visual interaction unit obtains the search condition of the user, and invokes the search module to firstly query the ocean data index library to obtain the database information meeting the condition;
2.2 The visual interaction unit displays the database information meeting the conditions in the electronic chart;
2.3 When the user inputs the refined search condition, the visual interaction unit generates the coding range of the data file according to the refined search condition, and searches the data file conforming to the coding range from the database conforming to the condition of the step 2.1); the visual interaction unit acquires a data file and performs visual display according to the data type;
the visual interaction unit in the step 2.3) obtains the data file and performs visual display according to the data type, and the method specifically comprises the following steps:
establishing a relation between a data display coordinate and a space geographic position, displaying three-dimensional scalar field data in the ocean data through a ray casting algorithm, and performing multi-view and multi-plane real-time refreshing and display on the three-dimensional scalar field through user control adjustment; the three-dimensional scalar field data comprises experimental region temperature, salinity, isothermal layer depth, mixed layer depth and synthetic mixed layer depth;
the data types include: for the scalar data of the ocean environment, displaying according to a three-dimensional scalar field mode; for marine environment vector data, displaying the marine environment vector data in a static or dynamic two-dimensional vector field mode; for common sensor data, drawing a static or dynamic line graph and a spectrogram according to experimental requirements;
the three-dimensional scalar field mode is displayed as follows: generating data to be displayed by interpolation of the original data, generating a scalar field image by a ray projection algorithm, enabling parallel light passing through pixels of a display area to pass through a body structure of the data to be displayed, and carrying out resampling and color and opacity synthesis on the interior of the data to obtain the result, namely the color and opacity values of the pixels of the display area;
the color composition is as follows: the non-ocean area data are determined to be black, the ocean data area is divided into nine sections according to the range of the ocean data area, the range value of each section is automatically divided, the data quantity falling in each section floats near the average value, and the pixel value of each point linearly changes gradually along with the distance from the boundary of the section;
the opacity is synthesized as: setting the same and highest opacity value of the interval with the smallest data for the divided nine color intervals, and sequentially and equidistantly reducing the opacity value upwards;
the two-dimensional vector field mode is displayed as follows: representing the scalar value of the vector field in a color cloud diagram mode, representing the vector direction of the arrow starting point position in an arrow direction, and representing the scalar value of the arrow starting point position in an arrow length mode;
performing coordinate transformation on the ocean data type displayed by the three-dimensional field according to the original data, so as to realize real-time multidimensional observation, selecting a position for one dimension, and performing section two-dimensional state display through interpolation;
for the data type with the twice sampling interval smaller than 30 minutes, linear interpolation is carried out every five minutes according to the sampling time, and the display interval is concentrated in 1 minute and linearly reduced so as to simulate the dynamic change between the two times of sampling;
2.4 For the temporary data file to be displayed, the data interface acquires the temporary data file and the type information thereof, and the data interface directly performs visual display at the corresponding position in the electronic chart according to the index information and the data type information thereof.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011547626.6A CN112612777B (en) | 2020-12-24 | 2020-12-24 | Ocean data management and visualization system and method based on MySQL database |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011547626.6A CN112612777B (en) | 2020-12-24 | 2020-12-24 | Ocean data management and visualization system and method based on MySQL database |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112612777A CN112612777A (en) | 2021-04-06 |
| CN112612777B true CN112612777B (en) | 2023-12-12 |
Family
ID=75245472
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011547626.6A Active CN112612777B (en) | 2020-12-24 | 2020-12-24 | Ocean data management and visualization system and method based on MySQL database |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112612777B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113190511B (en) * | 2021-04-21 | 2022-09-13 | 中国海洋大学 | Big data concurrent scheduling and accelerated processing method based on many-core cluster |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102999592A (en) * | 2012-11-19 | 2013-03-27 | 北京中海新图科技有限公司 | B/S architecture based global Argo multi-source marine data management and visualization system and method |
| KR101249825B1 (en) * | 2012-08-02 | 2013-04-03 | 주식회사 환경과학기술 | Visualizing system for marine geographic information and method thereof |
| CN103870542A (en) * | 2014-02-25 | 2014-06-18 | 中国人民解放军92859部队 | Storage method for marine environment data based on geographical fragmentation and dynamic tabulation |
| CN104268298A (en) * | 2014-10-27 | 2015-01-07 | 中电海康集团有限公司 | Method for creating database index and inquiring data |
| CN105069020A (en) * | 2015-07-14 | 2015-11-18 | 国家信息中心 | 3D visualization method and system of natural resource data |
| CN109885572A (en) * | 2019-02-20 | 2019-06-14 | 哈尔滨工程大学 | A kind of three-dimensional data coding and storage method for magnanimity marine environment data management |
| CN110807134A (en) * | 2019-10-10 | 2020-02-18 | 哈尔滨工程大学 | Ocean three-dimensional scalar field visualization method |
| CN110826183A (en) * | 2019-10-08 | 2020-02-21 | 广州博进信息技术有限公司 | Construction interaction method for multidimensional dynamic marine environment scalar field |
| CN112015741A (en) * | 2020-10-19 | 2020-12-01 | 武汉物易云通网络科技有限公司 | Method and device for storing massive data in different databases and tables |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7873904B2 (en) * | 2007-04-13 | 2011-01-18 | Microsoft Corporation | Internet visualization system and related user interfaces |
| US9734220B2 (en) * | 2012-12-04 | 2017-08-15 | Planet Os Inc. | Spatio-temporal data processing systems and methods |
| US9727670B2 (en) * | 2013-10-04 | 2017-08-08 | The Boeing Company | Horizon night view simulation |
| US20170308571A1 (en) * | 2016-04-20 | 2017-10-26 | Google Inc. | Techniques for utilizing a natural language interface to perform data analysis and retrieval |
-
2020
- 2020-12-24 CN CN202011547626.6A patent/CN112612777B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR101249825B1 (en) * | 2012-08-02 | 2013-04-03 | 주식회사 환경과학기술 | Visualizing system for marine geographic information and method thereof |
| CN102999592A (en) * | 2012-11-19 | 2013-03-27 | 北京中海新图科技有限公司 | B/S architecture based global Argo multi-source marine data management and visualization system and method |
| CN103870542A (en) * | 2014-02-25 | 2014-06-18 | 中国人民解放军92859部队 | Storage method for marine environment data based on geographical fragmentation and dynamic tabulation |
| CN104268298A (en) * | 2014-10-27 | 2015-01-07 | 中电海康集团有限公司 | Method for creating database index and inquiring data |
| CN105069020A (en) * | 2015-07-14 | 2015-11-18 | 国家信息中心 | 3D visualization method and system of natural resource data |
| CN109885572A (en) * | 2019-02-20 | 2019-06-14 | 哈尔滨工程大学 | A kind of three-dimensional data coding and storage method for magnanimity marine environment data management |
| CN110826183A (en) * | 2019-10-08 | 2020-02-21 | 广州博进信息技术有限公司 | Construction interaction method for multidimensional dynamic marine environment scalar field |
| CN110807134A (en) * | 2019-10-10 | 2020-02-18 | 哈尔滨工程大学 | Ocean three-dimensional scalar field visualization method |
| CN112015741A (en) * | 2020-10-19 | 2020-12-01 | 武汉物易云通网络科技有限公司 | Method and device for storing massive data in different databases and tables |
Non-Patent Citations (4)
| Title |
|---|
| 全球海洋再分析数据管理及可视化技术研究;温娜;《中国优秀硕士学位论文全文数据库 基础科学辑》;20190815;文章第8-13，17，47-53页 * |
| 国家科学技术奖励工作办公室主办.开放式虚拟地球集成共享平台及重大工程应用.《中国科学技术奖励年鉴2011年下》.中国科学技术奖励年鉴编辑部,2011,第1518页. * |
| 基于三维虚拟地球的海洋环境数据动态可视化研究;王想红;《中国博士学位论文全文数据库 基础科学辑》;20150215;文章第69-82页 * |
| 温娜.全球海洋再分析数据管理及可视化技术研究.《中国优秀硕士学位论文全文数据库 基础科学辑》.2019,第A010-9页. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112612777A (en) | 2021-04-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110826183B (en) | Construction interaction method of scalar field of multidimensional dynamic ocean environment | |
| US9418075B2 (en) | Automatic meta-neighborhood and annotation generation for maps | |
| JP3212113B2 (en) | Map information display method and device | |
| US20100177120A1 (en) | System and method for stretching 3d/4d spatial hierarchy models for improved viewing | |
| US20080051994A1 (en) | Representation and display of geographical popularity data | |
| Murodilov | Improving the methods of providing geo-information for the monitoring of territories and developing the basis of web-maps | |
| Kraak | The cartographic visualization process: from presentation to exploration | |
| CN112579845B (en) | Industrial big data display geographic information system platform | |
| CN112612777B (en) | Ocean data management and visualization system and method based on MySQL database | |
| Boukhelifa et al. | Uncertainty visualization: why might it fail? | |
| CN110197035A (en) | A kind of navigation channel change of Underwater Topography analysis system and method | |
| CN114064813A (en) | Method for inquiring and visualizing marine multidimensional data | |
| CN115168394A (en) | Plant health state evaluation system based on database | |
| CN111125237B (en) | NoSql database-based dynamic mapping method for weather numerical forecast products | |
| Uusitalo et al. | Virtual forest management: possibilities and challenges | |
| Guney | Rethinking GIS towards the vision of smart cities through CityGML | |
| Van Leusen et al. | 3D spatial analysis: The road ahead | |
| Eldawy et al. | A demonstration of Shahed: A MapReduce-based system for querying and visualizing satellite data | |
| Abdul Rahman et al. | 3D Geospatial Database Schema for Istanbul 3D City Model | |
| CN108959420A (en) | The method of spatio-temporal data visualization interface rapidly locating | |
| Krivoruchko et al. | Map-Studio: The specialized GIS integrating possibilities of geostatistics | |
| CN116910131B (en) | Linkage visualization method and system based on basic geographic entity database | |
| Schoemans et al. | MOVE: Interactive Visual Exploration of Moving Objects. | |
| Soracco | Visualizing and distributing decades of satellite ocean remote sensing data products | |
| CN116186189B (en) | Method and system for rapidly generating elevation live-action map model |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |