CN110781240A - Visual mining and application method of red tide data - Google Patents

Abstract

A mining and application method of red tide data visualization relates to the field of marine environmental science and technology. 1) Standardize the information of the original red tide events; 2) Analyze the historical records of red tide events, check and screen out independent red tide events, and complete the data preprocessing; 3) Based on the data preprocessed in step 2), design and draw a variety of content It can identify and analyze the composition, relationship, level and distribution of red tide information, select and optimize various types of charts; 4) Based on the data preprocessed in step 2), design dynamic timelines and 3D maps Source visualization fusion method to verify the feasibility and effectiveness of red tide information mining; 5) Based on steps 3) and 4), build a dashboard model, and design a more general red tide dynamic analysis chart for red tide information mining and analysis. It can effectively screen out the nodes of red tide zones and periods, and is widely used in red tide information mining.

Description

A method of mining and application of red tide data visualization

technical field

The invention relates to the field of marine environment science and technology, in particular to the combination of data preprocessing programmed by Matlab and multiple visualization fusion methods based on new Excel functions to form a layered and visualized data structure with only "one chart" A kind of mining and application method of red tide data visualization, which is a method of deep mining data information.

Background technique

The visualization of red tide data is mostly displayed by software related to professional fields such as remote sensing or GIS after forecasting (Xu Bo et al., Journal of Zhejiang University (Science Edition), 1008-9497(2004) 04-471-05, 31(4) : 471-475; Chen Yunzhi et al., Journal of Fuzhou University (Natural Science Edition), 1000-2243 (2013) 06-1002-07 2013, 41(6): 1002-1008; ) 01-0059-062013, 30(1):59-64), while the professional information mining using other fields of analysis and statistics software (especially the general office software EXCEL) in the early stage of prediction and forecasting is less. The intersection of different disciplines or fields can spark new ideas and perspectives.

Chinese patent CN201810966994.0 discloses a red tide data query method based on a graph model, the method includes: the construction of a red tide data graph model and the construction of a red tide data query language; the red tide data graph model RTGraph includes three kinds of data: point Data, edge data, and red tide edge data; the red tide edge data is a kind of point data, marked by attributes on the point; the red tide data query language includes: create statement, query statement, update statement, insert statement, delete statement. The red tide data is stored in the graph model according to a specific stage, and the edge data of the red tide is established, which can represent the relationship between the red tide data. The query speed and accuracy are improved, and red tide data can be fully utilized for research. Researchers can predict when and where the phase transition will occur, and take corresponding measures to reduce economic and ecological losses.

How to closely follow the concept of recognizing the ocean and serving the public in the "Smart Ocean" project, through the multi-source visual expression such as diversification and combination of static and dynamic charts, from different perspectives and ways to dig deep red tide information, and exploration can help users analyze and find It is a technical problem at this stage to find out the inherent laws of data and the visualization method of use value, and to obtain effective, objective and valuable red tide information, so as to help scientific research and related departments to provide technical support in innovation and application.

SUMMARY OF THE INVENTION

The purpose of the present invention is to provide a method for mining and applying red tide data visualization, combining non-professional software Excel and professional software Matlab, to design a more general visualization expression fusion method that can realize effective information mining and analysis of red tide, and more effectively help users Analyze and extract the inherent laws of red tide data, and provide technical support for scientific research, management and other related departments.

The present invention includes the following steps:

1) Standardize the information of the original red tide event; use matlab programming to realize the batch reading of files, then identify the structure of the data and the names of each item, modify the data in batches according to the standardized data items, and finally output the file;

2) Analyze the historical records of red tide events, check and filter out independent red tide events, and complete data preprocessing;

3) Based on the data preprocessed in step 2), design and draw data pivot tables and perspective charts of various contents, identify and analyze the composition, connection, level and distribution of red tide information, and select and optimize various types of charts;

4) Based on the data preprocessed in step 2), a multi-source visualization fusion method of dynamic timeline and three-dimensional map is designed to verify the feasibility and effectiveness of red tide information mining;

5) Based on step 3) and step 4), build a dashboard model, and design a more general red tide dynamic analysis chart for red tide information mining and analysis.

In step 1), the specific method for normalizing the information of the original red tide event may be:

First, use Matlab programming to read the data in batches, identify the structure of the original data and the names of each item through the software, and then check and input the title names of each column in the red tide event history file according to the standard data items, and use the Matlab correction program to make corresponding supplements and corrections. Sorting, batch-corrected data are output in batches through programming; the standardized data items are the title names of each column, which are set as: serial number, occurrence time, extinction time, duration, location, longitude, latitude, occurrence sea area, specific sea area , Maximum affected area, biological dominant species, highest density, loss situation.

In step 2), the specific method of the data preprocessing can be:

Check and screen out independent red tide events. Due to different statistical methods in the original data, there are two kinds of red tide events: single algal species red tide and mixed algal species red tide. In some original documents, there is a mixed algal species red tide event divided into multiple single algal species red tide events. In order to avoid the occurrence of red tide events, the starting and ending time and place of red tide events were checked, the red tide events of mixed algal species were unified, and the repeated information of branch statistics was deleted.

In step 3), the specific method for identifying and analyzing the composition, connection, level and distribution of red tide information may be:

Using the enhanced function of Excel's pivot table, it is possible to easily build complex models across data, and automatically group data by time. Users can view the content according to their needs, and select the corresponding sea area, area, etc. for data statistics through the filter button. The time grouping of data includes Seconds, minutes, hours, days, months, quarters, years, or a combination of these; analyze and design pivot tables with multiple contents, use pivot tables as dynamic data sources, select and optimize various chart types, and generate pivot charts ;Insert a perspective view, which can group and zoom in and out of other hierarchical structures in the data across time, and can select the corresponding sea area, area, etc. for graphical display according to the content that the user needs to view.

In step 4), the multi-source visualization fusion mode of the design dynamic timeline and the three-dimensional map is described to verify the feasibility and effectiveness of red tide information mining, and the specific steps can be:

(1) Use the Excel dynamic time line to filter the nodes of red tide information in different periods: Based on the function of Excel schedule, according to user needs, select the dynamic time range of a single year or multiple consecutive years for data statistics and visual display. Drag the time slider on the animation control toolbar to display the distribution of red tide information in different periods. Based on the function of Excel dynamic charts, select the corresponding items (such as time nodes or sub-regions) for visual display after data filtering, which is beneficial for users to split, view and analyze single-point data items;

(2) Use Excel 3D map to analyze the spatial relationship and law of red tide information: For red tide data containing location information, through the acquisition and identification of data spatial coordinate information, the data can be quickly mapped to the virtual earth, and the content to be expressed by the data can be quickly mapped. Display it on a suitable map such as a 2D floor plan or a 3D map, perform dynamic analysis and 3D display on thematic maps, convert form data with spatial information into thematic maps, realize data spatial visualization products, and improve data visibility and analysis. efficiency. For example, in a three-dimensional map showing the spatial distribution of red tide algae species over time, the dynamic changes in the location and frequency of previous red tide algae species can be demonstrated in chronological order according to the time axis.

(3) Based on the example of the original data of red tide event historical records, the validity and feasibility of mining red tide information by this fusion method are verified.

In step 5), a dashboard model is constructed, and the specific method for designing a more general red tide dynamic analysis chart is: design a dynamic chart for the occurrence of red tides in different algae species and different months, and build a red tide occurrence frequency dashboard model. Demand, by integrating various visual designs such as pivot table screening, mini-chart display, dashboard model, etc., to dynamically display the distribution characteristics of red tide occurrences, and at the same time, a comparison chart design for red tide outbreaks of two algae species has been added. It is necessary to independently select two algal species for comparative analysis to find the regularity of red tide occurrence of different algal species.

In order to be more complete, less omission, and more valuable information of red tide, the present invention utilizes and integrates multi-source visualization functions such as Excel dynamic time line and dynamic chart, identifies and analyzes the composition, connection, level and distribution of red tide information, and discriminates data time nodes. Or space demarcation and other subtle differences, build a dashboard model, etc., design a variety of visual expression fusion methods that can realize effective information mining and analysis of red tide, and test the effectiveness of the design through examples.

The beneficial effects of the present invention are:

The invention first uses Matlab to realize data preprocessing in batches, and then, based on the design of several visual charts in Excel, uses dynamic time lines to effectively screen out nodes with time-varying characteristics of red tides in different periods, and then combines the spatial distribution of red tide outbreaks in a three-dimensional map The static display and dynamic time display of red tide can effectively screen out the nodes of red tide zones and periods, and verify the feasibility and effectiveness of the fusion design of multiple visual expressions for red tide information mining. After that, combined with the functions of pivot table and sparkline in the data quick analysis module in Excel, and integrated various visualization methods such as pivot chart, sparkline, dynamic chart, etc., to build a dashboard model, and designed two more generalized models. Visual expression - "one chart": a dashboard model for the frequency distribution of red tides of different algal species and a dynamic chart model for the comparison of A/B algal species, which can quickly screen the data structure hierarchically, which is conducive to the subtle features of red tide information The discovery is of great significance for the discussion of the evolution law and mechanism of red tide outbreaks, and provides technical support for relevant departments such as scientific research and management. It can be widely used in red tide information mining.

Description of drawings

Fig. 1 is the flow chart of a kind of red tide data visualization mining and application method;

Figure 2 is an enlarged layered structure based on the 2000-2016 red tide data perspective;

Fig. 3 is the layered structure reduction based on the 2000-2016 red tide data perspective;

Figure 4 is a schematic diagram of a dynamic chart based on red tide information in Fujian Province from 2000 to 2016;

Figure 5 shows the temporal and spatial distribution of red tide duration in Fujian Province on the Excel dynamic timeline;

Figure 6 is a three-dimensional map showing the spatial distribution of the frequency of red tide outbreaks;

Fig. 7 is a three-dimensional map showing the spatial distribution of red tide algal species over time;

Figure 8 is a three-dimensional area map showing the grading distribution of the maximum area of red tide outbreaks in each sea area;

Figure 9 shows the information mining of the outbreak of red tides in the near-shore area of Fujian based on the fusion of multi-source visualization methods;

Figure 10 is a dynamic chart based on a dashboard model of red tide occurrence frequency;

Figure 11 shows the comparison of red tide outbreaks for two algal species.

Detailed ways

The following embodiments will further illustrate the present invention in conjunction with the accompanying drawings.

Referring to Fig. 1, the embodiment of the present invention includes the following steps:

1) Standardize the information of the original red tide event;

In this embodiment, the historical record data of red tide events from 2000 to 2016 is selected, and Matlab programming is used to read the data in batches, and the structure and distribution of the original data are identified through software, and then each column in the red tide event historical record file is checked and input according to the standard data items. Header names, complement and order their column headers. The normative data items and sorting can include the following: serial number, occurrence time, extinction time, duration, location, occurrence sea area, specific sea area, maximum affected area (square kilometers), biological dominant species, highest density, and loss.

Since part of the column heading "Location" in the original file is marked with place names, while the other part is marked with latitude and longitude; the column heading "Sea Area" is marked with place names at the county level or smaller, and although some areas point to the same place, the place names are not Unification; In summary, in order to facilitate subsequent data analysis, the data is standardized, the data structure and names are unified, and the column heading "Sea Areas" is divided into two columns, "Occurrence Sea Areas" and "Specific Sea Areas". Among them, "occurrence sea area" is marked with the name above the city level, and "specific sea area" is marked with the name of the specific sea area; "location" is marked with the specific latitude and longitude.

To facilitate the display of the spatial distribution of the locations of red tide events, two columns of "Longitude" and "Latitude" are set. After the title of the column of "occurred sea area" is unified, the central location of the red tide event is determined according to the location of the city-level locality combined with the "specific sea area", and the two columns of "longitude" and "latitude" are supplemented, as shown in Table 1.

Table 1

Use the Matlab correction program to supplement and sort accordingly. The corrected data is output in batches by programming.

2) Analyze the historical records of red tide events, check and filter out independent red tide events, and complete data preprocessing.

Test and filter out independent red tide events. Due to different statistical methods in the original data, there are two types of red tide events: single algal species red tide and mixed algal species red tide. In some original documents, a mixed algal species red tide event was divided into multiple single algal species red tide events, resulting in repeated statistics of red tide events. In order to avoid this situation, check the start and end time and place of red tide events, unify the red tide events of mixed algal species, and delete the duplicate information of their branch statistics;

3) Based on the data preprocessed in step 2), design and draw pivot tables and perspective charts with various contents, identify and analyze the composition, connection, level and distribution of red tide information, and select and optimize various types of charts.

First of all, using the enhanced function of the pivot table of Excel 2016, it is possible to easily build complex models across data, and automatically group the data by time. Users can view the content according to their needs, and select the corresponding sea area, area, etc. for data statistics through the filter button. For example: Click the corresponding year and quarter to see the grouping of various time display methods such as year, quarter, month, etc. Analyze and design pivot tables with various contents, use pivot tables as dynamic data sources, select and optimize various chart types, and generate pivot charts. For example: based on a pivot table, insert a pivot chart, the "+, -" buttons in the lower right corner of the pivot chart allow you to group across time and zoom in and out of other hierarchies in the data (Figures 2 and 3).

The two filter tabs on the right side of the perspective view can select the corresponding sea area, area, etc. for graphic display according to the content that the user needs to view.

4) Based on the data preprocessed in step 2), the effectiveness and feasibility of red tide information mining is verified by the design of the combination of dynamic timeline and multi-source visualization of 3D map.

Use Excel dynamic time line to filter the nodes of red tide information in different periods. The details are as follows: Based on the function of Excel schedule, according to user needs, select a single year or a dynamic time range of multiple consecutive years for data statistics and visual display (Figure 4). Drag the time slider on the animation control toolbar at the bottom of the screen to display the distribution of red tide information in different periods. Based on the function of Excel dynamic charts, selecting the corresponding items (such as time nodes or sub-regions) for visual display after data filtering is helpful for users to split, view and analyze single-point data items.

The spatial relationship and regularity of red tide information were analyzed by using Excel 3D map. The details are as follows: For red tide data containing location information, through the acquisition and identification of data space coordinate information, the data can be quickly mapped to the virtual earth, and the content to be expressed by the data can be displayed on a suitable map such as a two-dimensional plan or a three-dimensional map. Perform dynamic analysis and three-dimensional display of thematic maps, convert form data with spatial information into thematic maps, realize data spatial visualization products, and improve data visibility and analysis efficiency (Figures 5-7). For example, in a three-dimensional map showing the spatial distribution of red tide algae species over time, the dynamic changes in the location and frequency of previous red tide algae species can be demonstrated in chronological order according to the time axis.

In this example, based on the red tide data recorded in the history of red tide events from 2000 to 2016, the design of the combination of the dynamic time line and the multi-source visualization of the three-dimensional map has successfully excavated the characteristics of red tide zones in this period. Firstly, the three-dimensional map and three-dimensional area map of the high-incidence area of red tide distribution and the largest cumulative area are designed, and the eruption distribution area is divided into three areas: A, B, and C. Among them, Area A is mainly in the north, Fuding, Xiapu and Lianjiang in the waters north of the Yangtze River; Area B is in the middle, Fuqing, Pingtan, Putian and Quanzhou in the waters south of the Yangtze River; Area C is in the south, Neiwan Sea Area, Xiamen and Dongshan b. At the same time, the dynamic time line and dynamic chart of the number of red tide outbreaks from 2000 to 2016 were designed and drawn. The results showed that the number of red tide outbreaks in most sea areas decreased significantly in 2009, but increased significantly in 2010, which may be a turning point. In addition, looking at the distribution of the three divisions A, B, and C year by year, the B district increased significantly to 6 times in 2010, and the area also increased significantly, so it is divided into two periods: 2000-2009 and 2010-2016. Based on the information fusion of the above-mentioned various visual expressions, it is concluded that the number of red tide outbreaks and the maximum accumulated area show obvious temporal and spatial differences (Figure 8). The cumulative maximum area showed an opposite trend in each subregion, with the northern (Area) decreasing, the central (B) increasing, and the southern (C) decreasing (Figure 9).

5) Based on step 3) and step 4), build a dashboard model, and design a more general red tide dynamic analysis chart for red tide information mining and analysis.

A dynamic chart was designed for the occurrence of red tides in different months, and a dashboard model of red tide occurrence frequency was constructed. According to user needs, through the integration of various designs such as pivot table screening, sparkline display, and dashboard model, the distribution characteristics of red tide occurrences are dynamically displayed (Figure 10). At the same time, a comparison chart of red tide outbreaks of two algae species has been added in the design (Figure 11). Users can choose two algae species for comparison and analysis according to their needs, and find the law of red tide occurrence of different algae species.

Based on the Excel function module, the present invention normalizes the red tide data, combines the method of cross-time grouping and data structure layering in the data pivot table and the pivot chart, and easily constructs a complex model across data, and identifies and analyzes the composition and connection of red tide information. , level and distribution, to discriminate the differences in subtle information such as data time nodes or spatial boundaries; through the fusion of dynamic timelines, three-dimensional maps and other visualization methods, the data information is peeled off layer by layer, and the effectiveness of the fusion method is evaluated through examples; finally , which integrates various visualization methods such as perspective charts, sparklines, and dynamic charts to build a dashboard model, and design a more common visualization method for red tide dynamic analysis—“one chart” for red tide information mining. Examples prove that the method of the present invention can effectively screen out the nodes of red tide zones and periods, more effectively help users to analyze and extract the inherent laws of red tide data, and provide technical support for relevant departments such as scientific research and management.

Claims (7)

1. A visual mining and application method of red tide data is characterized by comprising the following steps:

1) carrying out normalized processing on the information of the original red tide event; reading files in batches by utilizing matlab programming, identifying the structure and the name of each project of data, correcting the data in batches according to the specified data project, and finally outputting the files;

2) analyzing the history of the red tide events, checking and screening out independent red tide events, and finishing data preprocessing;

3) designing and drawing a data pivot table and a perspective view of various contents based on the data preprocessed in the step 2), identifying and analyzing the composition, relation, hierarchy and distribution of red tide information, and selecting and optimizing various chart types;

4) based on the data preprocessed in the step 2), designing a multi-source visual fusion mode of a dynamic time line and a three-dimensional map, and verifying the feasibility and effectiveness of the red tide information mining;

5) based on the step 3) and the step 4), an instrument panel model is constructed, and a relatively universal red tide dynamic analysis chart is designed and used for red tide information mining analysis.

2. The method for mining and applying the visualization of red tide data as claimed in claim 1, wherein in step 1), the specific method for performing the normalization processing on the information of the original red tide event comprises:

firstly, reading data in batches by utilizing Matlab programming, identifying the structure and the item names of original data through software, then checking the title names of all columns in an input red tide event history record file according to a standard data item, correspondingly supplementing and sequencing by utilizing a Matlab modification program, and outputting the batch modified data in batches by programming.

3. The visual mining and application method of red tide data as claimed in claim 2, wherein the canonical data items, i.e. the column header names, are set as: sequence number, occurrence time, death time, duration, location, longitude, latitude, sea area of occurrence, specific sea area, maximum area of influence, species of biological dominance, highest density, loss condition.

4. The method for mining and applying the visualization of red tide data as claimed in claim 1, wherein in step 2), the specific method for preprocessing the data is: the method comprises the steps of detecting and screening independent red tide events, wherein two red tide events of single algae species red tide and mixed algae species red tide exist in original data due to different statistical modes, in part of original files, one mixed algae species red tide event is divided into a plurality of single algae species red tide events, repeated statistics of the red tide events is caused, in order to avoid the situation, the starting and ending time and the occurrence place of the red tide events are detected, the red tide events of the mixed algae species are unified, and repeated information of branch statistics is deleted.

5. The method as claimed in claim 1, wherein in step 3), the specific method for identifying and analyzing the composition, relationship, hierarchy and distribution of red tide information is as follows: the data perspective table enhancement function of Excel is utilized to realize that a complex model is easily built across data, the data are automatically grouped according to time, a user checks the content according to the requirement, selects a corresponding sea area and area through a screening button to carry out data statistics, and the time grouping of the data comprises the modes of second, minute, hour, day, month, quarter, year or the combination of the modes; analyzing and designing a data pivot table with various contents, taking the pivot table as a dynamic data source, selecting and optimizing various chart types, and generating a perspective view; inserting perspective, grouping and zooming in and out other hierarchies in the data in a time-span mode, and selecting corresponding sea areas and regions for graphical display according to contents which a user needs to view.

6. The visual mining and application method of red tide data according to claim 1, wherein in step 4), the feasibility and the effectiveness of the red tide information mining are verified by designing a multi-source visual fusion mode of a dynamic timeline and a three-dimensional map, and the specific steps are as follows:

(1) screening the nodes of the red tide information in different periods by using an Excel dynamic time line: based on the function of an Excel calendar, selecting a dynamic time range of a single year or a plurality of continuous years for data statistics and visual display according to the user requirements; dragging the time slider on the animation control toolbar can display the distribution condition of the red tide information in different periods; based on the function of the Excel dynamic chart, selecting a corresponding project for visual display after data screening, and facilitating a user to split, check and analyze a data single-point project; the corresponding project is a time node or a partition area;

(2) analyzing the spatial position relation and the rule of the red tide information by using an Excel three-dimensional map: for red tide data containing position information, the data are quickly mapped onto a virtual earth through acquisition and identification of data space coordinate information, contents to be expressed by the data are displayed in a two-dimensional plane map or a three-dimensional map, the thematic map is dynamically analyzed and three-dimensionally displayed, form data with space information are converted into the thematic map, a data space visualization product is realized, and the visibility and the analysis efficiency of the data are improved;

(3) and verifying the effectiveness and feasibility of the mining of the red tide information by the fusion mode based on the red tide event historical record original data.

7. The visual mining and application method of red tide data as claimed in claim 1, wherein in step 5), an instrument panel model is constructed, and the specific method for designing a relatively universal red tide dynamic analysis chart is as follows: the method comprises the steps of designing a dynamic chart aiming at the red tide occurrence conditions of different alga species and different months, constructing a red tide occurrence frequency instrument panel model, integrating perspective table screening, mini-graph display and multiple visual designs of the instrument panel model according to user requirements, dynamically displaying the distribution characteristics of the red tide occurrence conditions, simultaneously newly adding two alga species red tide outbreak condition comparison chart designs, and enabling a user to independently select two alga species according to requirements to perform comparison analysis and search the rules of the red tide occurrence of different alga species.

Images