CN110968636A - Multi-dimensional big data analysis and processing system for earthquake early warning - Google Patents
Multi-dimensional big data analysis and processing system for earthquake early warning Download PDFInfo
- Publication number
- CN110968636A CN110968636A CN201811129618.2A CN201811129618A CN110968636A CN 110968636 A CN110968636 A CN 110968636A CN 201811129618 A CN201811129618 A CN 201811129618A CN 110968636 A CN110968636 A CN 110968636A
- Authority
- CN
- China
- Prior art keywords
- data
- earthquake
- layer
- analysis
- early warning
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012545 processing Methods 0.000 title claims abstract description 39
- 238000007405 data analysis Methods 0.000 title claims abstract description 15
- 238000005516 engineering process Methods 0.000 claims abstract description 19
- 238000004458 analytical method Methods 0.000 claims abstract description 17
- 238000004140 cleaning Methods 0.000 claims abstract description 6
- 230000010354 integration Effects 0.000 claims abstract description 6
- 238000005065 mining Methods 0.000 claims abstract description 6
- 238000007781 pre-processing Methods 0.000 claims abstract description 6
- 238000013501 data transformation Methods 0.000 claims abstract description 5
- 230000009467 reduction Effects 0.000 claims abstract description 5
- 238000000034 method Methods 0.000 claims description 10
- 230000008569 process Effects 0.000 claims description 6
- 238000007726 management method Methods 0.000 claims description 4
- 230000000007 visual effect Effects 0.000 claims description 4
- 238000003909 pattern recognition Methods 0.000 claims description 3
- 238000013135 deep learning Methods 0.000 claims description 2
- 238000012544 monitoring process Methods 0.000 abstract description 15
- 230000004927 fusion Effects 0.000 abstract description 10
- 230000005540 biological transmission Effects 0.000 description 4
- 238000007418 data mining Methods 0.000 description 4
- 230000009286 beneficial effect Effects 0.000 description 3
- 230000003203 everyday effect Effects 0.000 description 3
- 230000007246 mechanism Effects 0.000 description 3
- 239000002243 precursor Substances 0.000 description 3
- 238000012800 visualization Methods 0.000 description 3
- 238000013079 data visualisation Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 238000012806 monitoring device Methods 0.000 description 2
- 230000008520 organization Effects 0.000 description 2
- 206010052428 Wound Diseases 0.000 description 1
- 208000027418 Wounds and injury Diseases 0.000 description 1
- 230000001133 acceleration Effects 0.000 description 1
- 230000002776 aggregation Effects 0.000 description 1
- 238000004220 aggregation Methods 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 230000015572 biosynthetic process Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000005672 electromagnetic field Effects 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000001939 inductive effect Effects 0.000 description 1
- 239000005433 ionosphere Substances 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 230000010076 replication Effects 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 238000000638 solvent extraction Methods 0.000 description 1
- 238000003786 synthesis reaction Methods 0.000 description 1
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/10—Services
- G06Q50/26—Government or public services
Landscapes
- Business, Economics & Management (AREA)
- Tourism & Hospitality (AREA)
- Marketing (AREA)
- Primary Health Care (AREA)
- Health & Medical Sciences (AREA)
- Economics (AREA)
- General Health & Medical Sciences (AREA)
- Human Resources & Organizations (AREA)
- Development Economics (AREA)
- Educational Administration (AREA)
- Strategic Management (AREA)
- Physics & Mathematics (AREA)
- General Business, Economics & Management (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Geophysics And Detection Of Objects (AREA)
- Alarm Systems (AREA)
Abstract
The invention discloses a multidimensional big data analysis and processing system facing earthquake early warning, which comprises: the system comprises a data acquisition layer, a data processing layer and an application layer; the data acquisition layer acquires source data from each big data source through a sensor, and performs data preprocessing such as data cleaning, data integration, data reduction, data transformation and the like on the acquired source data; the data processing layer receives the data preprocessed by the data acquisition layer and is mainly responsible for carrying out fusion and analysis, mining and statistics on the data and filing and retrieval of the data; the application layer is responsible for providing services such as earthquake prediction early warning, emergency rescue service, earthquake disaster analysis, earthquake multidimensional display and the like by applying results of the data processing layer after the data is completely mined and analyzed, and organically fusing and processing the ground monitoring data and the space-based observation data by adopting a big data technology to provide technical services for decision support.
Description
Technical Field
The invention belongs to the technical field of data mining, and relates to a multidimensional big data analysis and processing system for earthquake early warning.
Background
The disaster damage caused by earthquake is widely accepted internationally, and the damage to human is difficult to say, so that the monitoring of earthquake by governments in various countries is highly regarded. China is a country with multiple earthquakes, and the earthquakes already bring huge disasters and spiritual wounds to people in China. After the earthquake of the chenchenchen stations, the earthquake ground observation network covering the whole country and mainly monitoring important areas is gradually built in China, at present, more than 1500 earthquake stations are built, data generated every day reaches T-level volume, the number of subjects including underground water, geomagnetism, strain, stress, earthquake waves and the like is up to T, and the method has the characteristic of continuously and multi-angle observation of one area. The earthquake satellite monitoring is a novel monitoring means which is introduced in recent years, can observe earth change all weather and all time, and collects various types of information such as space electromagnetic field, ionosphere temperature, ion concentration and the like, and the French government has gone ahead. The China government launched one earthquake electromagnetic satellite in 2017, so that the full coverage monitoring in China is realized, the amount of earthquake observation data acquired by the arrival time satellite every day is huge, reaches hundreds of G every day, and relates to a plurality of subject parameters, wherein the parameters comprise various data structures, data types, graphic images, streaming media data and other forms. The space satellite observation and the ground station monitoring correspond remotely, the monitoring of the earthquake event can be more accurately realized through the organic integration of the space satellite observation and the ground station monitoring, the acquired data can provide a large amount of multi-parameter basic data information for earthquake monitoring analysis, and a huge data set is formed.
By analyzing and researching the multi-dimensional and continuous data information and mining the unknown rule knowledge, scientists can understand the earthquake more deeply, understand the mechanism of the earthquake source, reveal the mechanism and the geological activity rule of the earthquake, analyze the earthquake involved range, evaluate the disaster damage condition and the like, so as to carry out the work of prediction and early warning before earthquake, emergency rescue after earthquake and the like, and reduce the loss to the maximum extent. Therefore, the earthquake big data has great value and provides an uncommon opportunity for the development of the earthquake industry. However, seismic big data also brings great difficulties for data storage, transmission, processing, retrieval, display, and the like.
Disclosure of Invention
The invention aims to overcome the defects of the prior art and provide a multidimensional big data analysis and processing system facing earthquake early warning, which adopts a big data technology to organically fuse and process ground monitoring data and space-based observation data, realizes the centralized storage of line data, is more beneficial to data processing and analysis, is convenient to find earthquake regularity knowledge and disaster assessment problems caused by earthquake, provides technical service for decision support, and effectively solves the difficulties of storage, transmission, processing, retrieval and display of earthquake big data.
In order to solve the technical problems, the invention adopts the following technical scheme: a multi-dimensional big data analysis and processing system for earthquake early warning, which comprises: the system comprises a data acquisition layer, a data processing layer and an application layer; the data acquisition layer acquires source data from each big data source through a sensor, and performs data preprocessing such as data cleaning, data integration, data reduction, data transformation and the like on the acquired source data; the data processing layer receives the preprocessed data of the data acquisition layer and is mainly responsible for carrying out fusion and analysis, mining and statistics on the data and filing and retrieval of the data; the application layer is responsible for passing the data processing layer
For data
The result after the complete mining and analysis is applied to earthquake prediction and early warning, emergency rescue service, earthquake disaster analysis, earthquake multi-dimensional display and the like to provide services.
Further, the preprocessed data of the data acquisition layer is subjected to distributed storage management by adopting a NoSQL database in a cloud environment.
Furthermore, the data processing layer mainly adopts a MapReduce technology model, a deep learning technology, a visual display and virtual reality technology and a pattern recognition technology to effectively analyze and process the data.
Compared with the prior art, the invention has the following beneficial effects:
the scheme of the invention adopts a big data technology to organically fuse and process the ground monitoring data and the space-based observation data, realizes the centralized storage of the line data, is more beneficial to data processing and analysis, is convenient to find earthquake regularity knowledge and disaster assessment problems caused by earthquake, provides technical service for decision support, and effectively solves the difficulties of storage, transmission, processing, retrieval and display of earthquake big data.
Drawings
Fig. 1 is an overall framework diagram of a multidimensional big data analysis and processing system facing earthquake early warning.
Detailed Description
The present invention will now be described more fully hereinafter with reference to the accompanying drawings, in which embodiments of the invention are shown. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting of the invention.
Referring to fig. 1, the invention relates to a multidimensional big data analysis and processing system facing earthquake early warning, which comprises: the system comprises a data acquisition layer, a data processing layer and an application layer; the data acquisition layer acquires source data from each big data source through a sensor, and performs data preprocessing such as data cleaning, data integration, data reduction, data transformation and the like on the acquired source data; the data processing layer receives the preprocessed data of the data acquisition layer and is mainly responsible for carrying out fusion and analysis, mining and statistics on the data and filing and retrieval of the data; the application layer is used for providing services such as earthquake prediction early warning, emergency rescue service, earthquake disaster analysis, earthquake multidimensional display and the like through applying results of the data processing layer after the data are completely mined and analyzed.
The processing of the seismic monitoring big data is a complex process and comprises the steps of data acquisition, storage, preprocessing, warehousing, data analysis, knowledge discovery and knowledge application.
Various monitoring devices acquire massive multivariate data, and due to the huge data volume, the monitoring devices have a challenge in the storage stage. Seismic survey data originates from thousands of ground stations and space satellites distributed throughout the country. The data of the seismic stations are gathered step by step until the data reach the national station network center and are stored at a plurality of positions; the seismic satellite data is stored in a ground receiving station and can be regarded as an independent big data system. In consideration of the safety and confidentiality of seismic data, the seismic monitoring big data can be stored in a distributed-centralized mode, and data transmission is carried out by adopting a communication private network.
A cloud storage system is built by relying on an earthquake digital private network, each station is a cloud node, generation nodes, aggregation nodes and the like are aggregated step by step until a national platform network center, and a tree-shaped cloud storage structure covering the whole country is formed. Meanwhile, a plurality of secondary central nodes are established, a plurality of backup libraries are established, tasks and loads of national central nodes are shared, and a 'mesh' storage structure is formed.
The big data problem is mainly a data processing problem. Therefore, the seismic observation data need to be worth, and data analysis and processing are necessary. Although distributed processing techniques, cloud computing, and the like can meet some requirements, data processing methods are currently more suitable for centrally storing data. Therefore, the centralized storage of the data is carried out in the national center, the secondary center and the satellite receiving station, and the data processing and analysis are more facilitated.
High quality data is an important guarantee for knowledge discovery. However, the data collected by the sensors always have the problems of noise, missing, duplication, inconsistency and the like, and therefore, the data needs to be preprocessed. The preprocessing method mainly comprises data cleaning, data integration, data reduction, data transformation and the like.
The big data analysis technology is a basic task of a big data system, and needs to perform basic data operations on obtained data information, such as statistics, query, report forms, OLAP analysis, graph visualization analysis, data feature analysis, and the like, so as to provide basic data feature information for further knowledge discovery.
The final purpose of big data analysis is to serve people, and the earthquake observation big data processing system provides services for earthquake prediction analysis, earthquake early warning processing, emergency rescue, virtual reality, scene simulation, GIS synthesis, knowledge display and the like, helps people to make scientific decisions and reasonably arrange tasks before, during and after an earthquake disaster, and reduces loss to the maximum extent.
The first problem faced by big earthquake data is storage, and the traditional storage mode cannot meet the requirements. In the cloud storage, a plurality of cheap storage devices are integrated into storage resources which are put to the cloud end to provide access service for users, the storage space is saved through a virtualization technology, the storage efficiency is improved, and the elastic expansion is realized. The acceleration of mass data and the uncertainty of data organization need to be supported by a cloud storage system which is easy to expand, manage, flexible and low in cost. The NoSQL database is a distributed and non-relational database system for managing large-scale data sets, supports high concurrent reading and writing of structured and semi-structured data, and has great advantages on storage management and distributed parallel computation of mass data. One key feature of NoSQL is the ability to achieve "shared nothing" horizontal scaling, replication and data partitioning on multiple servers, and to perform efficient storage management of mass data in web2.0 environment.
For the observation of an earthquake in a region, station observation data from a foundation and satellite observation information of a space foundation exist, so that precious data resources acquired by the sensors need to be comprehensively considered, and an information fusion technology needs to be adopted. Data fusion is the collaborative combination of two or more image data for an event, with the expectation that more knowledge will be acquired than for a single image data. In the aspect of satellite remote sensing image fusion, remote sensing image fusion of multiple sensors at the same moment and remote sensing image fusion with complementary information under different imaging conditions belong to reasonable information fusion. Compared with a single information source image, the fused image is clearer, higher in resolution and better in visual effect. If the information fusion technology improves the precision of the seismic big data from the breadth, the data mining technology is more like regularly exploring the seismic big data from the depth. The seismic observation big data of the space-based and foundation have different levels of data mining methods in the aspects of semantic expression, information organization, knowledge discovery and the like, and the space-time big data also provides a stage which fully plays a role for the data mining technology.
Data visualization provides information services to people in a visual effect and in a manner consistent with human thinking, which is particularly important in the big data era. The large data is large in size, various in types and large in uncertain information ratio, so that the processing efficiency and the calculation result of a large data system are not ideal, and people can hardly imagine the problems in the large data system. The earthquake big data visualization and virtual reality display technology can display a huge space-based data set and a huge foundation data set in an intuitive form, people can observe the ingenuity and problems in the space-based data set, realize the multidimensional and dynamic visualization representation of earthquake elements, earthquake occurrence processes, earthquake monitoring and forecasting, earthquake generation mechanisms and the like, and provide intuitive information service for earthquake precursor early warning and earthquake-later emergency rescue.
Different information formats acquired by different observation stations and satellites are different, disciplines are various, and no structured data is abundant, so that the information needs to be processed by a NoSQL technology of a private cloud; the result is transmitted to a superior sink node through a digital private network, and enters a central node after being preprocessed by data cleaning and the like; performing pattern recognition and visualization analysis by combining the space-based data result processed by the satellite ground station, and determining the earthquake precursor information state; and finally, accurately positioning the earthquake-inducing precursor based on a GIS system.
The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (3)
1. The multidimensional big data analysis and processing system for earthquake early warning is characterized by comprising: the system comprises a data acquisition layer, a data processing layer and an application layer; the data acquisition layer acquires source data from each big data source through a sensor, and performs data preprocessing such as data cleaning, data integration, data reduction, data transformation and the like on the acquired source data; the data processing layer receives the preprocessed data of the data acquisition layer and is mainly responsible for fusing and analyzing the data, mining and counting the data and filing and retrieving the data; the application layer is used for providing services such as earthquake prediction early warning, emergency rescue service, earthquake disaster analysis, earthquake multidimensional display and the like through applying results of the data processing layer after the data are completely mined and analyzed.
2. The earthquake early warning-oriented multidimensional big data analysis and processing system according to claim 1, wherein the preprocessed data of the data acquisition layer is distributed storage and management by adopting a NoSQL database in a cloud environment.
3. The earthquake early warning-oriented multidimensional big data analysis and processing system according to claim 1, wherein the data processing layer mainly adopts a MapReduce technology model, a deep learning technology, a visual display and virtual reality technology and a pattern recognition technology to effectively analyze and process data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811129618.2A CN110968636A (en) | 2018-09-28 | 2018-09-28 | Multi-dimensional big data analysis and processing system for earthquake early warning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811129618.2A CN110968636A (en) | 2018-09-28 | 2018-09-28 | Multi-dimensional big data analysis and processing system for earthquake early warning |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110968636A true CN110968636A (en) | 2020-04-07 |
Family
ID=70026569
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811129618.2A Pending CN110968636A (en) | 2018-09-28 | 2018-09-28 | Multi-dimensional big data analysis and processing system for earthquake early warning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110968636A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112882089A (en) * | 2021-03-12 | 2021-06-01 | 太原美信达科技有限公司 | Earthquake forecasting system based on earth change magnetic field |
| CN113534238A (en) * | 2020-04-18 | 2021-10-22 | 中国石油化工股份有限公司 | System and method for data acquisition and data mining in seismic data processing process |
| CN113643425A (en) * | 2021-07-28 | 2021-11-12 | 中石化石油工程技术服务有限公司 | Petroleum and seismic result three-dimensional space visualization universal platform construction method based on open source GIS technology |
| CN114114382A (en) * | 2021-12-01 | 2022-03-01 | 北京大学深圳研究生院 | Monitoring data processing method for earthquake prediction, earthquake prediction method and system |
| CN114443883A (en) * | 2022-02-10 | 2022-05-06 | 北京永利信达科技有限公司 | Data processing method, system and medium based on big data and cloud computing |
-
2018
- 2018-09-28 CN CN201811129618.2A patent/CN110968636A/en active Pending
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113534238A (en) * | 2020-04-18 | 2021-10-22 | 中国石油化工股份有限公司 | System and method for data acquisition and data mining in seismic data processing process |
| CN113534238B (en) * | 2020-04-18 | 2024-03-29 | 中国石油化工股份有限公司 | System and method for data acquisition and data mining in seismic data processing process |
| CN112882089A (en) * | 2021-03-12 | 2021-06-01 | 太原美信达科技有限公司 | Earthquake forecasting system based on earth change magnetic field |
| CN112882089B (en) * | 2021-03-12 | 2023-08-01 | 太原美信达科技有限公司 | Earthquake prediction system based on earth changing magnetic field |
| CN113643425A (en) * | 2021-07-28 | 2021-11-12 | 中石化石油工程技术服务有限公司 | Petroleum and seismic result three-dimensional space visualization universal platform construction method based on open source GIS technology |
| CN114114382A (en) * | 2021-12-01 | 2022-03-01 | 北京大学深圳研究生院 | Monitoring data processing method for earthquake prediction, earthquake prediction method and system |
| CN114114382B (en) * | 2021-12-01 | 2023-10-13 | 北京大学深圳研究生院 | Monitoring data processing method for earthquake prediction, earthquake prediction method and system |
| CN114443883A (en) * | 2022-02-10 | 2022-05-06 | 北京永利信达科技有限公司 | Data processing method, system and medium based on big data and cloud computing |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110968636A (en) | Multi-dimensional big data analysis and processing system for earthquake early warning | |
| Li et al. | Big data in smart cities | |
| Krishnan et al. | OpenTopography: a services oriented architecture for community access to LIDAR topography | |
| CN103281351B (en) | Cloud service platform for high-efficiency remote sensing data processing and analysis | |
| Li et al. | Big data and cloud computing | |
| Xia et al. | Building a spatiotemporal index for earth observation big data | |
| CN109739820A (en) | A kind of E-government information service system based on big data analysis | |
| Zou et al. | MapReduce functions to remote sensing distributed data processing—Global vegetation drought monitoring as example | |
| Fang et al. | A service-oriented collaborative approach to disaster decision support by integrating geospatial resources and task chain | |
| Rahman et al. | The role of big data in disaster management | |
| He et al. | Earthquake networks based on space–time influence domain | |
| Albrecht et al. | Next-generation geospatial-temporal information technologies for disaster management | |
| CN106453468A (en) | Coal mining water inrush early warning method based on GIS technology | |
| Shuliang et al. | Big spatial data mining | |
| Zhang et al. | Map generation from large scale incomplete and inaccurate data labels | |
| Huang et al. | Spatial characteristics of coronavirus disease 2019 and their possible relationship with environmental and meteorological factors in Hubei province, China | |
| Chen et al. | Implementation of a geological disaster monitoring and early warning system based on multi-source spatial data: a case study of Deqin county, Yunnan province | |
| CN114372034A (en) | Access method based on remote sensing image map service | |
| Wu et al. | Research on data sharing architecture for ecological monitoring using Iot streaming data | |
| Steffenel et al. | Using a pervasive computing environment to identify secondary effects of the antarctic ozone hole | |
| Jo et al. | Constructing national geospatial big data platform: current status and future direction | |
| Peng et al. | Research on synchronization technology of geological data acquisition system based on NFC and mobile internet | |
| Feng et al. | [Retracted] Internet of Things System of Spatial Structure Sports Events Health Monitoring Based on Cloud Computing | |
| Li et al. | Research on spatial data management and application | |
| Wang et al. | A study on the organizational architecture and standard system of the Data Sharing Network of Earth System Science in China |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20200407 |