CN112667704A - Coal mine industry internet data middle platform system structure - Google Patents

Coal mine industry internet data middle platform system structure Download PDF

Info

Publication number
CN112667704A
CN112667704A CN202011534549.0A CN202011534549A CN112667704A CN 112667704 A CN112667704 A CN 112667704A CN 202011534549 A CN202011534549 A CN 202011534549A CN 112667704 A CN112667704 A CN 112667704A
Authority
CN
China
Prior art keywords
data
platform
big data
coal mine
big
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202011534549.0A
Other languages
Chinese (zh)
Inventor
张晓霞
苏上海
李首滨
李�昊
施展
冯月利
方乾
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Coal Science Research Institute
China Coal Research Institute CCRI
Original Assignee
Coal Science Research Institute
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Coal Science Research Institute filed Critical Coal Science Research Institute
Priority to CN202011534549.0A priority Critical patent/CN112667704A/en
Publication of CN112667704A publication Critical patent/CN112667704A/en
Pending legal-status Critical Current

Links

Images

Landscapes

  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The application provides a coal mine industry internet data middle platform architecture, includes: the system comprises a big data acquisition platform, a big data base platform, a big data management platform and a big data analysis platform; the big data acquisition platform is used for butting all data sources of a coal mine and acquiring data of all the data sources to acquire acquired data; the big data base platform is used for storing and calculating the acquired data; the big data management platform is used for managing and managing the acquired data stored by the big data basic platform; and the big data analysis platform is used for analyzing the acquired data and the processed data. Therefore, the centralized summarization of the data of all data sources such as all the service systems, the production systems, the monitoring systems and the like of the coal mine is realized, and the data value is mined in the data, so that the service is provided for the safety production of the coal mine, and the safety and the efficiency of the production are greatly improved.

Description

Coal mine industry internet data middle platform system structure
Technical Field
The application relates to the technical field of coal mine data processing, in particular to a coal mine industrial internet data middle platform system structure.
Background
In modern coal mining, various underground information provides guarantee for safe and efficient production of coal mines. With the development of science and technology, coal mines comprise more and more systems, and how to mine data values from a large amount of data generated by each system to provide services for the safety production of coal mines is a problem to be solved urgently.
Disclosure of Invention
The embodiment of the application provides a coal mine industry internet data middle platform architecture, includes: the system comprises a big data acquisition platform, a big data base platform, a big data management platform and a big data analysis platform; wherein the content of the first and second substances,
the big data acquisition platform is used for butting all data sources of a coal mine and acquiring data of all the data sources to obtain acquired data;
the big data base platform is used for storing and calculating the acquired data;
the big data governance platform is used for managing and governing the acquired data stored by the big data basic platform;
and the big data analysis platform is used for analyzing the acquired data and the processed data thereof.
In a possible implementation manner of the embodiment of the present application, the big data acquisition platform is specifically configured to:
and converting the communication protocol into a uniform format based on a protocol conversion technology, and acquiring real-time data with different sampling frequencies in the industrial equipment accessed to the big data acquisition platform.
In a possible implementation manner of the embodiment of the application, the big data acquisition platform is further configured to send data in the big data acquisition platform to a cloud and receive the data sent by the cloud based on a data transmission protocol.
In a possible implementation manner of the embodiment of the present application, the big data base platform is specifically configured to:
and according to the data type, performing distributed storage on the acquired data, wherein the data type comprises at least one of a relational type, a key value type and a full-text retrieval type.
In a possible implementation manner of the embodiment of the present application, the big data base platform is further configured to: the distributed computation is carried out on the collected data, the control and safety management are carried out on the data access, and the management is carried out on the user authority.
In a possible implementation manner of the embodiment of the present application, the big data governance platform is specifically configured to: and carrying out standard management, metadata management, quality management, safety management and life cycle management on the acquired data.
In a possible implementation manner of the embodiment of the application, the big data analysis platform includes a visualization analysis component, where the visualization analysis component includes at least one of a histogram component, a line graph component, a bar graph component, a pie graph component, and a scatter graph component.
In a possible implementation manner of the embodiment of the application, the big data analysis platform is specifically configured to perform data analysis based on the visual analysis component and perform multidimensional data analysis and report customization display based on a graphical dragging technology.
In a possible implementation manner of the embodiment of the present application, the big data analysis platform is further configured to construct a data model and an analytic data warehouse for data analysis.
Additional aspects and advantages of the present application will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the present application.
Drawings
The foregoing and/or additional aspects and advantages of the present application will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:
FIG. 1 is a schematic diagram of a coal mine industry Internet data center architecture provided by an embodiment of the present application;
FIG. 2 is a schematic diagram of data sources of a coal mine provided by an embodiment of the present application;
fig. 3 is a schematic data processing process diagram of a data center platform architecture according to an embodiment of the present application.
Detailed Description
Reference will now be made in detail to embodiments of the present application, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the drawings are exemplary and intended to be used for explaining the present application and should not be construed as limiting the present application.
The coal mine industry internet data center architecture of the embodiment of the application is described below with reference to the accompanying drawings.
Fig. 1 is a schematic diagram of a coal mine industry internet data center system structure provided in an embodiment of the present application.
The embodiment of the application provides a coal mine industrial internet data center system structure, which is used for centralizing and summarizing data of all data sources such as all service systems, production systems and monitoring systems of all butted coal mine data systems and mining data values in the data.
As shown in fig. 1, a coal mine industry internet data center platform architecture 100 according to an embodiment of the present application includes: big data collection platform 110, big data base platform 120, big data governance platform 130 and big data analysis platform 140.
The big data acquisition platform 110 is configured to interface with each data source of the coal mine, and perform data acquisition on each data source to obtain acquired data.
Fig. 2 is a schematic diagram of data sources of a coal mine according to an embodiment of the present disclosure.
In this embodiment, as shown in fig. 2, the coal mine includes a monitoring system, a production system, a service system, and the like. The monitoring and monitoring system can monitor a working face, a ventilation system, a safety system, a mine pressure system, a hydrological system, a fire area, a lifting system, a power supply line system, a washing and selecting system, a personnel system, geology, coal dust, main transportation bear, a drainage system, a loading system, a vehicle, an industrial television, rock burst and the like.
The production system comprises a fully mechanized mining face, a tunneling face, a main conveying system, an auxiliary conveying system, a power supply system, a drainage system, a ventilation system, a washing system, a loading system and the like.
The business system comprises a ground measuring system, production scheduling, ventilation, prevention, mechanical and electrical management, production technology, safety supervision, coal quality management, staff training, emergency rescue and the like.
During data acquisition, industrial equipment can be accessed to a big data acquisition platform based on an industrial Ethernet, an Internet of things, a location service network, a 5G network industrial network and the like so as to acquire data of each data source, wherein the data source is the described coal mine system.
The large data acquisition platform aims to design an elastic, universal and easily-deployed framework, butt joint each data source, deploy in an invasive or non-invasive mode, create metadata in the acquisition process and collect the metadata of the data for subsequent treatment and analysis.
The intrusive mode refers to a class provided by a user code needing to inherit the architecture; the non-intrusive mode does not need the information that the user code introduces the architecture code, and from the point of view of class writers, the existence of the architecture cannot be perceived.
In one embodiment, the big data collection platform 110 is specifically configured to convert a communication protocol into a uniform format based on a protocol conversion technique, and collect real-time data of different sampling frequencies in an industrial device accessing the big data collection platform 110.
In another embodiment, the big data collection platform 110 can also send data in the big data collection platform 110 to the cloud end through a data transmission protocol, and receive data sent by the cloud end. From this, data transmission in the big data acquisition platform to the high in the clouds, with data storage in the high in the clouds, not only save a large amount of costs for the coal mine enterprise, the security is high moreover.
In addition, the big data acquisition platform 110 can also perform preprocessing such as cleaning, data transformation, data reduction and the like on the acquired data so as to save storage space and improve data quality.
And the big data base platform 120 is used for storing and calculating the acquired data acquired by the big data acquisition platform 110. Specifically, different data storage manners may be selected for different data characteristics. The data characteristics comprise an industrial protocol, acquisition frequency, a data format, a data analysis mode and the like.
For example, data with the same collection frequency can be stored in the same disk, data with the same data format can be stored in the same disk, and the like. For example, the data format may be a relational type, a key type, or the like.
Taking a coal mine working face as an example, the coal mine working face data mainly comprises working face sensor data and monitoring video data. The sensor data is point-like time sequence data, each sensor only monitors certain or certain state information of a single device and only generates data when the data changes, and the data requirement is to support fast query of the data of a certain device in a certain time period.
High frequency data also exists in downhole equipment, and for storage and cleaning of high frequency data, a storage engine is required that supports high speed writing. For the storage of the monitoring video data, the storage of the video file itself is guaranteed.
In one embodiment, the big data base platform 120 may perform distributed storage of the collected data according to the data type, such as storing relational data into a distributed data environment, storing key-value data into a distributed data environment, and storing full-text retrieval data into a distributed data environment. In the embodiment, the distributed storage adopts an expandable system structure, uses a plurality of storage servers to store data, can be transversely expanded, and is suitable for storing massive coal mine data.
When the relational data are stored in a distributed mode, different storage strategies can be used according to different characteristics such as quick writing and quick reading; the distributed storage of full-text retrieval type data can also support Chinese and English word segmentation, and can quickly retrieve and query any field content in the data.
In the embodiment, the data are stored in a distributed manner, so that the storage efficiency is improved, and the data can be managed and analyzed conveniently in the later period.
In another embodiment, the big data base platform 120 may also perform distributed computation on the collected data, control and security management on data access, and management on user rights.
Specifically, the distributed computation can be applied to data preprocessing, data cleaning, data modeling and the like, and can be divided into offline batch processing and online real-time processing according to upper-layer business requirements. Specifically, data with low real-time requirements can be processed in batch off-line, such as geological data and the like; the data with high real-time performance, such as data related to a tunneling working face, can be processed on line in real time.
The distributed data offline batch processing supports good expansibility, fault tolerance and high throughput rate; the distributed data is processed on line in real time, the timeliness of data processing needs to be ensured, and each piece of data is processed without leakage; the data modeling method also has great influence on the efficiency of data processing, for example, the punctiform pressure data returned by the hydraulic support can be used for constructing a transient snapshot of the mine pressure for the use of a mine pressure analysis service.
Wherein the data modeling method is generally determined according to the data and the application scenario. For example, a traditional modeling mode can be adopted when the data volume is small, and deep learning can be considered when the data volume is large; if the label data is the label data, supervised learning can be selected; if it is non-label data, unsupervised learning may be selected. Wherein, if the labels are discrete in the supervised learning, a classification algorithm can be selected; if the labels are continuous, a regression algorithm or the like may be selected.
The big data base platform 120 can control and safely manage data access from the outside and non-trusted roles, realize 4A-level unified safety management of the data platform, and also can manage user authority of users, user groups and roles, realize account information management of the users and the user groups and control based on role access authority.
And the big data governance platform 130 is used for managing and governing the collected data stored by the big data base platform 120 so as to obtain the management data.
In this embodiment, the big data governance platform 130 is specifically configured to perform standard management, metadata management, quality management, security management, life cycle management, and the like on the collected data.
For example, incomplete data, defective data, data with duplicates, data with errors, etc. are processed to obtain usable and valid data for subsequent analysis.
The data is subjected to standard management, specifically, the service attribute and the technical attribute of the data are subjected to standard definition, so that the consistent understanding of the data is ensured; metadata management is carried out on the data, and particularly, enterprise metadata and managed enterprise metadata can be uniformly collected to provide technical extension support for data governance such as data standard, data quality and data asset management; performing quality management on data, specifically establishing a control mechanism by determining a data quality management target to ensure the authenticity, accuracy, continuity, integrity and timeliness of the data; the data are subjected to security management, specifically, data security standards and strategies are defined, a data security prevention and control system is designed according to data security requirements and supervision requirements, and the data are guaranteed to be legally and legally compliant, safely collected, transmitted, stored and used.
Because the particularity of the coal industry lies in interaction with the nature, the working surface is dynamically propelled, and data loss and other problems may be caused by safety accidents, platform security (including providing security assurance in acquisition, transmission, storage and other environments, which involve physical security of software and hardware facilities, network security and the like) and data security (including data encryption, desensitization, destruction and the like) need to be ensured.
To facilitate the mining of data values from data collected from various data sources, the collected data and processed data thereof may be analyzed by big data analysis platform 140.
In this embodiment, the big data analysis platform 140 may directly analyze the data collected by the big data collection platform 110, analyze the data obtained by processing the collected data by the big data management platform, or analyze the data stored in the big data base platform.
In performing the data analysis, the analysis can be performed in a variety of ways.
As one implementation, big data analytics platform 140 may include a visual analytics component through which data analytics are performed. The visualization analysis component includes, but is not limited to, a bar graph component, a line graph component, a bar graph component, a pie graph component, a scatter plot component, and the like. Therefore, the data analysis function can be used for acquiring the bar chart, the line chart, the bar chart, the pie chart, the scatter diagram and the like of the data by using the visual analysis component, the characteristics, the rules and the like of the data can be acquired by analyzing the diagrams, the data value is mined, and the visual analysis component is used for analyzing the data, so that the method is simple and visual.
As another implementation, the big data analysis platform 140 may further perform multidimensional data analysis and report customized display based on a graphical drag-and-drop technique. Taking the fully mechanized mining face as an example, the multiple dimensions may be a time dimension, a face dimension, a mine dimension, an equipment dimension, and the like, and analysis (e.g., analysis of rules of drilling, rotation, slicing, and the like) is performed based on these dimensions.
In addition, the big data analysis platform 140 may also construct a data model and an analytical data warehouse for data analysis, so that data can be analyzed quickly and in multiple dimensions through the data model and the analytical data warehouse.
Fig. 3 is a schematic data processing process diagram of a data center platform architecture according to an embodiment of the present application.
As shown in fig. 3, the big data collection platform may be used for automatic acquisition of downhole real-time data, such as working face propulsion condition data, mining area data, environmental data, personnel data, video data, and the like. The big data base platform is used for providing a distributed storage and calculation platform for processing mass data, and can be expanded along with the increase of the data size. The big data management platform can perform data processing on data in the big data basic platform, for example, data management and management can be performed, specifically, data processing can be performed on the data through a method system and a tool set, so that the problems of data accuracy, quality, sharing, safety and the like are solved, and data assets are formed. The big data analysis platform is used for data application and analysis and can provide functions of data query statistical analysis, algorithm model service and the like.
As shown in fig. 3, after the collected working face propulsion condition data, mining area data and the like sequentially pass through the data collection platform, the big data base platform, the data governance platform and the big data analysis platform, a working face pressure change rule, a relation between equipment and surrounding rock space, time and pressure, equipment operation and efficiency management, equipment predictive maintenance and repair and the like can be obtained.
It should be noted that the direction of the data flow in fig. 3 is only an example, specifically, the direction of the data flow may be different in different scenarios, for example, the big data acquisition platform directly sends the acquired data to the big data governance platform, the big data governance platform performs management and governance, and then the big data analysis platform is used to analyze the data processed by the big data governance platform.
The coal mine industry internet data center platform system structure comprises a big data acquisition platform, a big data base platform, a big data management platform and a big data analysis platform, wherein data of all data sources can be acquired through the big data acquisition platform, the data acquired by the big data acquisition platform are stored and calculated through the big data base platform, the data are managed and managed through the big data management platform, and the data acquired by the big data analysis platform and the data after being processed are analyzed through the big data analysis platform. Therefore, the centralized summarization of the data of all data sources such as all the service systems, the production systems, the monitoring systems and the like of the coal mine is realized, and the data value is mined in the data, so that the service is provided for the safety production of the coal mine, and the safety and the efficiency of the production are greatly improved.
Although embodiments of the present application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present application, and that variations, modifications, substitutions and alterations may be made to the above embodiments by those of ordinary skill in the art within the scope of the present application.

Claims (9)

1. A coal mine industry Internet data center platform architecture, comprising: the system comprises a big data acquisition platform, a big data base platform, a big data management platform and a big data analysis platform; wherein the content of the first and second substances,
the big data acquisition platform is used for butting all data sources of a coal mine and acquiring data of all the data sources to obtain acquired data;
the big data base platform is used for storing and calculating the acquired data;
and the big data governance platform is used for managing and governing the acquired data stored by the big data basic platform.
And the big data analysis platform is used for analyzing the acquired data and the processed data thereof.
2. The coal mine industry internet data middlebox architecture of claim 1, wherein the big data collection platform is specifically configured to:
and converting the communication protocol into a uniform format based on a protocol conversion technology, and acquiring real-time data with different sampling frequencies in the industrial equipment accessed to the big data acquisition platform.
3. The coal mine industry internet data center architecture of claim 1, wherein the big data collection platform is further configured to send data in the big data collection platform to a cloud and receive data sent by the cloud based on a data transmission protocol.
4. The coal mine industry internet data staging architecture of claim 1, wherein the big data base platform is specifically configured to:
and according to the data type, performing distributed storage on the acquired data, wherein the data type comprises at least one of a relational type, a key value type and a full-text retrieval type.
5. The coal mine industry internet data staging architecture of claim 4, wherein the big data base platform is further configured to perform distributed computing on the collected data, control and security management on data access, and management of user permissions.
6. The coal mine industry internet data staging system architecture of claim 1, wherein the big data governance platform is specifically configured to: and carrying out standard management, metadata management, quality management, safety management and life cycle management on the acquired data.
7. The coal mine industry internet data center architecture of claim 1, wherein the big data analytics platform comprises a visualization analytics component, wherein the visualization analytics component comprises at least one of a histogram component, a line graph component, a bar graph component, a pie graph component, and a scatter plot component.
8. The coal mine industry internet data middling platform architecture of claim 7, wherein the big data analysis platform is specifically configured to perform data analysis based on the visualization analysis component and perform multidimensional data analysis and report customization display based on a graphical drag-and-drop technique.
9. The coal mine industry internet data staging architecture of claim 7, wherein the big data analytics platform is further configured to build a data model and an analytical data warehouse for data analytics.
CN202011534549.0A 2020-12-22 2020-12-22 Coal mine industry internet data middle platform system structure Pending CN112667704A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011534549.0A CN112667704A (en) 2020-12-22 2020-12-22 Coal mine industry internet data middle platform system structure

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011534549.0A CN112667704A (en) 2020-12-22 2020-12-22 Coal mine industry internet data middle platform system structure

Publications (1)

Publication Number Publication Date
CN112667704A true CN112667704A (en) 2021-04-16

Family

ID=75408029

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011534549.0A Pending CN112667704A (en) 2020-12-22 2020-12-22 Coal mine industry internet data middle platform system structure

Country Status (1)

Country Link
CN (1) CN112667704A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113377776A (en) * 2021-06-29 2021-09-10 中煤能源研究院有限责任公司 Intelligent mine data management system, method, equipment and readable storage medium
CN116821104A (en) * 2022-08-18 2023-09-29 南通泽烁信息科技有限公司 Industrial Internet data processing method and system based on big data

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106874671A (en) * 2017-02-15 2017-06-20 淄博祥龙测控技术有限公司 A kind of algorithm of the data-driven for spontaneous fire in coal mine big data platform
CN108510146A (en) * 2017-12-28 2018-09-07 国家安全生产监督管理总局通信信息中心 Safety of Coal Mine Production method for prewarning risk and system

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106874671A (en) * 2017-02-15 2017-06-20 淄博祥龙测控技术有限公司 A kind of algorithm of the data-driven for spontaneous fire in coal mine big data platform
CN108510146A (en) * 2017-12-28 2018-09-07 国家安全生产监督管理总局通信信息中心 Safety of Coal Mine Production method for prewarning risk and system

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
刘香兰;: "煤矿安全生产大数据分析与管理平台设计研究", 煤炭工程, no. 06 *
杜毅博 等: "智能化煤矿大数据平台架构及数据处理关键技术研究", 煤炭科学技术, vol. 48, no. 07, pages 1 - 4 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113377776A (en) * 2021-06-29 2021-09-10 中煤能源研究院有限责任公司 Intelligent mine data management system, method, equipment and readable storage medium
CN116821104A (en) * 2022-08-18 2023-09-29 南通泽烁信息科技有限公司 Industrial Internet data processing method and system based on big data

Similar Documents

Publication Publication Date Title
US11258814B2 (en) Methods and systems for using embedding from Natural Language Processing (NLP) for enhanced network analytics
US10810225B2 (en) System and method for large scale data processing of source data
Zhao et al. An empirical investigation of practical log anomaly detection for online service systems
CN109558393B (en) Data model construction method, device, equipment and storage medium
US11601453B2 (en) Methods and systems for establishing semantic equivalence in access sequences using sentence embeddings
CN112667704A (en) Coal mine industry internet data middle platform system structure
US20150032743A1 (en) Analyzing files using big data tools
Zeufack et al. An unsupervised anomaly detection framework for detecting anomalies in real time through network system’s log files analysis
US10997193B2 (en) Managing queries in business intelligence platforms
US11741379B2 (en) Automated resolution of over and under-specification in a knowledge graph
CN114880405A (en) Data lake-based data processing method and system
CN114281877A (en) Data management system and method
CN117194919A (en) Production data analysis system
Alwan et al. Big data: Definition, characteristics, life cycle, applications, and challenges
CN113051319A (en) Redis-based large key detection method, system, device and storage medium
CN110928864A (en) Scientific research project management method and system
Bello et al. Integrating downhole temperature sensing datasets and visual analytics for improved gas lift well surveillance
Gjertsen et al. IADC Dull Code Upgrade: Photometric Classification and Quantification of the New Dull Codes
CN111209158B (en) Mining monitoring method and cluster monitoring system for server cluster
Arumugam et al. Revealing Patterns within the Drilling Reports Using Text Mining Techniques for Efficient Knowledge Management
Kelleher et al. Analyzing Data from Hydraulic Workover and Coiled Tubing Services
CN111027923A (en) Integrated collaborative management application platform based on petroleum big data
Saidi et al. Why the approach big data, is essential for the reliability-centered maintenance
CN111984499B (en) Fault detection method and device for big data cluster
Dykusova et al. Digitalization of the fuel and energy industry: A study of new technologies and their potential

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
CB02 Change of applicant information

Address after: 100013 12th floor, coal building, No.13 Heping Street, Chaoyang District, Beijing

Applicant after: General Coal Research Institute Co.,Ltd.

Address before: 100013 12th floor, coal building, No.13 Heping Street, Chaoyang District, Beijing

Applicant before: CHINA COAL Research Institute

CB02 Change of applicant information