CN111126873A - Shield TBM big data platform system based on Hadoop cluster architecture and construction method - Google Patents
Shield TBM big data platform system based on Hadoop cluster architecture and construction method Download PDFInfo
- Publication number
- CN111126873A CN111126873A CN201911399090.5A CN201911399090A CN111126873A CN 111126873 A CN111126873 A CN 111126873A CN 201911399090 A CN201911399090 A CN 201911399090A CN 111126873 A CN111126873 A CN 111126873A
- Authority
- CN
- China
- Prior art keywords
- data
- module
- hadoop cluster
- server
- analysis
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000010276 construction Methods 0.000 title claims abstract description 33
- 238000007726 management method Methods 0.000 claims abstract description 53
- 238000004458 analytical method Methods 0.000 claims abstract description 51
- 238000012544 monitoring process Methods 0.000 claims abstract description 39
- 238000000034 method Methods 0.000 claims abstract description 18
- 238000007405 data analysis Methods 0.000 claims abstract description 10
- 230000008569 process Effects 0.000 claims abstract description 10
- 238000003860 storage Methods 0.000 claims abstract description 8
- 238000004140 cleaning Methods 0.000 claims abstract description 7
- 238000013499 data model Methods 0.000 claims abstract description 4
- 238000005259 measurement Methods 0.000 claims description 8
- 238000009430 construction management Methods 0.000 claims description 6
- 230000005540 biological transmission Effects 0.000 claims description 5
- 238000012098 association analyses Methods 0.000 claims description 4
- 238000005065 mining Methods 0.000 claims description 3
- 230000001502 supplementing effect Effects 0.000 claims description 2
- 238000004519 manufacturing process Methods 0.000 abstract description 4
- 230000009286 beneficial effect Effects 0.000 abstract description 2
- 238000007781 pre-processing Methods 0.000 abstract 1
- 238000010586 diagram Methods 0.000 description 12
- 230000005641 tunneling Effects 0.000 description 11
- 238000011161 development Methods 0.000 description 6
- 239000011435 rock Substances 0.000 description 6
- 238000010219 correlation analysis Methods 0.000 description 5
- 208000022971 Tuberculous meningitis Diseases 0.000 description 4
- 208000001223 meningeal tuberculosis Diseases 0.000 description 4
- 239000002689 soil Substances 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 3
- 230000036544 posture Effects 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 238000013500 data storage Methods 0.000 description 2
- 238000006073 displacement reaction Methods 0.000 description 2
- 238000012351 Integrated analysis Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000009412 basement excavation Methods 0.000 description 1
- 239000004927 clay Substances 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 239000002131 composite material Substances 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 238000013523 data management Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 239000000428 dust Substances 0.000 description 1
- 230000002349 favourable effect Effects 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 230000008676 import Effects 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000006855 networking Effects 0.000 description 1
- 230000035515 penetration Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 239000002002 slurry Substances 0.000 description 1
- 239000013589 supplement Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/08—Construction
Landscapes
- Business, Economics & Management (AREA)
- Engineering & Computer Science (AREA)
- Human Resources & Organizations (AREA)
- Economics (AREA)
- Strategic Management (AREA)
- General Physics & Mathematics (AREA)
- Tourism & Hospitality (AREA)
- Entrepreneurship & Innovation (AREA)
- Theoretical Computer Science (AREA)
- Marketing (AREA)
- General Business, Economics & Management (AREA)
- Physics & Mathematics (AREA)
- Game Theory and Decision Science (AREA)
- Quality & Reliability (AREA)
- Operations Research (AREA)
- Development Economics (AREA)
- Educational Administration (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Primary Health Care (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention discloses a shield TBM big data platform system based on a Hadoop cluster architecture and a construction method, comprising the following steps: the data acquisition module is used for remotely acquiring and transmitting data in real time; the big data application module is used for processing data, establishing a data dictionary, preprocessing mass data through technical means such as cleaning and denoising, and then transmitting the mass data to a Hadoop cluster server for storage and analysis; the comprehensive analysis module performs data analysis and data modeling, and provides support for realizing application functions through various data models such as an image model, an operation monitoring model, a geographic environment model, a construction process model and the like; and in the data application, a data analysis result is issued through a web server, so that the intelligent monitoring module, the comprehensive analysis module, the cooperative management module and various big data deep applications of the shield TBM are realized. The method is beneficial to fully playing the basic resource role and the innovative engine role of data, and provides a basis for intelligent construction and manufacturing of the shield TBM.
Description
Technical Field
The invention relates to the technical field of big data of a shield TBM (Tunnel Boring Machine), in particular to a large data platform system of the shield TBM based on a Hadoop cluster architecture and a construction method thereof.
Background
The shield TBM is used as important equipment for underground engineering construction, is high in yield and efficiency, does not need to be removed in a large area, does not block traffic, is low in noise and does not settle on the ground in the using process, and therefore, the shield TBM has a wide application prospect. According to statistics, the number of shield TBMs in China is nearly 1800 at present. Due to the fact that the equipment is complex in structure and severe in construction environment, a plurality of problems and challenges are brought to the construction of the shield TBM. In the past, construction elements of the shield TBM are not associated and become information islands respectively, and the traditional construction management experience cannot completely meet the development requirement of the shield TBM construction method.
The information-based construction is a necessary trend of development of the shield TBM, but due to complexity and dispersity of tunnel construction, the information-based development of the field of the shield TBM is not realized, and some existing platforms only rely on a traditional single-server or multi-server architecture, and multiple sides are focused on monitoring, and data cannot be deeply excavated. With the development of the internet +, artificial intelligence and big data technology, how to realize the deep fusion of the shield TBM and the big data technology and promote the intelligent development of shield TBM manufacturing and construction is a key problem of green development of the industry.
Disclosure of Invention
The invention provides a shield TBM big data platform system and a construction method based on a Hadoop cluster architecture, aiming at overcoming the defects that in the prior art, data cannot be deeply excavated, monitoring is emphasized, each construction element of a shield TBM is lack of effective correlation and intelligentization, covering data acquisition, real-time monitoring, cooperative management and data deep excavation application, and providing data reference for scientific research, manufacturing and construction.
In order to achieve the purpose, the invention adopts the following technical scheme:
a shield TBM big data platform system based on a Hadoop cluster architecture is characterized by comprising a data acquisition module, a Hadoop cluster server and a shield TBM big data platform; the shield TBM big data platform comprises an intelligent monitoring module, a comprehensive analysis module, a cooperative management module and a big data application module; the Hadoop cluster server comprises a Hadoop server, a Kafka server and a database server; the data acquisition module comprises a special data acquisition device and data acquisition software matched with the special data acquisition device; the intelligent monitoring module mainly comprises map cruising, parameter real-time monitoring and video monitoring; the comprehensive analysis module mainly comprises an engineering section drawing module, a parameter analysis module, a segment attitude analysis module and a ground settlement analysis module; the cooperative management module mainly comprises a basic information module, a decision management module, a risk management module, an equipment management module, a construction management module and a measurement management module; the big data application module mainly comprises a data dictionary, a geological dictionary, an association analysis and a mobile platform.
Preferably, the special data acquisition device is configured on the shield tunneling machine or the upper computer. The special data acquisition device has the characteristic of small volume, meets the use environment of high temperature, high humidity, much dust, large noise and strong electromagnetic interference on site, and has the site data storage capacity of more than 3 months; the system has the capability of automatic restart after power failure and incoming call, and has the capability of multi-level networking.
Preferably, the Hadoop cluster server further comprises a data acquisition server and a web server; the Hadoop cluster server also includes Flume, HDFS, HBase, Hive, Nosql, MapReduce, Redis, Zookeeper, Spark, and Sqoop plug-ins.
The intelligent monitoring module comprises the following contents:
the map cruise has the functions of displaying an industry map, an enterprise map and a project map, can display enterprise information, equipment information, engineering information and construction information in real time, display engineering risk information in real time, download a scheme and early warn a nearest risk source.
The real-time monitoring of the parameters can realize the real-time monitoring of the construction data of shield TBMs of different manufacturers and different types, such as earth pressure shields, slurry shields and TBMs; the monitoring content mainly comprises state information, a key system and a guide system, and the communication state and the disconnection information of the equipment can be displayed in real time.
And video monitoring can be carried out, and the on-site video content can be remotely monitored in real time.
The comprehensive analysis module comprises the following contents:
the engineering section drawing module comprises an engineering plane and an engineering longitudinal section, wherein the engineering plane comprises information such as settlement monitoring, risk sources and ring numbers, the engineering longitudinal section comprises information such as settlement monitoring, risk sources, ring numbers and geological information, and real-time display and scaling of progress information and current geological information are achieved.
And parameter analysis, namely, inquiring and analyzing the tunneling parameters of the equipment according to a ring number mode and a time mode, and exporting an Excel file.
The segment attitude analysis module is used for analyzing the segment attitude of the shield tunneling machine, namely the deviation of the segment relative to the center line of the tunnel and the position of the shield tunneling machine in shield tunneling; the segment posture is inquired in a ring number mode and a time mode, the segment posture display mode has a block diagram mode and a line diagram mode, the display content comprises horizontal displacement data analysis and vertical displacement analysis, and data are exported in real time.
And the ground settlement analysis module is used for realizing the analysis of single settlement monitoring points and multiple settlement monitoring points, the analysis content comprises the settlement rate analysis and the settlement accumulated amount analysis, and the data can be exported.
The cooperation management module comprises the following contents:
the basic information module comprises engineering distribution, a comprehensive standing book and a project board, and can be displayed visually in the forms of a Sangji diagram, a chart and the like.
And decision management, including risk decision, progress decision and equipment decision, for counting risk information, progress information and equipment information and performing early warning auxiliary decision, and can be displayed visually in forms of a Sangji diagram, a chart and the like.
And risk management, which comprises engineering risk grade setting and risk source information inputting and displaying, tunneling parameter early warning setting and displaying, ground settlement early warning and displaying, shield TBM attitude early warning and displaying, parameter scheme early warning setting and displaying, station moving correction early warning and displaying, early warning information pushing management, and the early warning information can be displayed in the form of graphs and the like, and data can be exported.
And equipment management, including equipment account management, equipment record management, equipment real-time fault management, equipment maintenance scheme and tool management, equipment special scheme management and the like, and the equipment management is performed with image display in the form of diagrams and the like, and data can be exported.
Construction management, including construction progress management, process management, quality management, special project management, geological dictionary and entry management and skill talent management; construction report generation and the like, and data can be exported.
And measurement management, including guidance system standing book management, measurement instrument standing book, measurement personnel information management, control measurement management, DTA data management and data import management, wherein data can be exported.
The big data application module comprises the following contents:
and the data dictionary codes massive data of different manufacturers, different types of shield TBMs and different forms of description uniformly through the platform according to the principle of the same property.
And the geological dictionary classifies the shield TBM tunneling geology into soft soil, a composite stratum and a rock stratum by the platform according to geological classification rules and industry conventions, wherein the soft soil is used for subdividing silt, clay, sandy soil and pebbles, the soft soil is specifically subdivided by a standard penetration value, the rock is subdivided into igneous rock, sedimentary rock and metamorphic rock, and the rock is subdivided by compression strength and integrity.
And (4) performing correlation analysis, including key parameter correlation analysis, guide parameter correlation analysis and settlement parameter correlation analysis, performing multi-item correlation analysis on the correlation geology, the equipment type, the equipment diameter and other dimensions to give a tunneling reference suggested value, and displaying the tunneling reference suggested value in the form of a chart and the like in an image mode, wherein data can be exported.
A construction method of a shield TBM big data platform system based on a Hadoop cluster architecture is characterized by comprising the following steps:
(1-1) the data acquisition module carries out remote real-time acquisition and transmission on data to determine a data source and a data type acquired by the shield TBM equipment;
(1-2) the big data application module processes data, establishes a data dictionary, preprocesses mass data through technical means such as cleaning and denoising, and transmits the mass data to a Hadoop cluster server for storage and analysis;
(1-3) the Hadoop cluster server performs data storage and calculation, and performs classified storage on the preprocessed mass data;
(1-4) the comprehensive analysis module performs data analysis and data modeling, and provides support for realizing application functions through various data models such as an image model, an operation monitoring model, a geographic environment model, a construction process model and the like;
and (1-5) data application, namely issuing a data analysis result through a web server, realizing the deep application of an intelligent monitoring module, a comprehensive analysis module, a cooperative management module and various large data of the shield TBM, and performing engineering section graph drawing, segment attitude analysis and ground settlement analysis.
Preferably, the step (1-1) comprises the steps of:
(2-1) the data acquisition module analyzes a PLC variable address of the shield TBM equipment;
(2-2) the data acquisition module is provided with a special data acquisition unit and data acquisition software, and reads real-time running data of the equipment through a PLC variable address;
(2-3) the data acquisition module reads the guiding data in the form of reading database files of the shield machine or the upper computer and the like;
and (2-4) the data acquisition module sends data to the Hadoop cluster data acquisition server in real time to finish data acquisition.
Preferably, the data acquisition software sets an acquisition method through a software interface, and the acquisition method comprises direct acquisition of PLC data, acquisition of a file form and combined acquisition of the PLC data and the file form. The data acquisition frequency reaches the second level, and the system has data encryption transmission capability and automatic starting and automatic acquisition capability after power failure and incoming call.
Preferably, the step (1-2) comprises the steps of:
(3-1) the big data application module establishes a data dictionary, and carries out unified dictionary coding on data of different types, different manufacturers and different descriptions according to the principle of the same property;
(3-2) storing the data from the data acquisition server to the HDFS distributed storage system by using a Flume plug-in the Hadoop cluster server;
(3-3) carrying out data cleaning and denoising by using MapReduce in the Hadoop cluster server;
and (3-4) uploading the processed data to a shield TBM big data platform.
Preferably, the step (1-3) comprises the steps of:
(4-1) the Spark program acquires data from the queue of kafka;
(4-2) reading the configuration information of the system from a database of the redis by the Hadoop cluster server, and detecting and supplementing data;
(4-3) carrying out deep analysis and mining on heterogeneous mass data by using a Spark computing framework cluster and Zookeeper distributed cooperation service;
and (4-4) importing the analyzed data into the database server by using the Sqoop plug-in.
Therefore, the invention has the following beneficial effects:
the technical scheme of the invention innovatively applies Hadoop cluster architecture in the industry, establishes a high-performance distributed data offline and online analysis and calculation method, realizes open source elastic capacity expansion and multi-copy disaster tolerance, establishes a multi-channel and multi-protocol data acquisition system, establishes a shield TBM big data platform integrating intelligent monitoring, comprehensive analysis, cooperative management and big data application, breaks through the traditional single-server system mode, has the remarkable advantages of high data acquisition frequency, cluster monitoring, high data analysis speed and the like, forms a data dictionary and a geological dictionary, can draw a project section diagram in real time, analyze segment postures and subside, can directly acquire real-time data in the engineering implementation process, performs management monitoring and construction adjustment according to the data, directly reduces the risk occurrence probability, and can promote data resource integration and open sharing in the shield and tunneling technology industries, the method is favorable for fully playing the basic resource role and the innovative engine role of data, and provides a basis for intelligent construction and manufacturing of the shield TBM.
Drawings
FIG. 1 is a schematic diagram of a system of the present invention;
FIG. 2 is a schematic diagram of a shield TBM big data platform according to the present invention;
FIG. 3 is a schematic diagram of an intelligent monitoring module in an embodiment of the invention;
FIG. 4 is a schematic diagram of an integrated analysis module in an embodiment of the invention;
FIG. 5 is a schematic diagram of a collaboration management module in an embodiment of the invention;
FIG. 6 is a diagram of a big data application module in an embodiment of the invention.
In the figure, a data acquisition module 1, a Hadoop cluster server 2, a shield TBM big data platform 3, an intelligent monitoring module 4, a comprehensive analysis module 5, a cooperative management module 6, a big data application module 7, a Hadoop server 8, a Kafka server 9, a database server 10, a special data acquisition device 11, a map cruise module 12, a real-time parameter monitoring module 13, a video monitoring module 14, a project profile drawing module 15, a parameter analysis module 16, a segment attitude analysis module 17, a ground settlement analysis module 18, a basic information module 19, a decision management module 20, a risk management module 21, an equipment management 22, a construction management 23, a measurement management 24, a data dictionary 25, a geological dictionary 26, an association analysis 27 and a mobile platform 28 are arranged.
Detailed Description
The invention is further described in the following detailed description with reference to the drawings in which:
the embodiment shown in fig. 1 is a shield TBM big data platform system based on a Hadoop cluster architecture, and includes a data acquisition module 1, a Hadoop cluster server 2, and a shield TBM big data platform 3; as shown in fig. 2, the shield TBM big data platform includes an intelligent monitoring module 4, a comprehensive analysis module 5, a cooperative management module 6, and a big data application module 7; the Hadoop cluster server comprises a Hadoop server 8, a Kafka server 9 and a database server 10; the Hadoop cluster server also comprises a data acquisition server and a web server; the Hadoop cluster server also comprises Flume, HDFS, HBase, Hive, Nosql, MapReduce, Redis, Zookeeper, Spark and Sqoop plug-ins.
The data acquisition module comprises a special data acquisition device 11 and data acquisition software matched with the special data acquisition device, and the special data acquisition device is configured on the shield tunneling machine or the upper computer. The intelligent monitoring module mainly comprises a map cruise module 12, a parameter real-time monitoring module 13 and a video monitoring module 14; the comprehensive analysis module mainly comprises an engineering section drawing module 15, a parameter analysis module 16, a segment attitude analysis module 17 and a ground settlement analysis module 18; the cooperative management module mainly comprises a basic information module 19, a decision management module 20, a risk management module 21, an equipment management module 22, a construction management module 23 and a measurement management module 24; the big data application module mainly includes a data dictionary 25, a geological dictionary 26, an association analysis 27 and a mobile platform 28.
A construction method of a shield TBM big data platform system based on a Hadoop cluster architecture comprises the following steps:
s1, the data acquisition module carries out remote real-time acquisition and transmission on the data to determine the data source and the data type acquired by the shield TBM equipment;
step S11, the data acquisition module analyzes the PLC variable address of the shield TBM equipment;
step S12, the data acquisition module is provided with a special data acquisition unit and data acquisition software and reads real-time running data of the equipment through a PLC variable address; the data acquisition software sets an acquisition method through a software interface, and the acquisition method comprises direct acquisition of PLC data, acquisition of a file form and combination acquisition of the PLC data and the file form. The data acquisition frequency reaches the second level, and the system has data encryption transmission capability and automatic starting and automatic acquisition capabilities after power failure and incoming call;
step S13, the data acquisition module reads the guiding data by reading the database file of the shield machine or the upper computer;
step S14, the data acquisition module sends data to a Hadoop cluster data acquisition server in real time to finish data acquisition;
step S2, the big data application module processes data, establishes a data dictionary, preprocesses mass data through technical means such as cleaning and denoising, and transmits the mass data to a Hadoop cluster server for storage and analysis;
step S21, the big data application module establishes a data dictionary and carries out unified dictionary coding on data of different types, different manufacturers and different descriptions according to the principle of the same property;
step S22, storing the data from the data acquisition server to the HDFS distributed storage system by using a Flume plug-in the Hadoop cluster server;
step S23, cleaning and denoising data by using MapReduce in the Hadoop cluster server;
step S24, uploading the processed data to a shield TBM big data platform;
step S3, the Hadoop cluster server stores and calculates data, and stores the preprocessed mass data in a classified manner;
step S31, the Spark program acquires data from the queue of kafka;
step S32, the Hadoop cluster server reads the configuration information of the system from the database of the redis, and detects and supplements the data;
step S33, carrying out deep analysis and mining on heterogeneous mass data by using a Spark computing framework cluster and Zookeeper distributed cooperation service;
step S34, importing the analyzed data into a database server by using the Sqoop plug-in;
step S4, the comprehensive analysis module carries out data analysis and data modeling, and provides support for realizing application function through various data models such as an image model, an operation monitoring model, a geographic environment model, a construction process model and the like;
and S5, data application, namely issuing a data analysis result through a web server, realizing the intelligent monitoring module, the comprehensive analysis module, the cooperative management module and various large data depth applications of the shield TBM, and performing engineering section graph drawing, segment attitude analysis and ground settlement analysis.
It should be understood that this example is for illustrative purposes only and is not intended to limit the scope of the present invention. Further, it should be understood that various changes or modifications of the present invention may be made by those skilled in the art after reading the teaching of the present invention, and such equivalents may fall within the scope of the present invention as defined in the appended claims.
Claims (8)
1. A shield TBM big data platform system based on a Hadoop cluster architecture is characterized by comprising a data acquisition module (1), a Hadoop cluster server (2) and a shield TBM big data platform (3); the shield TBM big data platform comprises an intelligent monitoring module (4), a comprehensive analysis module (5), a cooperative management module (6) and a big data application module (7); the Hadoop cluster server comprises a Hadoop server (8), a Kafka server (9) and a database server (10); the data acquisition module comprises a special data acquisition device (11) and data acquisition software adapted to the special data acquisition device; the intelligent monitoring module mainly comprises a map cruise module (12), a parameter real-time monitoring module (13) and a video monitoring module (14); the comprehensive analysis module mainly comprises an engineering section drawing module (15), a parameter analysis module (16), a segment attitude analysis module (17) and a ground settlement analysis module (18); the cooperative management module mainly comprises a basic information module (19), decision management (20), risk management (21), equipment management (22), construction management (23) and measurement management (24); the big data application module mainly comprises a data dictionary (25), a geological dictionary (26), an association analysis (27) and a mobile platform (28).
2. The Hadoop cluster architecture-based shield TBM big data platform system according to claim 1, wherein the special data acquisition device is configured on a shield machine or an upper computer.
3. The Hadoop cluster architecture-based shield TBM big data platform system according to claim 1, wherein the Hadoop cluster server further comprises a data acquisition server and a web server; the Hadoop cluster server also comprises Flume, HDFS, HBase, Hive, Nosql, MapReduce, Redis, Zookeeper, Spark and Sqoop plug-ins.
4. A construction method of a shield TBM big data platform system based on the Hadoop cluster architecture of claims 1, 2 and 3 is characterized by comprising the following steps:
(4-1) the data acquisition module carries out remote real-time acquisition and transmission on data to determine the data source and the data type acquired by the shield TBM equipment;
(4-2) the big data application module processes data, establishes a data dictionary, preprocesses mass data through technical means such as cleaning and denoising, and transmits the mass data to a Hadoop cluster server for storage and analysis;
(4-3) the Hadoop cluster server stores and calculates data, and stores the preprocessed mass data in a classified manner;
(4-4) the comprehensive analysis module performs data analysis and data modeling, and provides support for realizing application functions through various data models such as an image model, an operation monitoring model, a geographic environment model, a construction process model and the like;
and (4-5) data application, namely issuing a data analysis result through a web server, realizing the deep application of an intelligent monitoring module, a comprehensive analysis module, a cooperative management module and various large data of the shield TBM, and performing engineering section graph drawing, segment attitude analysis and ground settlement analysis.
5. The construction method of the shield TBM big data platform system based on the Hadoop cluster architecture as claimed in claim 4, wherein the step (4-1) comprises the following steps:
(5-1) the data acquisition module analyzes a PLC variable address of the shield TBM equipment;
(5-2) the data acquisition module is provided with a special data acquisition unit and data acquisition software, and reads real-time running data of the equipment through a PLC variable address;
(5-3) the data acquisition module reads the guiding data in the form of reading database files of the shield machine or the upper computer and the like;
and (5-4) the data acquisition module sends data to the Hadoop cluster data acquisition server in real time to finish data acquisition.
6. The construction method of the shield TBM big data platform system based on the Hadoop cluster architecture as claimed in claim 5, wherein the data acquisition software sets the acquisition method through a software interface, and the acquisition method comprises direct acquisition of PLC data, file form acquisition and combination acquisition of PLC data and file form.
7. The construction method of the shield TBM big data platform system based on the Hadoop cluster architecture as claimed in claim 4, wherein the step (4-2) comprises the following steps:
(7-1) the big data application module establishes a data dictionary, and carries out unified dictionary coding on data of different types, different manufacturers and different descriptions according to the principle of the same property;
(7-2) storing the data from the data acquisition server to the HDFS distributed storage system by using a Flume plug-in the Hadoop cluster server;
(7-3) carrying out data cleaning and denoising by using MapReduce in the Hadoop cluster server;
and (7-4) uploading the processed data to a shield TBM big data platform.
8. The construction method of the shield TBM big data platform system based on the Hadoop cluster architecture as claimed in claim 4, wherein the step (4-3) comprises the following steps:
(8-1) the Spark program acquires data from the queue of kafka;
(8-2) reading the configuration information of the system from a database of the redis by the Hadoop cluster server, and detecting and supplementing data;
(8-3) carrying out deep analysis and mining on heterogeneous mass data by using a Spark computing framework cluster and Zookeeper distributed cooperation service;
and (8-4) importing the analyzed data into the database server by using the Sqoop plug-in.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911399090.5A CN111126873A (en) | 2019-12-30 | 2019-12-30 | Shield TBM big data platform system based on Hadoop cluster architecture and construction method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911399090.5A CN111126873A (en) | 2019-12-30 | 2019-12-30 | Shield TBM big data platform system based on Hadoop cluster architecture and construction method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111126873A true CN111126873A (en) | 2020-05-08 |
Family
ID=70505525
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911399090.5A Pending CN111126873A (en) | 2019-12-30 | 2019-12-30 | Shield TBM big data platform system based on Hadoop cluster architecture and construction method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111126873A (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111651499A (en) * | 2020-05-28 | 2020-09-11 | 上海卓越睿新数码科技有限公司 | Learning behavior score calculation method based on big data technology and mathematical algorithm |
| CN111967822A (en) * | 2020-08-13 | 2020-11-20 | 中铁工程装备集团技术服务有限公司 | WeChat public number-based shield machine service platform, service method and framework |
| CN112463755A (en) * | 2020-12-11 | 2021-03-09 | 同济大学 | Heterogeneous Internet of things big data storage and reading system and method based on HDFS |
| CN112835889A (en) * | 2021-01-12 | 2021-05-25 | 杨飞 | Heterogeneous system data integration method, system and equipment |
| CN113090274A (en) * | 2021-06-08 | 2021-07-09 | 北京建工土木工程有限公司 | Automatic cruise shield advance geological prediction system based on big data and working method |
| CN113379196A (en) * | 2021-05-17 | 2021-09-10 | 国网浙江省电力有限公司宁波供电公司 | Transformer equipment management evaluation system based on digital twin technology |
| CN114152191A (en) * | 2021-12-01 | 2022-03-08 | 上海上实龙创智能科技股份有限公司 | Image measuring device based on MapReduce |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107092990A (en) * | 2017-05-03 | 2017-08-25 | 西安电子科技大学 | The shield construction ground settlement forecast system and method analyzed based on big data |
| US20190278573A1 (en) * | 2018-03-06 | 2019-09-12 | Huazhong University Of Science And Technology | Method of memory estimation and configuration optimization for distributed data processing system |
| CN110359919A (en) * | 2019-07-26 | 2019-10-22 | 中铁隧道局集团有限公司 | A kind of shield machine construction risk prevention system method and system |
-
2019
- 2019-12-30 CN CN201911399090.5A patent/CN111126873A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107092990A (en) * | 2017-05-03 | 2017-08-25 | 西安电子科技大学 | The shield construction ground settlement forecast system and method analyzed based on big data |
| US20190278573A1 (en) * | 2018-03-06 | 2019-09-12 | Huazhong University Of Science And Technology | Method of memory estimation and configuration optimization for distributed data processing system |
| CN110359919A (en) * | 2019-07-26 | 2019-10-22 | 中铁隧道局集团有限公司 | A kind of shield machine construction risk prevention system method and system |
Non-Patent Citations (2)
| Title |
|---|
| 李凤远等: "建设盾构TBM工程大数据云平台 创新引领行业技术发展", 《工程机械与维修》 * |
| 赵炯等: "基于Redis盾构远程监控系统设计", 《隧道建设(中英文)》 * |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111651499A (en) * | 2020-05-28 | 2020-09-11 | 上海卓越睿新数码科技有限公司 | Learning behavior score calculation method based on big data technology and mathematical algorithm |
| CN111967822A (en) * | 2020-08-13 | 2020-11-20 | 中铁工程装备集团技术服务有限公司 | WeChat public number-based shield machine service platform, service method and framework |
| CN112463755A (en) * | 2020-12-11 | 2021-03-09 | 同济大学 | Heterogeneous Internet of things big data storage and reading system and method based on HDFS |
| CN112835889A (en) * | 2021-01-12 | 2021-05-25 | 杨飞 | Heterogeneous system data integration method, system and equipment |
| CN112835889B (en) * | 2021-01-12 | 2023-12-29 | 杨飞 | Heterogeneous system data integration method, system and equipment |
| CN113379196A (en) * | 2021-05-17 | 2021-09-10 | 国网浙江省电力有限公司宁波供电公司 | Transformer equipment management evaluation system based on digital twin technology |
| CN113090274A (en) * | 2021-06-08 | 2021-07-09 | 北京建工土木工程有限公司 | Automatic cruise shield advance geological prediction system based on big data and working method |
| CN114152191A (en) * | 2021-12-01 | 2022-03-08 | 上海上实龙创智能科技股份有限公司 | Image measuring device based on MapReduce |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111126873A (en) | Shield TBM big data platform system based on Hadoop cluster architecture and construction method | |
| CN103233741B (en) | Method for quantitatively evaluating surrounding rock stability of ultra-large underground cavern group in construction period | |
| CN111062361B (en) | River and lake shoreline sand production monitoring and analyzing method and device | |
| CN113065276A (en) | Intelligent construction method based on digital twins | |
| CN105627981A (en) | Mining subsidence deformation prediction and forecast system | |
| CN106679625A (en) | High-precision deformation monitoring method of wide-area electric iron tower based on Beidou system | |
| CN203849605U (en) | Smart pipe network | |
| CN111444568A (en) | Engineering monitoring BIM cooperative platform based on Internet of things | |
| CN103595813A (en) | Intelligent pipe network application system and obtaining method thereof | |
| CN104778742A (en) | Method for establishing 3D digital maintenance platform of aircraft based on 3D laser scanning technology | |
| CN109274737B (en) | C/S technical architecture system for oil field exploration, development and production | |
| CN111089560A (en) | Subway structure health monitoring and early warning system based on three-dimensional live-action model | |
| CN110912976A (en) | Railway construction equipment remote monitoring system | |
| CN104361586A (en) | Electric pole and tower maintenance and pre-warning system based on panoramas | |
| CN107133236A (en) | The method and apparatus of geographical position track optimizing | |
| CN111813877B (en) | Track extraction method and device | |
| CN107885156A (en) | Intelligent front end multimedia information acquisition system for lean monitoring power station equipment | |
| Bo et al. | Research on the data validity of a coal mine solid backfill working face sensing system based on an improved transformer | |
| CN113899405A (en) | Integrated online slope intelligent monitoring and early warning system and early warning method | |
| CN112288591A (en) | Three-dimensional management and control platform system based on laser point cloud map | |
| CN116863652A (en) | Geological disaster early warning method based on Internet adding technology | |
| CN105117480A (en) | Comprehensive display platform of coal field and transmission line based on goaf | |
| CN109147962A (en) | A kind of geologic prospect remote medical consultation with specialists and supervisory systems based on Internet of Things | |
| CN202023549U (en) | Remote real-time monitoring system for well drilling construction | |
| CN112882437B (en) | Intelligent tunneling system and control method based on 5G + big data tunnel boring machine |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |