CN113590626A - Multi-source heterogeneous data acquisition system and method for tunneling equipment - Google Patents
Multi-source heterogeneous data acquisition system and method for tunneling equipment Download PDFInfo
- Publication number
- CN113590626A CN113590626A CN202110885040.9A CN202110885040A CN113590626A CN 113590626 A CN113590626 A CN 113590626A CN 202110885040 A CN202110885040 A CN 202110885040A CN 113590626 A CN113590626 A CN 113590626A
- Authority
- CN
- China
- Prior art keywords
- data
- source
- main control
- control board
- database
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2282—Tablespace storage structures; Management thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/215—Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24552—Database cache management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Computational Linguistics (AREA)
- Quality & Reliability (AREA)
- Arrangements For Transmission Of Measured Signals (AREA)
Abstract
The invention discloses a multi-source heterogeneous data acquisition system and a method for tunneling equipment, wherein the system comprises the following steps: the system comprises a plurality of data sources, a plurality of data processing units and a plurality of data processing units, wherein the data sources are used for providing multi-source heterogeneous data of tunneling equipment; the main control board is used for monitoring a plurality of data sources in real time, acquiring data from the data sources according to the data source configuration parameters when the data sources are accessed, and sending the acquired data to the remote command center; the remote command center is used for receiving the data sent by the main control board, and storing the real-time data into different data tables in the first database according to different data types and data lengths when the received data are real-time data; when the received data is historical data, storing the historical data into different data tables in a second database according to different data types and different data lengths; the first database and the second database are different databases, multi-source data coordination collection can be carried out on tunneling equipment, data storage space is saved, data management capacity is improved, and follow-up data analysis is facilitated.
Description
Technical Field
The invention relates to the technical field of intelligent construction of tunnels, in particular to a multi-source heterogeneous data acquisition system and method for tunneling equipment.
Background
This section is intended to provide a background or context to the embodiments of the invention that are recited in the claims. The description herein is not admitted to be prior art by inclusion in this section.
In the current process of carrying out data acquisition on tunneling equipment, data acquisition equipment can only carry out data acquisition on one corresponding sensing equipment, so that the sensing equipment of the tunneling equipment is carried with too many data acquisition equipment, the data acquisition modes are uneven, and the difficulty is increased for subsequent data storage, data management and data analysis.
Therefore, a data acquisition scheme that can overcome the above technical problems is needed.
Disclosure of Invention
The embodiment of the invention provides a multi-source heterogeneous data acquisition system of tunneling equipment, which is used for realizing multi-source data coordinated acquisition of the tunneling equipment, saving data storage space, improving data management capacity and being beneficial to subsequent data analysis, and comprises the following components:
the system comprises a plurality of data sources, a plurality of data processing units and a plurality of data processing units, wherein the data sources are used for providing multi-source heterogeneous data of tunneling equipment;
the main control board is used for monitoring a plurality of data sources in real time, acquiring data from the data sources according to the data source configuration parameters when the data sources are accessed, and sending the acquired data to the remote command center;
the remote command center is used for receiving the data sent by the main control board, and storing the real-time data into different data tables in the first database according to different data types and data lengths when the received data are real-time data; when the received data is historical data, storing the historical data into different data tables in a second database according to different data types and different data lengths; wherein the first database and the second database are different databases.
The embodiment of the invention also provides a multi-source heterogeneous data acquisition method of the tunneling equipment, which is used for realizing multi-source data coordinated acquisition of the tunneling equipment, saving data storage space, improving data management capacity and being beneficial to subsequent data analysis, the method is applied to a main control board in a multi-source heterogeneous data acquisition system of the tunneling equipment, the multi-source heterogeneous data acquisition system of the tunneling equipment further comprises a remote command center and a plurality of data sources, and the method comprises the following steps:
monitoring a plurality of data sources in real time; the multiple data sources are used for providing multi-source heterogeneous data of the tunneling equipment;
when a data source is accessed, data are obtained from the data source according to the data source configuration parameters, and the obtained data are sent to a remote command center.
The embodiment of the invention also provides a multi-source heterogeneous data acquisition method for the tunneling equipment, which is used for realizing multi-source data coordinated acquisition on the tunneling equipment, saving data storage space, improving data management capacity and being beneficial to subsequent data analysis; the method comprises the following steps:
receiving data sent by a main control board; the main control board is used for monitoring a plurality of data sources in real time, and acquiring data from the data sources according to the data source configuration parameters when the data sources are accessed; the multiple data sources are used for providing multi-source heterogeneous data of the tunneling equipment;
when the received data is real-time data, storing the real-time data into different data tables in a first database according to different data types and data lengths;
when the received data is historical data, storing the historical data into different data tables in a second database according to different data types and different data lengths; wherein the first database and the second database are different databases.
The embodiment of the invention also provides a main control board in the multi-source heterogeneous data acquisition system of the tunneling equipment, which is used for realizing multi-source data coordinated acquisition of the tunneling equipment, saving data storage space, improving data management capability and being beneficial to subsequent data analysis; the main control board includes:
the monitoring module is used for monitoring a plurality of data sources in real time; the multiple data sources are used for providing multi-source heterogeneous data of the tunneling equipment;
and the data transmission template is used for acquiring data from the data source according to the data source configuration parameters and sending the acquired data to the remote command center when the data source is accessed.
The embodiment of the invention also provides a remote command center in the multi-source heterogeneous data acquisition system of the tunneling equipment, which is used for realizing multi-source data coordinated acquisition of the tunneling equipment, saving data storage space, improving data management capability and being beneficial to subsequent data analysis, and the remote command center comprises:
the data receiving module is used for receiving data sent by the main control board; the main control board is used for monitoring a plurality of data sources in real time, and acquiring data from the data sources according to the data source configuration parameters when the data sources are accessed; the multiple data sources are used for providing multi-source heterogeneous data of the tunneling equipment;
the first storage module is used for storing the real-time data into different data tables in a first database according to different data types and data lengths when the received data are the real-time data;
the second storage module is used for storing the historical data into different data tables in a second database according to different data types and data lengths when the received data are the historical data; wherein the first database and the second database are different databases.
The embodiment of the invention also provides computer equipment which comprises a storage, a processor and a computer program which is stored on the storage and can run on the processor, wherein the processor executes the computer program to realize the multi-source heterogeneous data acquisition method of the tunneling equipment.
The embodiment of the invention also provides a computer readable storage medium, which stores a computer program for executing the multi-source heterogeneous data acquisition method of the tunneling equipment.
The multisource heterogeneous data acquisition system of the tunneling equipment in the embodiment of the invention comprises the following components: the system comprises a plurality of data sources, a plurality of data processing units and a plurality of data processing units, wherein the data sources are used for providing multi-source heterogeneous data of tunneling equipment; the main control board is used for monitoring a plurality of data sources in real time, acquiring data from the data sources according to the data source configuration parameters when the data sources are accessed, and sending the acquired data to the remote command center; the remote command center is used for receiving the data sent by the main control board, and storing the real-time data into different data tables in the first database according to different data types and data lengths when the received data are real-time data; when the received data is historical data, storing the historical data into different data tables in a second database according to different data types and different data lengths; the first database and the second database are different databases, so that multi-source data coordination collection of the tunneling equipment can be realized, the data storage space is saved, the data management capability is improved, and the subsequent data analysis is facilitated.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts. In the drawings:
FIG. 1 is a schematic diagram of a multi-source heterogeneous data acquisition system of tunneling equipment in an embodiment of the invention;
fig. 2 is a schematic diagram of an embodiment of a multi-source heterogeneous data acquisition system of tunneling equipment in an embodiment of the invention;
fig. 3 is a schematic diagram of a specific example of a main control board in a multi-source heterogeneous data acquisition system of tunnel boring equipment in the embodiment of the present invention;
fig. 4 is a schematic flow chart of a multi-source heterogeneous data acquisition method of tunnel boring equipment applied to a main control board in the embodiment of the present invention;
fig. 5 is a schematic flow chart of a multi-source heterogeneous data acquisition method of tunneling equipment applied to a remote command center in the embodiment of the present invention;
fig. 6 is a schematic diagram of a main control board in a multi-source heterogeneous data acquisition system of tunnel boring equipment in an embodiment of the present invention;
fig. 7 is a schematic diagram of a remote command center in a multi-source heterogeneous data acquisition system of tunneling equipment in an embodiment of the present invention;
fig. 8 is a schematic structural diagram of a computer device according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the embodiments of the present invention are further described in detail below with reference to the accompanying drawings. The exemplary embodiments and descriptions of the present invention are provided to explain the present invention, but not to limit the present invention.
Fig. 1 is a schematic view of a multi-source heterogeneous data acquisition system of tunneling equipment in an embodiment of the present invention, and as shown in fig. 1, the multi-source heterogeneous data acquisition system of tunneling equipment in the embodiment of the present invention may include:
the system comprises a plurality of data sources 101, a plurality of data processing units and a plurality of data processing units, wherein the data sources 101 are used for providing multi-source heterogeneous data of tunneling equipment;
the main control board 102 is used for monitoring the plurality of data sources 101 in real time, acquiring data from the data sources according to the data source configuration parameters when the data sources are accessed, and sending the acquired data to the remote command center;
the remote command center 103 is used for receiving the data sent by the main control board 102, and storing the real-time data into different data tables in the first database according to different data types and data lengths when the received data is the real-time data; when the received data is historical data, storing the historical data into different data tables in a second database according to different data types and different data lengths; wherein the first database and the second database are different databases.
As can be seen from the multi-source heterogeneous data acquisition system of the tunneling equipment shown in fig. 1, in the embodiment of the present invention, a main control board in the multi-source heterogeneous data acquisition system of the tunneling equipment monitors a plurality of data sources providing multi-source heterogeneous data of the tunneling equipment in real time, acquires data from the data sources according to data source configuration parameters, and transmits the acquired data to a remote command center, and the remote command center stores the real-time data and historical data of the tunneling equipment according to the difference between the data type and the data length, so that the multi-source data coordinated acquisition of the tunneling equipment can be realized, the data storage space is saved, the data management capability is improved, and the subsequent data analysis is facilitated.
In one embodiment, the plurality of data sources may be one or more of the following types of data sources: programmable logic controller PLC, hobbing cutter detection module, vibration monitoring module, analog quantity data module.
In one embodiment, the main control board may be further configured to: and carrying out the protocol conversion of the middleware on the accessed data source.
In specific implementation, a main control board in the multi-source heterogeneous data acquisition system of the tunneling equipment is provided with a plurality of communication interfaces, and can be in butt joint with a plurality of data sources through a communication protocol adaptive to the communication interfaces, so that the data sources are obtained; when the communication interface of the main control board does not support the communication protocol adopted by the data source, the main control board can perform intermediate plug-in protocol conversion on the accessed data source, namely, the main control board can perform plug-in type extension on the data source and convert the communication protocol adopted by the data source into the communication protocol adaptive to the communication interface of the main control board. For example, the communication protocols that the main control board in the multi-source heterogeneous data acquisition system of the tunneling equipment can support include: the communication protocol adopted by the accessed data source is a bus protocol CAN, and the main control board CAN convert the intermediate plug-in protocol of the accessed data source, convert the bus protocol CAN into a transmission control protocol TCP and then acquire the data source through the transmission control protocol TCP.
In one embodiment, the main control board is specifically configured to: multithreading real-time monitoring is carried out on a plurality of data sources; and when the condition that the power-on of the data source is monitored and the communication is smooth is monitored, accessing the data source.
In one embodiment, the main control board may be specifically configured to: when a data source is accessed, selecting a communication rate and a data acquisition frequency adaptive to the data source according to the configuration parameters of the data source, and acquiring data from the data source; and selecting a data pushing frequency adaptive to the data source according to the data source configuration parameters, and sending the acquired data to the remote command center.
In one embodiment, the main control board is further configured to: performing handshake detection with a remote command center to determine a network state; if the network state is normal, the acquired data are sent to a remote command center; if the network state is abnormal, the acquired data is temporarily stored in the local database, the handshake detection is carried out again with the remote command center until the network state is normal, and the acquired data is sent to the remote command center.
When the main control board monitors that a data source requests to access, whether a communication protocol of the data source is adaptive to a communication interface of the main control board or not can be judged, and a middle plug-in protocol conversion is carried out on the data source corresponding to the communication protocol which is not supported by the communication interface, then the main control board can read configuration parameters of the data source, select a communication rate and a data acquisition frequency which are adaptive to the data source, acquire data from the data source, and also select a data push frequency which is adaptive to the data source according to the configuration parameters of the data source for sending the acquired data to a remote command center;
before the main control board sends the acquired data to the remote command center, in order to ensure that the network state can carry out remote data transmission, the main control board can carry out handshake detection with the remote command center, and the main control board carries out interaction detection for several times with the remote command center until the normal network communication can be ensured, and then sends the data acquired from the data source to the remote command center, so that the data can be prevented from being lost due to network abnormality in the remote transmission process.
In specific implementation, after the remote command center receives the data sent by the main control board, whether the received data is real-time data or historical data can be judged firstly, and when the data received by the remote command center is real-time data, the real-time data can be stored in different data tables in the first database according to different data types and data lengths; when the data received by the remote command center is historical data, the historical data can be stored in different data tables in a second database according to different data types and different data lengths; wherein the first database and the second database are different databases. Therefore, the remote command center can distinguish real-time data from historical data when storing the data, the integrity of the data is guaranteed, the data is structurally stored in a database, and the phenomenon that too many vacant positions exist in a data table and storage space is wasted is avoided.
In one embodiment, the first database may be a Redis database; and/or, the second database may be a MongoDB database.
In one embodiment, the remote command center may be further operable to: data source configuration parameters stored by a remote maintenance main control board; and after the configuration parameters of the data source are modified, triggering the multi-source heterogeneous data acquisition system of the tunneling equipment to restart.
In one embodiment, the tunneling equipment multi-source heterogeneous data acquisition system may further include: the field scheduling center is used for maintaining the data source configuration parameters stored by the main control board on site; and after the configuration parameters of the data source are modified, triggering the multi-source heterogeneous data acquisition system of the tunneling equipment to restart.
During specific implementation, the remote command center can receive data sent by the main control board and perform classified storage, and can remotely maintain data source configuration parameters stored by the main control board, and the remote command center can remotely modify the data source configuration parameters through the main control board according to actual conditions in a data receiving process, so that the communication rate and the data acquisition frequency of the data acquired by the main control board from the data source can be adjusted, and the acquired data is sent to the data push frequency of the remote command center, namely the remote command center can control the process of acquiring heterogeneous data of tunneling equipment.
Due to the distance, the remote command center has limitation on the control of the multi-source heterogeneous data acquisition process of the tunneling equipment, cannot control the data acquisition process in real time, and timely modifies the data source configuration parameters. Therefore, a field scheduling center can be set, the data source configuration parameters stored by the main control board can be maintained in a close range on the field, the data source configuration parameters can be modified in time when an emergency situation is faced, and the reliability of the multisource heterogeneous data acquisition process of the tunneling equipment is improved.
Fig. 2 is a schematic diagram of a specific example of a multi-source heterogeneous data acquisition system of tunneling equipment in an embodiment of the present invention. As shown in fig. 2, a specific example of the multi-source heterogeneous data acquisition system of the tunneling equipment in the embodiment of the present invention may include:
the data source 201 specifically includes: siemens series PLC (Programmable Logic Controller) 2011, hob detection module 2012, vibration monitoring module 2013, analog data module 2014, main control board 202, remote command center 203, field scheduling center 204, database 205 and Web visual interface 206. As shown in fig. 2, the PLC2011 provides the tunneling equipment multi-source heterogeneous data to the main control board 202 through the siemens ethernet protocol S7, the hob detection module 2012 provides the tunneling equipment multi-source heterogeneous data to the main control board 202 through the modbus communication protocol, the vibration monitoring module 2013 provides the tunneling equipment multi-source heterogeneous data to the main control board 202 through the transmission control protocol TCP, the analog quantity data module 2014 converts the analog quantity data into digital quantity data through the AD analog quantity acquisition module, and provides the tunneling equipment multi-source heterogeneous data to the main control board 202; the main control board 202 and the remote command center 203 carry out handshake detection, and after the network state is determined to be normal, the acquired data is sent to the remote command center 203 through a message transmission protocol MQTT; when the received data is real-time data, the remote command center 203 stores the real-time data into different data tables in a Redis database according to the difference of the data type and the data length, and when the received data is historical data, the remote command center stores the historical data into different data tables in a MongoDB database according to the difference of the data type and the data length; the remote command center 203 remotely maintains the data source configuration parameters stored by the main control board 202; the field scheduling center 204 maintains the data source configuration parameters stored by the main control board 202 on the field through the Web visualization interface 206.
Fig. 3 is a schematic diagram of a specific example of a main control board in a multi-source heterogeneous data acquisition system of tunneling equipment in an embodiment of the present invention. As shown in fig. 3, the main control board in the multi-source heterogeneous data acquisition system of the tunneling equipment in the embodiment of the present invention may include: the system comprises a power management module 301, a 485 communication module 302, a wireless communication module 303, a TCP/IP communication module 304, an AD analog quantity acquisition module 305 and a minimum system module 306.
As shown in fig. 3, the power management module 301 is configured to control power of the main control board; the 485 communication module 302 can be used for the main control board to communicate with the RS485 communication interface through a modbus communication protocol, to obtain data from the data, and/or to send the obtained data to a remote command center; the wireless communication module 303 may be configured to enable the main control board to obtain data from a data source through wireless communication, and/or send the obtained data to a remote command center; the TCP/IP communication module 304 may be configured to enable the main control board to obtain data from a data source through a transmission control protocol TCP and an internet protocol IP, and/or send the obtained data to a remote command center; the AD analog acquisition module 305 may be configured to convert analog data of a data source into digital data, and send the digital data to a remote command center; the minimum system module 306 controls the lowest configuration of the normal operation of the main control board system, in the figure, the main control model of the main control board is MCIMX6Y2CVM08AB, the main frequency of the main control chip is 792MHz, the DDR memory is 512MB, the memory control embedded multimedia memory Card EMMC on the main control chip is 8GB, the external memory space of the flash memory Card TF-Card is 32GB, and the system operation main control board is not lower than the configuration.
The embodiment of the invention also provides a multi-source heterogeneous data acquisition method for the tunneling equipment, which is described in the following embodiment. The principle of the device for solving the problems is similar to that of a multi-source heterogeneous data acquisition system of the tunneling equipment, so the implementation of the device can refer to that of the multi-source heterogeneous data acquisition system of the tunneling equipment, and repeated parts are not described again.
Fig. 4 is a schematic flow chart of a multi-source heterogeneous data acquisition method of tunneling equipment applied to a main control board in the embodiment of the present invention. As shown in fig. 4, a process of the multi-source heterogeneous data acquisition method for tunneling equipment applied to a main control board in the embodiment of the present invention may include:
and 402, when a data source is accessed, acquiring data from the data source according to the data source configuration parameters, and sending the acquired data to a remote command center.
In one implementation, the plurality of data sources are one or more of the following types of data sources:
programmable logic controller PLC, hobbing cutter detection module, vibration monitoring module, analog quantity data module.
In one implementation, the method further comprises performing middleware protocol conversion on the accessed data source.
In one implementation, real-time listening to a plurality of data sources includes:
multithreading real-time monitoring is carried out on a plurality of data sources;
and when the condition that the power-on of the data source is monitored and the communication is smooth is monitored, accessing the data source.
In one implementation, when a data source is accessed, acquiring data from the data source according to the data source configuration parameters, and sending the acquired data to a remote command center, the method includes:
when a data source is accessed, selecting a communication rate and a data acquisition frequency adaptive to the data source according to the configuration parameters of the data source, and acquiring data from the data source;
and selecting a data pushing frequency adaptive to the data source according to the data source configuration parameters, and sending the acquired data to the remote command center.
In one implementation, further comprising:
performing handshake detection with a remote command center to determine a network state;
if the network state is normal, the acquired data are sent to a remote command center;
if the network state is abnormal, the acquired data is temporarily stored in the local database, the handshake detection is carried out again with the remote command center until the network state is normal, and the acquired data is sent to the remote command center.
Fig. 5 is a schematic flow chart of a multi-source heterogeneous data acquisition method of tunneling equipment applied to a remote command center in the embodiment of the present invention. As shown in fig. 5, a flow of the multi-source heterogeneous data acquisition method for tunneling equipment applied to a remote command center in the embodiment of the present invention may include:
In one implementation, the plurality of data sources are one or more of the following types of data sources:
programmable logic controller PLC, hobbing cutter detection module, vibration monitoring module, analog quantity data module.
In one implementation, the first database is a Redis database; and/or the second database is a MongoDB database.
In one implementation, data source configuration parameters stored by a master control board are maintained remotely;
and after the configuration parameters of the data source are modified, triggering the multi-source heterogeneous data acquisition system of the tunneling equipment to restart.
Fig. 6 is a main control board in the multi-source heterogeneous data acquisition system of the tunneling equipment in the embodiment of the present invention. As shown in fig. 6, the main control board in the multi-source heterogeneous data acquisition system of the tunneling equipment in the embodiment of the present invention may include:
a monitoring module 601, configured to monitor multiple data sources in real time; the multiple data sources are used for providing multi-source heterogeneous data of the tunneling equipment;
and the data transmission template 602 is used for acquiring data from the data source according to the data source configuration parameters and sending the acquired data to the remote command center when the data source is accessed.
Fig. 7 is a remote command center in the multi-source heterogeneous data acquisition system of the tunneling equipment in the embodiment of the present invention. As shown in fig. 7, the remote command center in the multi-source heterogeneous data acquisition system of the tunneling equipment in the embodiment of the present invention may include:
a data receiving module 701, configured to receive data sent by a main control board; the main control board is used for monitoring a plurality of data sources in real time, and acquiring data from the data sources according to the data source configuration parameters when the data sources are accessed; the multiple data sources are used for providing multi-source heterogeneous data of the tunneling equipment;
a first storage module 702, configured to store, when the received data is real-time data, the real-time data into different data tables in a first database according to different data types and different data lengths;
the second storage module 703 is configured to, when the received data is historical data, store the historical data into different data tables in the second database according to different data types and different data lengths; wherein the first database and the second database are different databases.
Based on the aforementioned inventive concept, as shown in fig. 8, the present invention further provides a computer apparatus 800, which includes a memory 810, a processor 820, and a computer program 830 stored in the memory 810 and operable on the processor 820, wherein the processor 820 implements the aforementioned multi-source heterogeneous data acquisition method for tunneling equipment when executing the computer program 830.
The embodiment of the invention also provides a computer readable storage medium, which stores a computer program for executing the multi-source heterogeneous data acquisition method of the tunneling equipment.
In summary, the multi-source heterogeneous data acquisition system of the tunneling equipment in the embodiment of the present invention includes: the system comprises a plurality of data sources, a plurality of data processing units and a plurality of data processing units, wherein the data sources are used for providing multi-source heterogeneous data of tunneling equipment; the main control board is used for monitoring a plurality of data sources in real time, acquiring data from the data sources according to the data source configuration parameters when the data sources are accessed, and sending the acquired data to the remote command center; the remote command center is used for receiving the data sent by the main control board, and storing the real-time data into different data tables in the first database according to different data types and data lengths when the received data are real-time data; when the received data is historical data, storing the historical data into different data tables in a second database according to different data types and different data lengths; the first database and the second database are different databases, so that multi-source data coordination collection of the tunneling equipment can be realized, the data storage space is saved, the data management capability is improved, and the subsequent data analysis is facilitated.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The above-mentioned embodiments are intended to illustrate the objects, technical solutions and advantages of the present invention in further detail, and it should be understood that the above-mentioned embodiments are only exemplary embodiments of the present invention, and are not intended to limit the scope of the present invention, and any modifications, equivalent substitutions, improvements and the like made within the spirit and principle of the present invention should be included in the scope of the present invention.
Claims (23)
1. The utility model provides a heterogeneous data acquisition system of multisource of tunneling equipment which characterized in that includes:
the system comprises a plurality of data sources, a plurality of data processing units and a plurality of data processing units, wherein the data sources are used for providing multi-source heterogeneous data of tunneling equipment;
the main control board is used for monitoring a plurality of data sources in real time, acquiring data from the data sources according to the data source configuration parameters when the data sources are accessed, and sending the acquired data to the remote command center;
the remote command center is used for receiving the data sent by the main control board, and storing the real-time data into different data tables in the first database according to different data types and data lengths when the received data are real-time data; when the received data is historical data, storing the historical data into different data tables in a second database according to different data types and different data lengths; wherein the first database and the second database are different databases.
2. The multi-source heterogeneous data acquisition system of tunneling equipment according to claim 1, wherein the plurality of data sources are one or more of the following types of data sources:
programmable logic controller PLC, hobbing cutter detection module, vibration monitoring module, analog quantity data module.
3. The multi-source heterogeneous data acquisition system of tunneling equipment according to claim 1, wherein the main control board is further configured to: and carrying out the protocol conversion of the middleware on the accessed data source.
4. The multi-source heterogeneous data acquisition system of tunneling equipment according to claim 1, wherein the main control board is specifically configured to:
multithreading real-time monitoring is carried out on a plurality of data sources;
and when the condition that the power-on of the data source is monitored and the communication is smooth is monitored, accessing the data source.
5. The multi-source heterogeneous data acquisition system of tunneling equipment according to claim 1, wherein the main control board is specifically configured to:
when a data source is accessed, selecting a communication rate and a data acquisition frequency adaptive to the data source according to the configuration parameters of the data source, and acquiring data from the data source;
and selecting a data pushing frequency adaptive to the data source according to the data source configuration parameters, and sending the acquired data to the remote command center.
6. The multi-source heterogeneous data acquisition system of tunneling equipment according to claim 1, wherein the main control board is further configured to:
performing handshake detection with a remote command center to determine a network state;
if the network state is normal, the acquired data are sent to a remote command center;
if the network state is abnormal, the acquired data is temporarily stored in the local database, the handshake detection is carried out again with the remote command center until the network state is normal, and the acquired data is sent to the remote command center.
7. The tunneling apparatus multi-source heterogeneous data acquisition system of claim 1, wherein the first database is a Redis database; and/or the second database is a MongoDB database.
8. The tunneling apparatus multi-source heterogeneous data acquisition system of claim 1, wherein the remote command center is further configured to:
data source configuration parameters stored by a remote maintenance main control board;
and after the configuration parameters of the data source are modified, triggering the multi-source heterogeneous data acquisition system of the tunneling equipment to restart.
9. The tunneling apparatus multi-source heterogeneous data acquisition system of claim 1, further comprising:
the field scheduling center is used for maintaining the data source configuration parameters stored by the main control board on site; and after the configuration parameters of the data source are modified, triggering the multi-source heterogeneous data acquisition system of the tunneling equipment to restart.
10. A multi-source heterogeneous data acquisition method for tunneling equipment is applied to a main control board in a multi-source heterogeneous data acquisition system of the tunneling equipment, the multi-source heterogeneous data acquisition system of the tunneling equipment further comprises a remote command center and a plurality of data sources, and the method comprises the following steps:
monitoring a plurality of data sources in real time; the multiple data sources are used for providing multi-source heterogeneous data of the tunneling equipment;
when a data source is accessed, data are obtained from the data source according to the data source configuration parameters, and the obtained data are sent to a remote command center.
11. The method of claim 10, wherein the plurality of data sources are one or more of:
programmable logic controller PLC, hobbing cutter detection module, vibration monitoring module, analog quantity data module.
12. The method of claim 10, further comprising:
and carrying out the protocol conversion of the middleware on the accessed data source.
13. The method of claim 10, wherein listening in real-time to a plurality of data sources comprises:
multithreading real-time monitoring is carried out on a plurality of data sources;
and when the condition that the power-on of the data source is monitored and the communication is smooth is monitored, accessing the data source.
14. The method of claim 10, wherein upon access by a data source, obtaining data from the data source based on data source configuration parameters, and sending the obtained data to a remote command center, comprises:
when a data source is accessed, selecting a communication rate and a data acquisition frequency adaptive to the data source according to the configuration parameters of the data source, and acquiring data from the data source;
and selecting a data pushing frequency adaptive to the data source according to the data source configuration parameters, and sending the acquired data to the remote command center.
15. The method of claim 10, further comprising:
performing handshake detection with a remote command center to determine a network state;
if the network state is normal, the acquired data are sent to a remote command center;
if the network state is abnormal, the acquired data is temporarily stored in the local database, the handshake detection is carried out again with the remote command center until the network state is normal, and the acquired data is sent to the remote command center.
16. A multi-source heterogeneous data acquisition method for tunneling equipment is characterized in that the method is applied to a remote command center in a multi-source heterogeneous data acquisition system of the tunneling equipment, and the multi-source heterogeneous data acquisition system of the tunneling equipment further comprises a main control board and a plurality of data sources; the method comprises the following steps:
receiving data sent by a main control board; the main control board is used for monitoring a plurality of data sources in real time, and acquiring data from the data sources according to the data source configuration parameters when the data sources are accessed; the multiple data sources are used for providing multi-source heterogeneous data of the tunneling equipment;
when the received data is real-time data, storing the real-time data into different data tables in a first database according to different data types and data lengths;
when the received data is historical data, storing the historical data into different data tables in a second database according to different data types and different data lengths; wherein the first database and the second database are different databases.
17. The method of claim 16, wherein the plurality of data sources are one or more of:
programmable logic controller PLC, hobbing cutter detection module, vibration monitoring module, analog quantity data module.
18. The method of claim 16, wherein the first database is a Redis database; and/or the second database is a MongoDB database.
19. The method of claim 16, further comprising:
data source configuration parameters stored by a remote maintenance main control board;
and after the configuration parameters of the data source are modified, triggering the multi-source heterogeneous data acquisition system of the tunneling equipment to restart.
20. A main control board in a multi-source heterogeneous data acquisition system of tunneling equipment is characterized in that the multi-source heterogeneous data acquisition system of the tunneling equipment further comprises a remote command center and a plurality of data sources; the main control board includes:
the monitoring module is used for monitoring a plurality of data sources in real time; the multiple data sources are used for providing multi-source heterogeneous data of the tunneling equipment;
and the data transmission template is used for acquiring data from the data source according to the data source configuration parameters and sending the acquired data to the remote command center when the data source is accessed.
21. A remote command center in a multi-source heterogeneous data acquisition system of tunneling equipment is characterized in that the multi-source heterogeneous data acquisition system of the tunneling equipment further comprises a main control board and a plurality of data sources; the remote command center includes:
the data receiving module is used for receiving data sent by the main control board; the main control board is used for monitoring a plurality of data sources in real time, and acquiring data from the data sources according to the data source configuration parameters when the data sources are accessed; the multiple data sources are used for providing multi-source heterogeneous data of the tunneling equipment;
the first storage module is used for storing the real-time data into different data tables in a first database according to different data types and data lengths when the received data are the real-time data;
the second storage module is used for storing the historical data into different data tables in a second database according to different data types and data lengths when the received data are the historical data; wherein the first database and the second database are different databases.
22. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the method of any of claims 10 to 19 when executing the computer program.
23. A computer-readable storage medium storing a computer program for performing the method of any one of claims 10 to 19.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110885040.9A CN113590626A (en) | 2021-08-03 | 2021-08-03 | Multi-source heterogeneous data acquisition system and method for tunneling equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110885040.9A CN113590626A (en) | 2021-08-03 | 2021-08-03 | Multi-source heterogeneous data acquisition system and method for tunneling equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113590626A true CN113590626A (en) | 2021-11-02 |
Family
ID=78254264
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110885040.9A Pending CN113590626A (en) | 2021-08-03 | 2021-08-03 | Multi-source heterogeneous data acquisition system and method for tunneling equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113590626A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116028192A (en) * | 2023-03-29 | 2023-04-28 | 中电科大数据研究院有限公司 | Multi-source heterogeneous data acquisition method, device and storage medium |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109067618A (en) * | 2018-09-06 | 2018-12-21 | 北京奥技异科技发展有限公司 | Distributed real-time data IOT acquisition system and method |
| CN109460401A (en) * | 2018-09-30 | 2019-03-12 | 中铁隧道局集团有限公司 | A kind of intelligentized shield TBM data acquisition and complementing method |
| CN110188102A (en) * | 2019-05-24 | 2019-08-30 | 重庆邮电大学 | A kind of factory's multi-source energy consumption data parallel processing system (PPS) and method |
| CN110781248A (en) * | 2019-09-27 | 2020-02-11 | 浙江省北大信息技术高等研究院 | Multi-source heterogeneous data acquisition method and device |
| CN111897875A (en) * | 2020-07-31 | 2020-11-06 | 平安科技（深圳）有限公司 | Fusion processing method and device for urban multi-source heterogeneous data and computer equipment |
| CN112329099A (en) * | 2020-10-27 | 2021-02-05 | 中国铁路设计集团有限公司 | Intelligent matching method for main tunnel of railway mountain tunnel based on custom database |
| CN112364000A (en) * | 2020-10-29 | 2021-02-12 | 广西电网有限责任公司南宁供电局 | System and method for multi-source heterogeneous data fusion in power industry |
| CN112433998A (en) * | 2020-11-20 | 2021-03-02 | 广东电网有限责任公司佛山供电局 | Multisource heterogeneous data acquisition and convergence system and method based on power system |
| CN112883095A (en) * | 2021-03-02 | 2021-06-01 | 南京德奈特系统科技有限责任公司 | Method, system, equipment and storage medium for multi-source heterogeneous data convergence |
-
2021
- 2021-08-03 CN CN202110885040.9A patent/CN113590626A/en active Pending
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109067618A (en) * | 2018-09-06 | 2018-12-21 | 北京奥技异科技发展有限公司 | Distributed real-time data IOT acquisition system and method |
| CN109460401A (en) * | 2018-09-30 | 2019-03-12 | 中铁隧道局集团有限公司 | A kind of intelligentized shield TBM data acquisition and complementing method |
| CN110188102A (en) * | 2019-05-24 | 2019-08-30 | 重庆邮电大学 | A kind of factory's multi-source energy consumption data parallel processing system (PPS) and method |
| CN110781248A (en) * | 2019-09-27 | 2020-02-11 | 浙江省北大信息技术高等研究院 | Multi-source heterogeneous data acquisition method and device |
| CN111897875A (en) * | 2020-07-31 | 2020-11-06 | 平安科技（深圳）有限公司 | Fusion processing method and device for urban multi-source heterogeneous data and computer equipment |
| CN112329099A (en) * | 2020-10-27 | 2021-02-05 | 中国铁路设计集团有限公司 | Intelligent matching method for main tunnel of railway mountain tunnel based on custom database |
| CN112364000A (en) * | 2020-10-29 | 2021-02-12 | 广西电网有限责任公司南宁供电局 | System and method for multi-source heterogeneous data fusion in power industry |
| CN112433998A (en) * | 2020-11-20 | 2021-03-02 | 广东电网有限责任公司佛山供电局 | Multisource heterogeneous data acquisition and convergence system and method based on power system |
| CN112883095A (en) * | 2021-03-02 | 2021-06-01 | 南京德奈特系统科技有限责任公司 | Method, system, equipment and storage medium for multi-source heterogeneous data convergence |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116028192A (en) * | 2023-03-29 | 2023-04-28 | 中电科大数据研究院有限公司 | Multi-source heterogeneous data acquisition method, device and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20200304342A1 (en) | Intelligent monitoring gateway for numerical control machine tool based on internet of things | |
| CN107908166B (en) | Manufacturing MES data acquisition system of automobile based on industry internet | |
| CN109257208A (en) | A kind of information integrated system and method based on OPC UA | |
| CN110879565A (en) | Dual-computer redundancy control system and redundancy control/fault monitoring method and device thereof | |
| CN103922227B (en) | Hoisting crane monitoring management system | |
| CN103744356A (en) | Intelligent dynamically-configurable controller of tool based on DSP (Digital Signal Processor)/FPGA (Field Programmable Gate Array) and control method | |
| CN105446196B (en) | A kind of Intelligent spraying system and its control method with real-time monitoring and control | |
| CN110502719A (en) | Edge calculations model management system and method | |
| CN112055047B (en) | Data communication method based on publishing and subscribing for intelligent coal-fired power plant | |
| CN115086379B (en) | Numerical control machine tool data acquisition method based on edge calculation and virtualization technology | |
| CN105589410A (en) | Workshop monitoring and management system based on Android platform | |
| CN113590626A (en) | Multi-source heterogeneous data acquisition system and method for tunneling equipment | |
| CN102880146B (en) | Profibus device monitoring and managing system | |
| CN102880149A (en) | Distributed type control system and data report generation method | |
| CN204650226U (en) | A kind of data collector and data acquisition system (DAS) | |
| CN110658752A (en) | Data acquisition system applied to remote cloud monitoring | |
| CN110334900B (en) | Data processing method | |
| CN202815502U (en) | A data acquisition system of a filter stick molding device of a Siemens PLC control system in cigarette manufacturing industry | |
| CN103944271A (en) | Intelligent transformer substation system based on Internet of Things technology and control method thereof | |
| CN113114732B (en) | Automatic data point aligning system and method suitable for multiple links of transformer substation | |
| CN113741311B (en) | Bus control system and method with sub-period high-frequency processing function | |
| CN116095146A (en) | Source configuration method of relay protection information model of main substation and sub station based on block chain | |
| CN112822034B (en) | Service subscription mode-based data transmission method and system between main and distribution network systems | |
| CN113194124B (en) | Distributed real-time database-based DCS background multi-node directory synchronization method | |
| CN110531724B (en) | Remote measurement automatic sampling definition method based on smart power grid dispatching control system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |