CN112667728B - Visual single machine data acquisition method in wharf efficiency analysis - Google Patents

Visual single machine data acquisition method in wharf efficiency analysis Download PDF

Info

Publication number
CN112667728B
CN112667728B CN202110011595.0A CN202110011595A CN112667728B CN 112667728 B CN112667728 B CN 112667728B CN 202110011595 A CN202110011595 A CN 202110011595A CN 112667728 B CN112667728 B CN 112667728B
Authority
CN
China
Prior art keywords
management
data
module
rule
extraction
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202110011595.0A
Other languages
Chinese (zh)
Other versions
CN112667728A (en
Inventor
梁海玉
黄清波
郜长庆
黄雷
原好
范文静
熊会
钱晶晶
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shanghai Zhenghua Heavy Industries Co Ltd
Original Assignee
Shanghai Zhenghua Heavy Industries Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shanghai Zhenghua Heavy Industries Co Ltd filed Critical Shanghai Zhenghua Heavy Industries Co Ltd
Priority to CN202110011595.0A priority Critical patent/CN112667728B/en
Publication of CN112667728A publication Critical patent/CN112667728A/en
Application granted granted Critical
Publication of CN112667728B publication Critical patent/CN112667728B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Abstract

The invention discloses a visualized single-machine data acquisition method in wharf efficiency analysis in the technical field of single-machine data acquisition, which adopts the latest front-edge ETL technology to realize data acquisition of various wharf single-machine devices; by introducing an expression engine technology, the expression of diversified data types and formats by a unified expression is realized; a task scheduling technology is introduced to realize task scheduling; then converting and processing the irregular data source into standard and clean effective data through the acquisition system; by adopting the front-end and back-end separation technology, a visual front-end configuration interface is provided, the application of a web end is supported, and the visual single-machine data acquisition method saves a great deal of manpower and material resources, greatly improves the usability of the system, ensures that the whole acquisition system data is more accurate and efficient, provides a friendly visual interface and improves the user experience.

Description

Visual single machine data acquisition method in wharf efficiency analysis
Technical Field
The invention relates to the technical field of single-machine data acquisition, in particular to a visual single-machine data acquisition method in wharf efficiency analysis.
Background
With the development of container automation wharf business, users pay more attention to the efficiency of wharf equipment operation, a wharf efficiency analysis system is basically a wharf standard system, the analysis of single-machine operation efficiency is important, the core of efficiency analysis is equipment operation data analysis, and the key of successful data acquisition and processing is often that of the project.
Because the great difference of the service of the equipment and the wharf in the prior art leads to great difference of the equipment, the wharf equipment at present mainly comprises a double-trolley shore bridge, a single-trolley shore bridge, a tire crane, a track crane and the like, the difference of the wharf service mainly comprises a plurality of definitions of each wharf for managing the operation flow, and the complexity of collecting the operation data of the equipment is increased by integrating the aspects, such as: how to realize the persistence of the collected data of the equipment operation; how to realize the unified collection rule of the equipment operation data; how to realize the scheduling management of a multitasking acquisition plan; how to realize the application of the scheduling configuration of the acquisition plan on the web side; how to implement user account rights management; how to realize the technical problem of the operation state monitoring of the acquisition system.
Disclosure of Invention
The invention aims to provide a visual single machine data acquisition method in wharf efficiency analysis, which solves the technical problems that in the prior art, equipment operation data acquisition is complicated, equipment operation acquisition data cannot be durable, acquisition rules of the equipment operation data cannot be unified, scheduling management of a multi-task acquisition plan cannot be realized, application of an acquisition system on a web end cannot be realized, user account authority management and operation state monitoring of the acquisition system cannot be realized, and the like.
In order to achieve the above purpose, the present invention adopts the following technical scheme:
according to one aspect of the present invention, there is provided a method of visual stand-alone data acquisition in dock efficiency analysis, comprising the steps of,
step 1, the TBI ETL platform uniformly extracts data from the InfluxDB time sequence database to realize data acquisition of various wharf single-machine devices; adopting ZPMC OPC technology, and pushing and persisting the single-machine data into InfluxDB through middleware to finish single-machine data persistence;
step 2, introducing an expression engine technology to realize the expression of diversified data types and formats by using a unified expression; and a task scheduling technology is introduced to realize task scheduling. The method comprises the steps of carrying out a first treatment on the surface of the
Step 3, summarizing all the data extracted in the step 1 according to the extraction requirement, defining the generation of each piece of data as a standard rule, and converting an irregular data source into standard and clean effective data; and 4, providing a visual front-end configuration interface by adopting a front-end and back-end separation technology, so that the application of the scheduling configuration of the acquisition plan on the web end is realized, and the running process of the back-end scheduling task is not influenced.
According to the visual single machine data acquisition method in the dock efficiency analysis of the above aspect of the present invention, the TBI ETL platform in the step 1 includes:
the source library management module is used for configuring an InfluxDB database address and information of equipment and TAG points in TBI OPD engineering and is used for executing the extraction module; and/or
The target base management module is used for realizing the configuration of basic information of the target base, namely, the data extracted from InfluxDB is processed and finally stored in which tables in which database are stored, namely, the connection character strings of the target database and the table information in the target database are configured; and/or
The scheduling module is used for managing, namely, configuring scheduling information of different execution frequencies for the execution extraction module to use for scheduling management; and/or
The extraction execution module is used for realizing the function that one set of platform meets the use of all projects; and/or
The monitoring log module is used for realizing the management of log system information extraction; and/or
And the system information management module is used for maintaining the core basic information of the TBI ETL platform.
According to the visual single machine data acquisition method in the dock efficiency analysis of the above aspect of the invention, the source library management module comprises HOST management, data source management, data table management, TAG management and EQUIP management.
According to the visual single machine data acquisition method in the dock efficiency analysis of the above aspect of the invention, the target library management module comprises data source management of a target library, data table management of the target library and table information management of the target library.
According to the visual single machine data acquisition method in the dock efficiency analysis of the above aspect of the invention, the extraction execution module management comprises an extraction rule configuration module; a configuration for providing a series of expressions to achieve various acquisition requirements; and/or a rule extraction analysis module for realizing the function of generating data from the InfluxDB database into the target database through processing.
According to the visual single machine data acquisition method in the dock efficiency analysis of the above aspect of the invention, the monitoring log module comprises extraction history management and/or TAG point final record management.
According to the visual stand-alone data acquisition method in the dock efficiency analysis of the above aspect of the invention, the system information management module comprises user management, menu management, role management and dictionary management.
According to the visual single machine data acquisition method in the dock efficiency analysis of the aspect of the invention, the step 2 comprises the following steps:
s21, using an expression engine of the extraction rule based on the google averager in the step 1 to realize expression of the data extraction rule;
s22, a Spring integrated Quartz mode is used, namely development of timing tasks and realization of a scheduling module are realized, and reliable and efficient normal execution of data extraction is realized.
The method for visual single machine data acquisition in the dock efficiency analysis according to the above aspect of the present invention, wherein the rule in the step 3 comprises
A Start Rule (Start Rule), i.e., a service data generation Rule; and/or
An initial start Rule (StartInit Rule), i.e. an update Rule when service data is generated; and/or
Update Rule (Update Rule), i.e. business data Update Rule; and/or
An End Rule (End Rule), i.e., a service data End Rule; and/or
The initial end Rule (EndInit Rule), i.e. the update Rule at the end of the service data.
According to the visual single machine data acquisition method in the dock efficiency analysis of the above aspect of the invention, the step 4 comprises the following steps:
s41, deploying a B/S architecture by the system, and completing configuration management of the system;
s42, providing a Rest API interface at the rear end, and facilitating interaction between the front end and the rear end;
s43, providing an friendly visual configuration interface and a monitoring interface by adopting a front-rear end separation technology, and monitoring the running condition of the system in real time;
s44, adopting a SpringSecurity+JWT rights management technology, and carrying out refined rights management to ensure the safety of the system;
s45, log back log management is introduced, the application range of the log is finely controlled, and the running condition of the system is monitored.
By adopting the technical scheme, the invention has the following advantages:
compared with the traditional wharf efficiency analysis technology, the visualized single machine data acquisition method provided by the invention saves a great amount of manpower and material resources, and greatly improves the usability of the system; introducing an expression engine technology, and regularly carding scattered data to form standardization; the whole acquisition system data is more accurate and efficient, a B/S architecture is adopted, a front-end and back-end separation technology is realized, a friendly visual interface is provided, and user experience is improved.
Drawings
Other features, objects and advantages of the present invention will become more apparent upon reading of the detailed description of non-limiting embodiments, made with reference to the following drawings.
FIG. 1 is a architectural solution screenshot of a visual stand-alone data acquisition of the present invention;
FIG. 2 is a data rule description screenshot of the present invention;
FIG. 3 is a screen shot of a visual stand-alone data acquisition system deployment architecture of the present invention;
FIG. 4 is a data source management screen shot of the present invention;
FIG. 5 is a acquisition task scheduling management screen shot of the present invention;
FIG. 6 is an acquisition configuration management screen shot of the present invention;
FIG. 7 is a collection monitoring management screen shot of the present invention;
fig. 8 is a system information management screen shot of the present invention.
Detailed Description
The detailed features and advantages of the present invention will be readily apparent to those skilled in the art from the following detailed description, claims, and drawings that follow, taken in conjunction with the accompanying drawings.
Examples
FIG. 1 illustrates an architectural solution screenshot of a visualized stand-alone data acquisition; FIG. 2 shows a screenshot of a data rule specification; the invention provides a visualized single machine data acquisition method in wharf efficiency analysis, which specifically comprises the following steps of flow,
step 1: the TBI ETL platform in the data acquisition service layer uniformly extracts data from the InfluxDB time sequence database to realize data acquisition of various wharf single-machine devices in the single-machine device data; in one embodiment, the InfluxDB time sequence database is introduced (the data source is unified), all single machine signal history data are unified and persisted to the InfluxDB time sequence database, the data safety is ensured, the TBI ETL platform is unified for acquiring the data from the InfluxDB time sequence database, the unification of the data source format is ensured, and the data persistence is realized as shown in a specific figure 1.
The TBI ETL platform comprises 6 sub-modules, namely a source library management module, a target library management module, a scheduling module management, an extraction execution module management, a monitoring log module and a system information management module.
Wherein: the source library management module mainly comprises HOST management, data source management, data table management, TAG management and EQUIP management; in one embodiment, the source bank management module is used for configuring the InfluxDB database address, and in another embodiment, the source bank management module is used for information of equipment and TAG points in TBI OPD engineering and used for executing extraction module.
The target library management module comprises data source management of a target library, data table management of the target library and table information management of the target library; in one embodiment, the objective database management module mainly realizes objective database basic information configuration, namely, the data extracted from the InfluxDB time sequence database is processed and finally stored in which tables in which database are stored, and the connection character strings of the objective database and the table information in the objective database are configured.
In one embodiment, the function of the scheduling module management is mainly to configure scheduling information of different execution frequencies for the execution extraction module to use for scheduling management.
The extraction execution module management mainly comprises an extraction rule configuration module and an extraction rule analysis module. The extraction rule configuration module can realize the configuration of various acquisition requirements through a series of expression methods, the extraction rule analysis module is used for analyzing the configuration rules, and the platform-based expression rule analysis module can accurately complete the acquisition and processing work of data and realize the function of generating the data from the InfluxDB database to the target database through processing. In one embodiment, the extraction execution module management provides extraction category management and provides extraction device management, in another embodiment, the extraction execution module management provides association device rule management, sort rule management, and base rule management functions, and in yet another embodiment, the extraction execution module management provides extraction target table management. The extraction execution module realizes the function that one set of platform meets the use of all projects.
The monitoring log module mainly realizes the management of log system information extraction, including extraction history management and final record management of TAG points.
The system information management module is mainly used for maintaining core basic information of the system, including user management, menu management, role management and dictionary management.
In one embodiment, step 2: adopting ZPMC OPC technology, pushing and persisting single-machine data into InfluxDB through middleware to complete single-machine data persistence, and collecting data in Influxdb into a target database through rules to realize persistence of extracted data, wherein the persistence is shown in figure 1;
wherein: step 2 specifically includes the following embodiments;
in one embodiment, S21: the expression engine based on the google averager is used to realize a regular, lightweight and high-performance Java expression execution engine, so that the data extraction rule is expressed by an expression, and the unified collection rule of the data is realized;
in another embodiment, S22: the timing tasks are easily completed by using Spring to integrate Quartz, the realization of a scheduling module can ensure reliable and efficient normal execution of data extraction, different time scheduling settings can be realized by different extractions, the control of execution, suspension, stop and the like is convenient, and the scheduling management of a multi-task acquisition plan is realized.
Step 3: the standardized definition of the data generation Rule, summarizing all the data extraction requirements, in one embodiment, defining the generation of one data record into 5 standard rules, specifically including a Start Rule (Start Rule), namely a service data generation Rule; and/or an initial start Rule (Startlnit Rule), i.e. an update Rule when service data is generated; and/or Update Rule (Update Rule), i.e., business data Update Rule; and/or End Rule (End Rule), i.e., business data End Rule; and/or an initial end Rule (endnit Rule), i.e. an update Rule at the end of service data, is specifically shown in fig. 2, and the standardization of the data generation Rule provides a basis for the standardized development of the system.
FIG. 3 illustrates a visual stand-alone data acquisition system deployment architecture screen shot; FIG. 4 illustrates a data source management screen shot; FIG. 5 illustrates an acquisition task scheduling management screen shot; FIG. 6 illustrates acquisition configuration management screen shots; FIG. 7 illustrates acquisition monitoring management screen shots; FIG. 8 illustrates a system information management screen shot;
step 4: and a front-end and rear-end separation technology is adopted to provide a visual front-end configuration interface so as to realize the application of the visual single-machine data acquisition system on the web end.
The step 4 comprises the following embodiments:
in one embodiment, S41: the system deploys a B/S architecture, and the configuration management of the completed system is specifically shown in FIG. 3;
in another embodiment, S42: the rear end provides a Rest API interface, so that the interaction of the front end and the rear end is convenient;
in yet another embodiment, S43: providing an friendly visual configuration interface and a monitoring interface by adopting a front-back end separation technology, and monitoring the running condition of the system in real time;
in yet another embodiment, S44: adopting a SpringSecurity+JWT rights management technology, carrying out refined rights management, ensuring the security of the system and realizing the rights management of a user account;
in the next embodiment, S45: and (3) log management is introduced, the application range of the log is finely controlled, the running condition of the system is monitored, and the running state monitoring of the acquisition system is realized.
As one embodiment of the invention, by combining the implementation experience of the existing KPI project and carrying out repeated debugging, a set of data acquisition platform which can be applied to various types of wharf projects is realized, wherein project variability can be subjected to user definition through visual configuration, the application examples of the data acquisition platform are specifically shown in fig. 4, 5, 6, 7 and 8, the response speed of the wharf KPI project is greatly improved through a visual single-machine data acquisition method in wharf efficiency analysis, a diversified wharf KPI system is realized in shorter time, and the later implementation cost of development team projects is reduced; meanwhile, the expandability and the configurability of the platform can further improve the overall business level of the wharf. At present, similar product cases are rarely seen at home and abroad, and the innovation fills a gap in the domestic field.
Finally, it is pointed out that while the invention has been described with reference to a specific embodiment thereof, it will be understood by those skilled in the art that the above embodiments are provided for illustration only and not as a definition of the limits of the invention, and various equivalent changes or substitutions may be made without departing from the spirit of the invention, therefore, all changes and modifications to the above embodiments shall fall within the scope of the appended claims.

Claims (9)

1. The visualized single machine data acquisition method in wharf efficiency analysis is characterized by comprising the following steps of:
step 1, the TBI ETL platform uniformly extracts data from the InfluxDB time sequence database to realize data acquisition of various wharf single-machine devices;
step 2, introducing an expression engine technology to realize the expression of diversified data types and formats by using a unified expression; a task scheduling technology is introduced to realize task scheduling;
step 3, summarizing all the data extracted in the step 1 according to the extraction requirement, defining the generation of each piece of data as a standard rule, and converting an irregular data source into standard and clean effective data;
step 4, a front-end and back-end separation technology is adopted to provide a visual front-end configuration interface to realize the application of the scheduling configuration of the acquisition plan on the web end, and the operation process of the back-end scheduling task is not influenced;
the TBI ETL platform in the step 1 comprises
The source library management module: the information of equipment and TAG points in TBI OPD engineering is used for configuring InfluxDB database addresses and executing extraction modules; and/or
The target library management module is used for realizing the basic information configuration of the target library, namely, the data extracted from InfluxDB is processed and finally stored in which tables in which database are stored, namely, the connection character strings of the target database and the table information in the target database are configured; and/or
The scheduling module is used for managing the scheduling information mainly for configuring the scheduling information with different execution frequencies for the execution and extraction module to perform scheduling management; and/or
The extraction execution module is used for realizing the function that one set of platform meets the use of all projects; and/or
And a monitoring log module: the management of log system information extraction is realized; and/or
And a system information management module: and the system is used for maintaining the core basic information of the TBI ETL platform.
2. The visual stand-alone data collection method of claim 1, wherein the source library management module comprises HOST management, data source management, data table management, TAG management, and EQUIP management.
3. The visual stand-alone data collection method of claim 1, wherein the target library management module comprises data source management of the target library, data table management of the target library, and table information management of the target library.
4. The visual stand-alone data collection method of claim 1, wherein the extraction execution module management comprises an extraction rule configuration module; a configuration for providing a series of expressions to achieve various acquisition requirements; and/or a rule extraction analysis module for realizing the function of generating data from the InfluxDB database into the target database through processing.
5. A method of visual stand-alone data acquisition in a dock efficiency analysis according to claim 1, wherein the monitoring log module includes extraction history management and/or TAG point last record management.
6. The visual stand-alone data collection method of claim 1, wherein the system information management module comprises user management, menu management, character management, and dictionary management.
7. The method for visual stand-alone data collection in dock efficiency analysis of claim 1, wherein step 2 comprises the steps of:
s21, using an expression engine of the extraction rule based on the google averager in the step 1 to realize expression of the data extraction rule;
s22, a Spring integrated Quartz mode is used, namely development of timing tasks and realization of a scheduling module are realized, and reliable and efficient normal execution of data extraction is realized.
8. The method for visual stand-alone data collection in a dock efficiency analysis according to claim 1, wherein the rules in step 3 include
Start rule: generating rules of service data; and/or
Initial start rule: updating rules during business data generation; and/or
Updating rules: updating rules of service data; and/or
Ending the rule: business data end rules; and/or
Initial end rule: and updating the rule when the service data is finished.
9. The method for visual stand-alone data collection in a dock efficiency analysis of claim 1, wherein said step 4 comprises the steps of:
s41, deploying a B/S architecture by the system, and completing configuration management of the system;
s42, providing a Rest API interface at the rear end, and facilitating interaction between the front end and the rear end;
s43, providing an friendly visual configuration interface and a monitoring interface by adopting a front-rear end separation technology, and monitoring the running condition of the system in real time;
s44, adopting a SpringSecurity+JWT rights management technology, and carrying out refined rights management to ensure the safety of the system;
s45, log back log management is introduced, the application range of the log is finely controlled, and the running condition of the system is monitored.
CN202110011595.0A 2021-01-06 2021-01-06 Visual single machine data acquisition method in wharf efficiency analysis Active CN112667728B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110011595.0A CN112667728B (en) 2021-01-06 2021-01-06 Visual single machine data acquisition method in wharf efficiency analysis

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110011595.0A CN112667728B (en) 2021-01-06 2021-01-06 Visual single machine data acquisition method in wharf efficiency analysis

Publications (2)

Publication Number Publication Date
CN112667728A CN112667728A (en) 2021-04-16
CN112667728B true CN112667728B (en) 2023-11-21

Family

ID=75413091

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110011595.0A Active CN112667728B (en) 2021-01-06 2021-01-06 Visual single machine data acquisition method in wharf efficiency analysis

Country Status (1)

Country Link
CN (1) CN112667728B (en)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2017166644A1 (en) * 2016-03-31 2017-10-05 乐视控股(北京)有限公司 Data acquisition method and system
KR101907656B1 (en) * 2017-08-29 2018-10-12 주식회사 아모티 Trouble diagnosis and management system for electrical vehicle charger
CN109669983A (en) * 2018-12-27 2019-04-23 杭州火树科技有限公司 Visualize multi-data source ETL tool
CN111984709A (en) * 2019-05-23 2020-11-24 云南青年学园科技有限公司 Visual big data middle station-resource calling and algorithm

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2017166644A1 (en) * 2016-03-31 2017-10-05 乐视控股(北京)有限公司 Data acquisition method and system
KR101907656B1 (en) * 2017-08-29 2018-10-12 주식회사 아모티 Trouble diagnosis and management system for electrical vehicle charger
CN109669983A (en) * 2018-12-27 2019-04-23 杭州火树科技有限公司 Visualize multi-data source ETL tool
CN111984709A (en) * 2019-05-23 2020-11-24 云南青年学园科技有限公司 Visual big data middle station-resource calling and algorithm

Also Published As

Publication number Publication date
CN112667728A (en) 2021-04-16

Similar Documents

Publication Publication Date Title
CN109861844B (en) Cloud service problem fine-grained intelligent tracing method based on logs
US8719784B2 (en) Assigning runtime artifacts to software components
CN102810057B (en) A kind of method of log
US11269822B2 (en) Generation of automated data migration model
CN106649100A (en) Automatic test method and system
JP2017033521A (en) Computer utilization system for integrating and displaying (presenting) heterogeneous information and computer utilization method
CN101772760A (en) Database management program and database management device
CN102053912A (en) Device and method for automatically testing software based on UML (unified modeling language) graphs
CN112559306B (en) User behavior track obtaining method and device and electronic equipment
CN108021509B (en) Test case dynamic sequencing method based on program behavior network aggregation
CN105512044A (en) Method and system for updating object base used for keyword drive test
CN105138621A (en) Configuration optimization system and method for Sybase ASE database
Allier et al. Restructuring object-oriented applications into component-oriented applications by using consistency with execution traces
US8086997B2 (en) Detecting aspectual behavior in unified modeling language artifacts
CN105487971A (en) Method and system for recording and reproducing operation steps in software testing
CN106250429A (en) A kind of data pick-up method based on sqoop
CN112667728B (en) Visual single machine data acquisition method in wharf efficiency analysis
CN101930401B (en) Detection object-based software vulnerability model detection method
CN101937395B (en) Detected object program feature extraction method for vulnerability detection
CN105760281A (en) Method for monitoring and analyzing performance data of Linux system server
CN101373506B (en) Method for detecting software loophole model based on loophole model
Wang et al. Design pattern detection in Eiffel systems
Konopka et al. Untangling development tasks with software developer's activity
CN107220116A (en) Sandbox environment task processing method and system under a kind of NUMA architecture
CN116627609A (en) Hive batch processing-based scheduling method and device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant