CN116070171A - Twin data fusion platform - Google Patents

Twin data fusion platform Download PDF

Info

Publication number
CN116070171A
CN116070171A CN202310202915.XA CN202310202915A CN116070171A CN 116070171 A CN116070171 A CN 116070171A CN 202310202915 A CN202310202915 A CN 202310202915A CN 116070171 A CN116070171 A CN 116070171A
Authority
CN
China
Prior art keywords
data
twin
fusion
alarm
subsystem
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310202915.XA
Other languages
Chinese (zh)
Inventor
刘凯
杨波
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangzhou Yunyun Shutwin Technology Co ltd
Original Assignee
Guangzhou Yunyun Shutwin Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangzhou Yunyun Shutwin Technology Co ltd filed Critical Guangzhou Yunyun Shutwin Technology Co ltd
Priority to CN202310202915.XA priority Critical patent/CN116070171A/en
Publication of CN116070171A publication Critical patent/CN116070171A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24Querying
    • G06F16/245Query processing
    • G06F16/2458Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
    • G06F16/2462Approximate or statistical queries
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/25Integrating or interfacing systems involving database management systems
    • G06F16/258Data format conversion from or to a database
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/28Databases characterised by their database models, e.g. relational or object models
    • G06F16/283Multi-dimensional databases or data warehouses, e.g. MOLAP or ROLAP

Landscapes

  • Engineering & Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Probability & Statistics with Applications (AREA)
  • Fuzzy Systems (AREA)
  • Computational Linguistics (AREA)
  • Software Systems (AREA)
  • Mathematical Physics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention relates to the technical field of digital twinning and discloses a twinning data fusion platform which comprises a central processing unit, a twinning data acquisition module, a twinning data fusion subsystem and a display module, wherein the central processing unit is respectively connected with the twinning data acquisition module, the twinning data fusion subsystem and the display module; the twin data acquisition module is used for acquiring twin data of each subsystem and comprises a data acquisition mode and a communication protocol of each subsystem; the twin data fusion subsystem comprises a streaming computation engine and an online analysis processing OLAP engine; the display module is used for displaying the data fused by the twin data fusion subsystem according to different display modes selected by a user. The twin data fusion platform provided by the invention realizes the access, conversion and integration of operation and maintenance big data from data preparation and data structure design to data in a stream engine, and realizes the mapping and data synchronization of a digital twin model and a physical entity after data fusion.

Description

Twin data fusion platform
Technical Field
The invention relates to the technical field of digital twinning, in particular to a twinning data fusion platform.
Background
The rapid development of modern informatization technologies such as the Internet of things, big data, digital twinning and the like can help management to be more refined, comprehensive and intelligent, and the development of management is promoted. Among these informatization techniques, the digital twin technique can effectively improve informatization and intelligence levels of operation and maintenance management. A digital twin is a digitized model of a physical object that requires real-time receipt of data from the physical object, thereby evolving continuously to maintain consistency with the physical object. For example, a building digital twin records information (virtual awareness of a person) and build information (physical entity of a building) generated by the building during various activities throughout the life cycle. However, in the process of mapping the operation and maintenance data from acquisition to digital twin, many difficulties still exist, such as data dispersion and storage dispersion, and independent construction of each subsystem makes it difficult for the subsystems to perform data communication and linkage, even if the subsystems are in different network segments, and physical networks cannot be communicated. Meanwhile, the independence of the operation and maintenance subsystems enables the information collection modes and the information uploading protocols of the subsystems to be different. The data docking of each subsystem needs customized development, and the workload is excessive. The project period is long, the data acquisition and storage capacity are extremely large, and the timeliness of data query is poor. The inability of digital twin model information to interface with the fortune dimension, the lack of model information makes mapping the fortune dimension to physical entities in the digital twin difficult.
In the field of data fusion, when modeling physical object data, using TSL language to describe the abstract physical object, sacrificing the availability of a common user to obtain more accurate physical object description; in the storage of digital twin model information, a user is required to define equipment and the hierarchical relation of the equipment one by one on a visual operation interface, the hierarchical relation coincides with the equipment hierarchical relation information contained in the information modeling, manpower and material resources are wasted in repeated construction, when data access is performed, the data source is required to be converted into an MQTT protocol to issue a message to a specified service when private data is faced, and a large amount of development work is brought for the data access of various protocols. The data fusion method lacks connection to digital twin model information, and needs to develop code processing on a data source when facing heterogeneous data, so that data fusion and data visualization are not consistent.
Disclosure of Invention
The invention provides a twin data fusion platform, which realizes the access, conversion and integration of operation and maintenance big data from data preparation and data structure design to data in a stream engine, and realizes the mapping and data synchronization of a digital twin model and a physical entity after data fusion.
The invention provides a twin data fusion platform which comprises a central processing unit, a twin data acquisition module, a twin data fusion subsystem and a display module, wherein the central processing unit is respectively connected with the twin data acquisition module, the twin data fusion subsystem and the display module;
the twin data acquisition module is used for acquiring twin data of each subsystem and comprises a data acquisition mode and a communication protocol of each subsystem;
the twin data fusion subsystem comprises a stream computing engine and an online analysis processing OLAP engine, wherein the stream computing engine adopts a Kappa framework and is used for converting data sources of different protocols, converting data structures in the data sources, performing dimension connection, guaranteeing consistency of stateful computation and performing data aggregation by utilizing a window mechanism, and tasks of the stream computing engine comprise a data source layer, a data conversion layer, a data precipitation layer and a data dimension layer; the OLAP engine adopts ClickHouse as a data storage service for storing data calculated and aggregated by the streaming computing engine, adopts aggregation, slicing and drill-down of fact data for data query analysis, utilizes MongoDB as a database for storing data conversion mapping relations of a data cube structure, a dimension structure and the streaming computing engine, and relies on an ECharts chart library to create a data visualization tool for carrying out a code-free configuration chart, and comprises a real-time data interface and a message subscription interface, wherein the real-time data interface is used for acquiring the latest data written by a data conversion layer of the streaming computing engine and taking the data cube as a basic unit, and the message subscription interface is used for subscribing the computing data published by the data precipitation layer in real time;
the display module is used for displaying the data fused by the twin data fusion subsystem according to different display modes selected by a user.
Further, the data source layer is used for receiving data of other systems according to a specified protocol and precipitating the received data into the Kafka service; the data conversion layer is used for connecting data to a data dimension, aggregating the data by utilizing a window mechanism, issuing the latest data into Redis according to keywords, and converting the data structure of the data source layer into a data structure defined in a data cube of the data warehouse; the data precipitation layer is used for writing the data processed in the data conversion layer into a data warehouse and issuing a data result calculated in real time; the data dimension layer is used for persistence and dimension discovery of dimensions in a data cube.
The system further comprises a data alarm module, wherein the data alarm module is connected with the central processing unit and is used for defining alarm rules according to data characteristics and a data acquisition mode so as to alarm data source abnormality or data abnormality; the alarm rule comprises null data or data interruption alarm, numerical value or increment abnormality alarm and keyword alarm, wherein the null data or data interruption alarm is used for setting continuous collection of null values of set quantity or pushing alarm messages when new data are not received within set time, the numerical value or increment abnormality alarm is used for pushing the alarm messages when the collected data value exceeds a set range or the increment of the data is smaller than a set value, the keyword alarm is used for setting a plurality of keywords, and the alarm messages are pushed when the keywords appear.
Further, the streaming computing engine establishes streaming service when data is accessed, performs compatibility on batch processing data or streaming data, converts data with different protocols and different forms into streaming data, adds a processing time watermark, and finally deposits the streaming data into Kafka; the method comprises the following steps:
when data of the MQTT protocol is accessed, a user configures basic information of the MQTT data source, including names, broker addresses, topic names and service quality grades, after the stored information is finished, a FlinkController starts a data source layer task in a cluster, the task firstly reads the persisted stored MQTT source information, then uses an eclipse page ho software development kit to collect subscription data in corresponding MQTT service subscription information, uses a data source layer function operator to collect subscription data, uses a mapping operator to add and process information such as time watermark, data source information identification and the like, and finally deposits the subscription data into a Kafka stream.
Further, the twin data fusion subsystem further includes a data structure conversion rule, where the data structure conversion rule is a data structure conversion rule from a data source to an OLAP data cube, and after the conversion rule is adopted, the flankcontroller starts a data conversion task in the cluster, and specifically includes:
obtaining an input stream from Kafka according to a data conversion mapping rule;
converting the data structure into a data structure in an OLAP data cube by using a Flink FlatMapFile operator;
acquiring dimension information defined in a data cube from a data dimension stream, and connecting the dimension information to each piece of data by using a CoProcessFunction operator according to key values of the data;
updating the latest data into Redis by using RichSinkFunction operator to supply service;
checking whether the data needs to be aggregated by using a time window, if so, opening the time window to aggregate the data by using a ProcessWindow function operator according to a defined rule;
the output data stream is passed to the data precipitation layer.
Further, the fusion of the single data metadata in the twin data fusion subsystem is specifically as follows: designing an OLAP data cube and dimension data structure, defining and configuring data alarm rules to complete data preparation, further configuring data access information, receiving data of a data source by a constructed data fusion system, defining and configuring data conversion and mapping rules, and storing the data after calculation and aggregation in a data warehouse for developing practical functions and data visualization.
Further, the display module comprises a data statistics unit, a data classification unit and a report generation unit, wherein the data statistics unit is used for counting the data fused by the twin data fusion subsystem according to a user instruction, the data classification unit is used for classifying the data fused by the twin data fusion subsystem according to the user instruction, and the report generation unit is used for generating a statistical report for the data fused by the twin data fusion subsystem according to the user instruction.
The beneficial effects of the invention are as follows:
the invention provides a data fusion route based on the Flink frame in the aspects of data preparation, access, conversion and visualization, the data fusion route based on the Flink frame can realize access of data with different protocols and different forms, and an Aviator script analysis tool is introduced during data conversion, so that the requirements of user-defined complex data conversion and filtering rules can be met. The collected information is mapped with the streaming data in the data fusion route, and the visualization effect of each data of the operation and maintenance system can be obtained without codes by depending on a self-developed data visualization engine, so that a user can be supported to analyze the data more freely, and the data value can be mined. The data generated by the physical object continuously can be mapped into the digital twin model with low delay through the system fusion, so that the model evolution is consistent with the physical object, and the historical state of the physical object improves the data value density in the data fusion process, and the data value density is stored in a lasting mode. The application of visual programming in data conversion and mapping can be further researched, so that the whole data fusion process is completely code-free, and the real-time property and consistency of the digital twin model and the physical object are better ensured.
Drawings
FIG. 1 is a schematic diagram of a twin data fusion platform according to the present invention.
FIG. 2 is a schematic diagram of a twin data fusion subsystem according to the present invention.
The achievement of the objects, functional features and advantages of the present invention will be further described with reference to the accompanying drawings, in conjunction with the embodiments.
Detailed Description
It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
As shown in fig. 1, the invention provides a twin data fusion platform, which comprises a central processing unit, a twin data acquisition module, a twin data fusion subsystem and a display module, wherein the central processing unit is respectively connected with the twin data acquisition module, the twin data fusion subsystem and the display module;
the twin data acquisition module is used for acquiring twin data of each subsystem and comprises a data acquisition mode and a communication protocol of each subsystem;
the twin data fusion subsystem includes a streaming computing engine and an online analytical processing OLAP engine.
The method comprises the steps that a Kappa architecture is adopted by a streaming computing engine, the streaming computing engine is used for converting data sources of different protocols, converting data structures in the data sources, performing dimension connection, guaranteeing consistency of stateful computation, and performing data aggregation by utilizing a window mechanism, wherein a common streaming computing engine framework comprises Storm, spark, streaming, flink and the like; the data source layer is used for receiving data of other systems according to a specified protocol and precipitating the received data into the Kafka service; the data conversion layer is used for connecting data to a data dimension, aggregating the data by utilizing a window mechanism, issuing the latest data into Redis according to keywords, and converting the data structure of the data source layer into a data structure defined in a data cube of the data warehouse; the data precipitation layer is used for writing the data processed in the data conversion layer into a data warehouse and issuing a data result calculated in real time; the data dimension layer is used for persistence and dimension discovery of dimensions in a data cube.
The OLAP engine realizes the functions of data storage, data query analysis and data visualization, adopts ClickHouse as a data storage service for storing data calculated and aggregated by the stream computing engine, adopts aggregation, slicing and drill-down of the fact data for data query analysis, and allows a user to normalize and conveniently operate the fact data when querying and analyzing the data; and storing the data cube structure, the dimension structure and the data conversion mapping relation of the stream computing engine by using the MongoDB as a storage data cube structure, and realizing the persistent deployment of the service. Creating a data visualization tool by means of an ECharts chart library so as to perform a code-free configuration chart, wherein the OLAP engine comprises a real-time data interface and a message subscription interface, the real-time data interface is used for acquiring the latest data which is written into Redis by a data conversion layer of the streaming computing engine and takes a data cube as a basic unit, and the message subscription interface is used for subscribing the computing data published by the data precipitation layer in real time;
the data access is the first step of data fusion, the streaming service is established when the streaming computing engine is accessed, the batch processing data or the streaming data are compatible, the data with different protocols and different forms are converted into the streaming data, the processing time watermark is added, and finally the streaming data is deposited in Kafka; the data access in the forms of MQTT, mySQL database, kafka stream, restfulApi and the like is realized by relying on the data source layer function operator provided by the Flink framework.
The data access of the MQTT protocol is specifically as follows:
when data of the MQTT protocol is accessed, a user configures basic information of the MQTT data source, including names, broker addresses, topic names and service quality grades, after the stored information is finished, a FlinkController starts a data source layer task in a cluster, the task firstly reads the persisted stored MQTT source information, then uses an eclipse page ho software development kit to collect subscription data in corresponding MQTT service subscription information, uses a data source layer function operator to collect subscription data, uses a mapping operator to add and process information such as time watermark, data source information identification and the like, and finally deposits the subscription data into a Kafka stream. In order to provide an out-of-box message queue service, a MosquittoMQTT service is deployed in the Flink cluster. Because the service is in the same environment as the flank cluster, the data source layer tasks in the cluster can simply subscribe to messages published by the service.
The display module is used for displaying the data fused by the twin data fusion subsystem according to different display modes selected by a user; the display module comprises a data statistics unit, a data classification unit and a report generation unit, wherein the data statistics unit is used for counting the data fused by the twin data fusion subsystem according to a user instruction, the data classification unit is used for classifying the data fused by the twin data fusion subsystem according to the user instruction, and the report generation unit is used for generating a statistical report for the data fused by the twin data fusion subsystem according to the user instruction.
The twin data fusion platform provided by the invention further comprises a data alarm module, wherein the data alarm module is connected with the central processing unit and is used for defining an alarm rule according to data characteristics and a data acquisition mode so as to perform data source abnormality or data abnormality alarm; the alarm rule comprises null data or data interruption alarm, numerical value or increment abnormality alarm and keyword alarm, wherein the null data or data interruption alarm is used for setting continuous collection of null values of set quantity or pushing alarm messages when new data are not received within set time, the numerical value or increment abnormality alarm is used for pushing the alarm messages when the collected data value exceeds a set range or the increment of the data is smaller than a set value, the keyword alarm is used for setting a plurality of keywords, and the alarm messages are pushed when the keywords appear.
Data conversion and mapping are the core steps in the data fusion process. After successful access to the data source, there are often a number of problems in that the data structure in the data source is different from that of the OLAP data cube; the enumerated values exist in the data source to be converted; the data value density in the data source is low, and the data needs to be aggregated by using a time window; the data in the data source must be filtered; the data in the data source must be linked to the information of the digital twin model. Therefore, the twin data fusion subsystem further includes a data structure conversion rule, where the data structure conversion rule is a data structure conversion rule from a data source to an OLAP data cube, and after the conversion rule is adopted, the flankcontroller starts a data conversion task in the cluster, and specifically includes:
s1, acquiring an input stream from Kafka according to a data conversion mapping rule;
s2, converting the data structure into a data structure in an OLAP data cube by using a Flink FlatMapFile operator;
s3, acquiring dimension information defined in a data cube from a data dimension stream, and connecting the dimension information to each piece of data by using a CoProcessFunction operator according to key values of the data;
s4, updating the latest data into Redis by adopting a RichSinkFunction operator so as to be used by the application service;
s5, checking whether the data needs to be aggregated by using a time window, if so, opening the time window by using a ProcessWindow function operator according to a defined rule;
s6, outputting the data stream to a data precipitation layer.
It is contemplated that each subsystem may need to define complex transformation rules when performing data mapping. Therefore, the Aviator script is introduced to define more complex data conversion rules and data filtering rules, and the logic of the Aviator expression is merged into the flat map flow operator in the step S2, so that the purpose that the user can customize the complex data conversion and filtering rules is achieved.
The fusion of single data metadata in the twin data fusion subsystem is specifically as follows: designing an OLAP data cube and dimension data structure, defining and configuring data alarm rules to complete data preparation, further configuring data access information, receiving data of a data source by a constructed data fusion system, defining and configuring data conversion and mapping rules, and storing the data after calculation and aggregation in a data warehouse for developing practical functions and data visualization.
The invention provides a data fusion route based on the Flink frame in the aspects of data preparation, access, conversion and visualization, the data fusion route based on the Flink frame can realize access of data with different protocols and different forms, and an Aviator script analysis tool is introduced during data conversion, so that the requirements of user-defined complex data conversion and filtering rules can be met. The collected information is mapped with the streaming data in the data fusion route, and the visualization effect of each data of the operation and maintenance system can be obtained without codes by depending on a self-developed data visualization engine, so that a user can be supported to analyze the data more freely, and the data value can be mined. The data generated by the physical object continuously can be mapped into the digital twin model with low delay through the system fusion, so that the model evolution is consistent with the physical object, and the historical state of the physical object improves the data value density in the data fusion process, and the data value density is stored in a lasting mode. The application of visual programming in data conversion and mapping can be further researched, so that the whole data fusion process is completely code-free, and the real-time property and consistency of the digital twin model and the physical object are better ensured.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, apparatus, article, or method. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, apparatus, article or method that comprises the element.
The foregoing description is only of the preferred embodiments of the present invention and is not intended to limit the scope of the invention, and all equivalent structures or equivalent processes using the descriptions and drawings of the present invention or directly or indirectly applied to other related technical fields are included in the scope of the invention.

Claims (7)

1. The twin data fusion platform is characterized by comprising a central processing unit, a twin data acquisition module, a twin data fusion subsystem and a display module, wherein the central processing unit is respectively connected with the twin data acquisition module, the twin data fusion subsystem and the display module;
the twin data acquisition module is used for acquiring twin data of each subsystem and comprises a data acquisition mode and a communication protocol of each subsystem;
the twin data fusion subsystem comprises a stream computing engine and an online analysis processing OLAP engine, wherein the stream computing engine adopts a Kappa framework and is used for converting data sources of different protocols, converting data structures in the data sources, performing dimension connection, guaranteeing consistency of stateful computation and performing data aggregation by utilizing a window mechanism, and tasks of the stream computing engine comprise a data source layer, a data conversion layer, a data precipitation layer and a data dimension layer; the OLAP engine adopts ClickHouse as a data storage service for storing data calculated and aggregated by the streaming computing engine, adopts aggregation, slicing and drill-down of fact data for data query analysis, utilizes MongoDB as a database for storing data conversion mapping relations of a data cube structure, a dimension structure and the streaming computing engine, and relies on an ECharts chart library to create a data visualization tool for carrying out a code-free configuration chart, and comprises a real-time data interface and a message subscription interface, wherein the real-time data interface is used for acquiring the latest data written by a data conversion layer of the streaming computing engine and taking the data cube as a basic unit, and the message subscription interface is used for subscribing the computing data published by the data precipitation layer in real time;
the display module is used for displaying the data fused by the twin data fusion subsystem according to different display modes selected by a user.
2. The twin data fusion platform of claim 1, wherein the data source layer is configured to receive data from other systems according to a specified protocol and to deposit the received data into a Kafka service; the data conversion layer is used for connecting data to a data dimension, aggregating the data by utilizing a window mechanism, issuing the latest data into Redis according to keywords, and converting the data structure of the data source layer into a data structure defined in a data cube of the data warehouse; the data precipitation layer is used for writing the data processed in the data conversion layer into a data warehouse and issuing a data result calculated in real time; the data dimension layer is used for persistence and dimension discovery of dimensions in a data cube.
3. The twin data fusion platform of claim 1, further comprising a data alarm module connected to the central processor, the data alarm module defining alarm rules for data source anomalies or data anomaly alarms based on data characteristics and data acquisition patterns; the alarm rule comprises null data or data interruption alarm, numerical value or increment abnormality alarm and keyword alarm, wherein the null data or data interruption alarm is used for setting continuous collection of null values of set quantity or pushing alarm messages when new data are not received within set time, the numerical value or increment abnormality alarm is used for pushing the alarm messages when the collected data value exceeds a set range or the increment of the data is smaller than a set value, the keyword alarm is used for setting a plurality of keywords, and the alarm messages are pushed when the keywords appear.
4. The twin data fusion platform of claim 1, wherein the streaming computing engine establishes streaming services when data is accessed, is compatible with batch processing data or streaming data, converts data of different protocols and different forms into streaming data, adds a processing time watermark, and finally deposits the streaming data into Kafka; the method comprises the following steps:
when data of the MQTT protocol is accessed, a user configures basic information of the MQTT data source, including names, broker addresses, topic names and service quality grades, after the stored information is finished, a FlinkController starts a data source layer task in a cluster, the task firstly reads the persisted stored MQTT source information, then uses an eclipse page ho software development kit to collect subscription data in corresponding MQTT service subscription information, uses a data source layer function operator to collect subscription data, uses a mapping operator to add and process information such as time watermark, data source information identification and the like, and finally deposits the subscription data into a Kafka stream.
5. The twin data fusion platform of claim 1, wherein the twin data fusion subsystem further comprises a data structure conversion rule, the data structure conversion rule is a data structure conversion rule from a data source to an OLAP data cube, and after the conversion rule is adopted, a flankcontroller starts a data conversion task in a cluster, and the method specifically comprises:
obtaining an input stream from Kafka according to a data conversion mapping rule;
converting the data structure into a data structure in an OLAP data cube by using a Flink FlatMapFile operator;
acquiring dimension information defined in a data cube from a data dimension stream, and connecting the dimension information to each piece of data by using a CoProcessFunction operator according to key values of the data;
updating the latest data into Redis by using RichSinkFunction operator to supply service;
checking whether the data needs to be aggregated by using a time window, if so, opening the time window to aggregate the data by using a ProcessWindow function operator according to a defined rule;
the output data stream is passed to the data precipitation layer.
6. The twin data fusion platform of claim 1, wherein the fusion of individual data metadata in the twin data fusion subsystem is specifically: designing an OLAP data cube and dimension data structure, defining and configuring data alarm rules to complete data preparation, further configuring data access information, receiving data of a data source by a constructed data fusion system, defining and configuring data conversion and mapping rules, and storing the data after calculation and aggregation in a data warehouse for developing practical functions and data visualization.
7. The twin data fusion platform of claim 1, wherein the display module comprises a data statistics unit, a data classification unit and a report generation unit, the data statistics unit is used for counting the data fused by the twin data fusion subsystem according to user instructions, the data classification unit is used for classifying the data fused by the twin data fusion subsystem according to the user instructions, and the report generation unit is used for generating a statistical report for the data fused by the twin data fusion subsystem according to the user instructions.
CN202310202915.XA 2023-03-03 2023-03-03 Twin data fusion platform Pending CN116070171A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310202915.XA CN116070171A (en) 2023-03-03 2023-03-03 Twin data fusion platform

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310202915.XA CN116070171A (en) 2023-03-03 2023-03-03 Twin data fusion platform

Publications (1)

Publication Number Publication Date
CN116070171A true CN116070171A (en) 2023-05-05

Family

ID=86173281

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310202915.XA Pending CN116070171A (en) 2023-03-03 2023-03-03 Twin data fusion platform

Country Status (1)

Country Link
CN (1) CN116070171A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117135181A (en) * 2023-08-15 2023-11-28 三一重型装备有限公司 Data transmission method and device of fully-mechanized mining equipment and electronic equipment

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117135181A (en) * 2023-08-15 2023-11-28 三一重型装备有限公司 Data transmission method and device of fully-mechanized mining equipment and electronic equipment
CN117135181B (en) * 2023-08-15 2024-06-18 三一重型装备有限公司 Data transmission method and device of fully-mechanized mining equipment and electronic equipment

Similar Documents

Publication Publication Date Title
CN112181960B (en) Intelligent operation and maintenance framework system based on AIOps
JPH06502505A (en) entity management system
WO2022083684A1 (en) Road network operation management method and device, storage medium, and terminal
CN112286957B (en) API application method and system of BI system based on structured query language
CN113868306A (en) Data modeling system and method based on OPC-UA specification
CN113420009B (en) Electromagnetic data analysis device, system and method based on big data
CN112148578A (en) IT fault defect prediction method based on machine learning
JP7442001B1 (en) Comprehensive failure diagnosis method for hydroelectric power generation units
CN116070171A (en) Twin data fusion platform
CN112464123B (en) Water quality monitoring data visualization system and method based on micro-service
CN114372084A (en) Real-time processing system for sensing stream data
CN114218218A (en) Data processing method, device and equipment based on data warehouse and storage medium
CN112749153A (en) Industrial network data management system
CN112241424A (en) Air traffic control equipment application system and method based on knowledge graph
Ribeiro et al. A data integration architecture for smart cities
CN115439015B (en) Local area power grid data management method, device and equipment based on data middleboxes
CN113255026B (en) CAD collaborative design method based on semantic information exchange
Huang Geopubsubhub: A geospatial publish/subscribe architecture for the world-wide sensor web
CN115391429A (en) Time sequence data processing method and device based on big data cloud computing
CN113342874A (en) Wind power big data analysis system and process based on cloud computing
CN114625763A (en) Information analysis method and device for database, electronic equipment and readable medium
CN112180861A (en) Integrated bridge monitoring system
Yuwana et al. An ontology tropical weather model for sensor network interoperability
CN115914379B (en) Data exchange device and data exchange system
US20230401347A1 (en) Plant infrastructure modelling

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination