CN116186132A - Multi-system multi-service visual system access tool - Google Patents

Multi-system multi-service visual system access tool Download PDF

Info

Publication number
CN116186132A
CN116186132A CN202211688534.9A CN202211688534A CN116186132A CN 116186132 A CN116186132 A CN 116186132A CN 202211688534 A CN202211688534 A CN 202211688534A CN 116186132 A CN116186132 A CN 116186132A
Authority
CN
China
Prior art keywords
data
task
synchronization
real
time
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202211688534.9A
Other languages
Chinese (zh)
Inventor
谢莉蕊
石伟
刘永进
曹玉龙
谢超男
王玅
张文茹
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Aerospace Intelligent Technology Development Co ltd
Original Assignee
Beijing Aerospace Intelligent Technology Development Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Aerospace Intelligent Technology Development Co ltd filed Critical Beijing Aerospace Intelligent Technology Development Co ltd
Priority to CN202211688534.9A priority Critical patent/CN116186132A/en
Publication of CN116186132A publication Critical patent/CN116186132A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/25Integrating or interfacing systems involving database management systems
    • G06F16/252Integrating or interfacing systems involving database management systems between a Database Management System and a front-end application
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/21Design, administration or maintenance of databases
    • G06F16/215Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/25Integrating or interfacing systems involving database management systems
    • G06F16/258Data format conversion from or to a database
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/27Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02DCLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00Energy efficient computing, e.g. low power processors, power management or thermal management

Abstract

The invention provides a multi-system multi-service visual system access tool. A multi-system multi-service visualization system access tool, comprising: the data source management module is used for managing the data links; the data target template management module is used for managing the data target templates; the data synchronization module is used for providing a synchronous task configuration scheme; the task management module is used for creating a data synchronization task and managing the data synchronization task; and the visualization module is used for carrying out visual display on the data synchronization task. The key of the invention is to provide a visual system access tool for multiple services and multiple systems, which realizes the real-time collection of different types of data in enterprises, establishes the connection relation of different system data, analyzes and sorts the data, ensures that the data of different systems in the enterprises have due value, realizes the data-driven enterprise production and decision making, and ensures that the enterprise operation and decision making are more scientific and reasonable.

Description

Multi-system multi-service visual system access tool
Technical Field
The invention relates to the technical field of data processing, in particular to a multi-system multi-service visual system access tool.
Background
As manufacturing industry transformation and upgrading steps continue to accelerate, today enterprises have a large number of systems, such as warehouse management systems, manufacturing execution systems, and enterprise resource planning systems. The data island phenomenon is caused by the influence of factors such as enterprise culture and technical architecture of an enterprise, and different systems of the same enterprise are not communicated, so that the strength of the data is difficult to release, and the data cannot exert due value.
Disclosure of Invention
Aiming at the problems, the application provides a multi-system multi-service visual system access tool which can collect data of different systems, establish connection relation of the data of the different systems and enable the data of the different systems in enterprises to exert due value.
The technical scheme of the invention is as follows: a multi-system multi-service visualization system access tool, comprising:
the data source management module is used for managing data links, including newly-built data sources, test links, modification links, copy links, query links and deletion links;
the data target template management module is used for managing the data target templates, and comprises the steps of creating a data target template, modifying the target template, rapidly establishing a data link in a templatization mode and deleting the data target template, wherein the data target template is a data relation mapping model between a system and a database;
the data synchronization module is used for providing a synchronous task configuration scheme, and comprises a one-key real-time synchronization scheme, an incremental real-time synchronization scheme and an offline synchronization scheme;
the task management module is used for creating a data synchronization task and managing the data synchronization task, and comprises creating the task, starting the task, monitoring the task, editing the task, copying the task, resetting the task and deleting the task;
and the visualization module is used for carrying out visual display on the data synchronization task.
Further, for the data synchronization module, the day of the synchronization task configuration is the T-th day, and the method further includes:
for the one-key real-time synchronization scheme, firstly synchronizing the full-quantity data, then synchronizing the incremental data to the target end in real time, synchronizing the task configuration on the same day, writing the full-quantity data of the source table into a T-1 partition in the target table at one time, merging the incremental data of the source table with the full-quantity data, and writing the combined incremental data of the source table into the T-1 partition of the target table in real time; on the N th day of synchronous task configuration, the incremental data of the source table and the full data are combined and then written into the T+N-1 partition of the table in real time;
for the incremental real-time synchronization scheme, only incremental data is synchronized to the target.
Further, for the offline synchronization scheme, the method further includes:
the offline full-increment synchronization is carried out, the full-quantity data is firstly synchronized, then the incremental data is periodically synchronized to a target end, the full-quantity data of the source table is written into the target table T-1 partition once on the same day when the synchronization task is configured, and the incremental data of the source table is periodically written into the target table T+N-1 partition on the t+N day when the synchronization task is configured;
periodically synchronizing the full data of the source table to the target end, configuring the T+1th day by the synchronous task, and periodically writing the full data of the source table into the T partition of the target table;
the method comprises the steps of carrying out one-time full-volume synchronization, synchronizing source table full-volume data to a target end at one time, and writing the source table full-volume data into a target table T-1 partition at one time on the same day of synchronous task configuration;
periodically performing incremental synchronization, namely periodically synchronizing incremental data of a source table to a target end, configuring the T+1th day by a synchronous task, and periodically writing the incremental data of the source table into a T partition of the target table;
and (3) performing one-time increment synchronization, namely synchronizing the source table increment data to the target end, wherein the task is configured on the current day T, and the source table increment data is written into the target table T-1 partition at one time.
Further, for the one-touch real-time synchronization scheme, the method further includes:
the real-time synchronization of the data is realized through the data real-time reading unit, the data conversion unit and the data writing unit, and for a single one-key real-time synchronization scheme, the data read by the data real-time reading unit is supported to be cleaned together by the plurality of data conversion units, and the data write-in units are supported to output the data cleaned by the data conversion units to a target database of a target end;
the one-key real-time synchronization scheme comprises real-time synchronization of single-table incremental data and real-time synchronization of whole-library incremental data;
the method includes the steps of synchronizing incremental data of a single table in real time, including data filtering, character string replacement and data desensitization processing on an input data source, and outputting the processed data to a target database of a target end;
and for real-time synchronization of the whole library incremental data, writing the data into a homonym table in the target end by default, and if the homonym table does not exist, establishing a new homonym table by default.
Further, for the offline synchronization scheme, the method further includes:
supporting the establishment of an offline synchronous task through a configuration mode configuration task, a script mode configuration task and an OpenAPI configuration task;
for the configuration task through the configuration mode, providing guide type development guidance, and completing the configuration work of the data synchronization task through visual filling and guidance of the next step;
for configuration tasks through script modes, providing script type development guidance, and completing data synchronous development by directly writing a JSON script of data synchronization;
for a configuration task through an OpenAPI, a data synchronization task is created by calling a related interface through the OpenAPI.
Further, the method further comprises the following steps:
the task rule management module is used for creating rules of data verification, data operation and data processing and managing the created rules;
the task rule management module comprises:
the rule creation unit is used for customizing rules of data verification, data operation and data processing;
a rule enabling and disabling unit for enabling and disabling the rule,
the rule editing unit is used for modifying the content of the rule, including modifying and storing information such as data distribution destination, data processing and the like under the rule, deleting the rule, enabling the rule in a state not to allow editing, and modifying and deleting the rule after the rule is disabled.
Further, the data operation and maintenance module is used for checking the operation details of the data synchronization task, and monitoring the resources used for operating the data synchronization task, and comprises the steps of acquiring the overall operation condition of the data synchronization task and the change trend of resource scheduling during the operation of the data synchronization task;
the data operation and maintenance module comprises a real-time synchronous task operation and maintenance unit and an offline synchronous task operation and maintenance unit.
Further, the method further comprises the following steps:
for a real-time synchronous task operation and maintenance unit, displaying running state distribution images of real-time synchronous tasks in a selected date, displaying the data synchronous speed of the real-time synchronous tasks, counting the sum of the speeds, and displaying the names of the tasks with the highest delay in a preset number;
for an offline synchronous task operation and maintenance unit, displaying an operation state distribution image of a real-time synchronous task in a selected date, and displaying a data synchronous quantity of the offline synchronous task in the selected date, wherein the data synchronous quantity comprises a total data synchronous quantity, a total public network flow and a total record quantity; displaying the names of the latest successful tasks of the preset number, and displaying the names of the latest failed tasks of the preset number.
The invention has the following advantages:
the key of the invention is to provide a visual system access tool for multiple services and multiple systems, which realizes the real-time collection of different types of data in enterprises, establishes the connection relation of different system data, analyzes and sorts the data, ensures that the data of different systems in the enterprises have due value, realizes the data-driven enterprise production and decision making, and ensures that the enterprise operation and decision making are more scientific and reasonable.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to the structures shown in these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a schematic structural diagram of a multi-system multi-service visual system access tool according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, some embodiments of the present application will be described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the present application. However, those of ordinary skill in the art will understand that in the various embodiments of the present application, numerous technical details have been set forth in order to provide a better understanding of the present application. However, the technical solutions claimed in the present application can be implemented without these technical details and with various changes and modifications based on the following embodiments.
Example 1
Referring to fig. 1, embodiment 1 of the present invention provides a multi-system multi-service visualization system access tool, including:
the data source management module is used for managing data links, including newly-built data sources, test links, modification links, copy links, query links and deletion links;
it should be added that the data source is used for establishing a link with the database of the user, the data source comprises a data link, and for a system needing to perform data access, the data can be transmitted to the database of the user through the data link.
The data target template management module is used for managing the data target templates, and comprises the steps of creating a data target template, modifying the target template, rapidly establishing a data link in a templatization mode and deleting the data target template, wherein the data target template is a data relation mapping model between a system and a database;
it should be added that the data relationship mapping model may be set manually or a default scheme may be used, the default scheme being the complete transmission of data.
The data synchronization module is used for providing a synchronous task configuration scheme, and comprises a one-key real-time synchronization scheme, an incremental real-time synchronization scheme and an offline synchronization scheme;
it is to be added that, a plurality of synchronous task configuration schemes for providing services for different service scenes are recorded in the data synchronization module, and a synchronous solution for performing different data synchronization scenes among a plurality of data sources is provided.
The task management module is used for creating a data synchronization task and managing the data synchronization task, and comprises creating the task, starting the task, monitoring the task, editing the task, copying the task, resetting the task and deleting the task.
And the visualization module is used for carrying out visual display on the data synchronization task, for example, an access tool can be connected with a visual large screen of an enterprise, and related information of the data synchronization task can be visually displayed on the visual large screen.
In an alternative embodiment, the day of the configuration of the synchronization task is the T-th day, and for the data synchronization module, the method further includes:
for the one-key real-time synchronization scheme, firstly synchronizing the full-quantity data, then synchronizing the incremental data to the target end in real time, synchronizing the task configuration on the same day, writing the full-quantity data of the source table into a T-1 partition in the target table at one time, merging the incremental data of the source table with the full-quantity data, and writing the combined incremental data of the source table into the T-1 partition of the target table in real time; on the N th day of synchronous task configuration, the incremental data of the source table and the full data are combined and then written into the T+N-1 partition of the table in real time;
it is to be added that when the one-key real-time synchronization scheme synchronizes data, the full-volume data and the incremental data are operated separately, the target table is divided into partitions corresponding to different times, and the full-volume data and the incremental data in the source table are written into different partitions in the target table respectively.
For the incremental real-time synchronization scheme, only incremental data is synchronized to a target end;
it is to be added that the incremental real-time synchronization scheme only aims at the incremental data and performs synchronization processing on the incremental data.
Further, for offline synchronization schemes, including offline full-delta synchronization, periodic full-delta synchronization, disposable full-delta synchronization, periodic delta synchronization, and disposable delta synchronization, in particular:
for offline full-increment synchronization, synchronizing full-quantity data firstly, then synchronizing increment data to a target end, configuring a current day by a synchronization task, writing the full-quantity data of a source table into a target table T-1 partition at one time, configuring a t+N day by the synchronization task, and periodically writing the increment data of the source table into the target table T+N-1 partition;
it should be added that the offline full-increment synchronization includes synchronization processing of full-amount data and increment data, where the increment data is periodically synchronized, and a specific period may be reasonably set according to practical situations, for example, synchronization processing is performed on the increment data once every 1-3 days.
Periodically synchronizing the full data of the source table to the target end, configuring the T+1th day by the synchronous task, and periodically writing the full data of the source table into the T partition of the target table;
it should be noted that the periodic full-volume synchronization is a process of periodically synchronizing full-volume data.
The method comprises the steps of carrying out one-time full-volume synchronization, synchronizing source table full-volume data to a target end at one time, and writing the source table full-volume data into a target table T-1 partition at one time on the same day of synchronous task configuration;
it should be added that the one-time full-volume synchronization is to synchronize the full-volume data at one time.
Periodically performing incremental synchronization, namely periodically synchronizing incremental data of a source table to a target end, configuring the T+1th day by a synchronous task, and periodically writing the incremental data of the source table into a T partition of the target table;
it should be noted that the data synchronization object of the periodic incremental synchronization is incremental data, and is performed in a periodic synchronization manner.
And (3) performing one-time increment synchronization, namely synchronizing the source table increment data to the target end, wherein the task is configured on the current day T, and the source table increment data is written into the target table T-1 partition at one time.
It should be noted that the data synchronization object of the one-time incremental synchronization is incremental data, but the adopted synchronization mode is one-time synchronization, unlike the periodic incremental synchronization.
In an alternative embodiment, for the one-touch real-time synchronization scheme, further comprising:
the real-time synchronization of the data is realized through the data real-time reading unit, the data conversion unit and the data writing unit, and for a single one-key real-time synchronization scheme, the data read by the data real-time reading unit is supported to be cleaned together by the plurality of data conversion units, and the data write-in units are supported to output the data cleaned by the data conversion units to a target database of a target end;
the data real-time reading unit is used for acquiring the data of the system needing data synchronization in real time, the data conversion unit is used for performing data cleaning processing on the data acquired by the data real-time reading unit, the data writing unit is used for outputting the data cleaned by the data conversion unit to a target database of a target end, and in the actual data synchronization process, the number of the data conversion unit and the number of the data writing unit can be reasonably selected according to the actual situation in order to improve the synchronization speed.
Still further, the one-key real-time synchronization scheme includes real-time synchronization of single-table incremental data and real-time synchronization of whole-library incremental data, wherein:
the method includes the steps of synchronizing incremental data of a single table in real time, including data filtering, character string replacement and data desensitization processing on an input data source, and outputting the processed data to a target database of a target end;
it should be added that, the data filtering refers to performing rule filtering on the data, for example, the size of a filtering field, etc., and the data conforming to the rule is reserved; string replacement refers to replacing a field of a string type by a string replacement node; the data desensitization refers to the desensitization of the real-time synchronous single-table data, and the desensitized data is output to a target database.
For real-time synchronization of the incremental data of the whole library, writing the data into a homonym table in a target end by default, and if the homonym table does not exist, establishing a new homonym table by default;
it is to be added that when the whole library incremental data is synchronized in real time, a same name field mapping mode is adopted to write the source end field into the target end same name field, and under the condition that the same name field exists in the target end, the corresponding content in the source end field can be synchronized into the target end.
In an alternative embodiment, for an offline synchronization scheme, further comprising:
support the creation of offline synchronization tasks through configuration mode configuration tasks, through script mode configuration tasks, through OpenAPI configuration tasks, specifically:
for the configuration task through the configuration mode, providing guide type development guidance, and completing the configuration work of the data synchronization task through visual filling and guidance of the next step;
it is added that the configuration mode configuration task is suitable for application of primary developers and operators with low field code level, and can assist the operators to rapidly complete the configuration work of the data synchronization task and complete the creation of the offline synchronization task.
For configuration tasks through script modes, providing script type development guidance, and completing data synchronous development by directly writing a JSON script of data synchronization;
it is added that the script mode configuration task is suitable for a developer or an implementation personnel with a certain code foundation to be familiar with the related codes of the database, and the script mode can provide richer and flexible capability to realize fine configuration management.
For a configuration task through an OpenAPI, a data synchronization task is created through an OpenAPI call related interface, and the data of a source end is synchronized to a target end through shortcut.
In an alternative embodiment, further comprising:
the task rule management module is used for creating rules of data verification, data operation and data processing and managing the created rules;
specifically, the task rule management module includes:
the rule creation unit is used for customizing rules of data verification, data operation and data processing;
it is added that relevant rules including screening, filtering, preprocessing functions may be created.
A rule enabling and disabling unit for enabling and disabling the rule,
the rule editing unit is used for modifying the content of the rule, including modifying and storing information such as data distribution destination, data processing and the like under the rule, deleting the rule, enabling the rule in a state not to allow editing, and modifying and deleting the rule after the rule is disabled.
In an alternative implementation manner, the embodiment of the present invention provides a multi-system multi-service visualization system access tool, which further includes:
the data operation and maintenance module is used for checking the operation details of the data synchronization task, and monitoring the resources used for operating the data synchronization task, and comprises the steps of obtaining the overall operation condition of the data synchronization task and the change trend of resource scheduling during the operation of the data synchronization task.
Specifically, the data operation and maintenance module comprises a real-time synchronous task operation and maintenance unit and an offline synchronous task operation and maintenance unit, wherein:
for a real-time synchronous task operation and maintenance unit, displaying running state distribution images of real-time synchronous tasks in a selected date, displaying the data synchronous speed of the real-time synchronous tasks, counting the sum of the speeds, and displaying the names of the tasks with the highest delay in a preset number;
it should be added that the running state distribution image can be a pie chart, and an instance list of the corresponding state can be entered by clicking a certain color block in the pie chart; the names of the top 10 tasks of delay are shown in this embodiment in order to quickly locate the high-delay task.
For an offline synchronous task operation and maintenance unit, displaying an operation state distribution image of a real-time synchronous task in a selected date, and displaying a data synchronous quantity of the offline synchronous task in the selected date, wherein the data synchronous quantity comprises a total data synchronous quantity, a total public network flow and a total record quantity; displaying the names of the latest successful tasks of the preset number, and displaying the names of the latest failed tasks of the preset number.
It should be added that 10 latest failure examples and latest success examples are respectively displayed in the embodiment, so that a user can conveniently further know the actual running condition of the offline synchronous task.
It will be understood that modifications and variations will be apparent to those skilled in the art from the foregoing description, and it is intended that all such modifications and variations be included within the scope of the following claims. Parts of the specification not described in detail belong to the prior art known to those skilled in the art.

Claims (8)

1. A multi-system multi-service visualization system access tool, comprising:
the data source management module is used for managing data links, including newly-built data sources, test links, modification links, copy links, query links and deletion links;
the data target template management module is used for managing the data target templates, and comprises the steps of creating a data target template, modifying the target template, rapidly establishing a data link in a templatization mode and deleting the data target template, wherein the data target template is a data relation mapping model between a system and a database;
the data synchronization module is used for providing a synchronous task configuration scheme, and comprises a one-key real-time synchronization scheme, an incremental real-time synchronization scheme and an offline synchronization scheme;
the task management module is used for creating a data synchronization task and managing the data synchronization task, and comprises creating the task, starting the task, monitoring the task, editing the task, copying the task, resetting the task and deleting the task;
and the visualization module is used for carrying out visual display on the data synchronization task.
2. The multi-system multi-service visualization system access tool of claim 1 wherein for the data synchronization module, the synchronization task configuration day is the T-th day, further comprising:
for the one-key real-time synchronization scheme, firstly synchronizing the full-quantity data, then synchronizing the incremental data to the target end in real time, synchronizing the task configuration on the same day, writing the full-quantity data of the source table into a T-1 partition in the target table at one time, merging the incremental data of the source table with the full-quantity data, and writing the combined incremental data of the source table into the T-1 partition of the target table in real time; on the N th day of synchronous task configuration, the incremental data of the source table and the full data are combined and then written into the T+N-1 partition of the table in real time;
for the incremental real-time synchronization scheme, only incremental data is synchronized to the target.
3. The multi-system multi-service visualization system access tool of claim 2, further comprising, for the offline synchronization scheme:
the offline full-increment synchronization is carried out, the full-quantity data is firstly synchronized, then the incremental data is periodically synchronized to a target end, the full-quantity data of the source table is written into the target table T-1 partition once on the same day when the synchronization task is configured, and the incremental data of the source table is periodically written into the target table T+N-1 partition on the t+N day when the synchronization task is configured;
periodically synchronizing the full data of the source table to the target end, configuring the T+1th day by the synchronous task, and periodically writing the full data of the source table into the T partition of the target table;
the method comprises the steps of carrying out one-time full-volume synchronization, synchronizing source table full-volume data to a target end at one time, and writing the source table full-volume data into a target table T-1 partition at one time on the same day of synchronous task configuration;
periodically performing incremental synchronization, namely periodically synchronizing incremental data of a source table to a target end, configuring the T+1th day by a synchronous task, and periodically writing the incremental data of the source table into a T partition of the target table;
and (3) performing one-time increment synchronization, namely synchronizing the source table increment data to the target end, wherein the task is configured on the current day T, and the source table increment data is written into the target table T-1 partition at one time.
4. A multi-system multi-service visualization system access tool as recited in claim 3, further comprising, for the one-touch real-time synchronization scheme:
the real-time synchronization of the data is realized through the data real-time reading unit, the data conversion unit and the data writing unit, and for a single one-key real-time synchronization scheme, the data read by the data real-time reading unit is supported to be cleaned together by the plurality of data conversion units, and the data write-in units are supported to output the data cleaned by the data conversion units to a target database of a target end;
the one-key real-time synchronization scheme comprises real-time synchronization of single-table incremental data and real-time synchronization of whole-library incremental data;
the method includes the steps of synchronizing incremental data of a single table in real time, including data filtering, character string replacement and data desensitization processing on an input data source, and outputting the processed data to a target database of a target end;
and for real-time synchronization of the whole library incremental data, writing the data into a homonym table in the target end by default, and if the homonym table does not exist, establishing a new homonym table by default.
5. The multi-system multi-service visualization system access tool of claim 4, further comprising, for an offline synchronization scheme:
supporting the establishment of an offline synchronous task through a configuration mode configuration task, a script mode configuration task and an OpenAPI configuration task;
for the configuration task through the configuration mode, providing guide type development guidance, and completing the configuration work of the data synchronization task through visual filling and guidance of the next step;
for configuration tasks through script modes, providing script type development guidance, and completing data synchronous development by directly writing a JSON script of data synchronization;
for a configuration task through an OpenAPI, a data synchronization task is created by calling a related interface through the OpenAPI.
6. The multi-system multi-service visualization system access tool of claim 5, further comprising:
the task rule management module is used for creating rules of data verification, data operation and data processing and managing the created rules;
the task rule management module comprises:
the rule creation unit is used for customizing rules of data verification, data operation and data processing;
a rule enabling and disabling unit for enabling and disabling the rule,
the rule editing unit is used for modifying the content of the rule, including modifying and storing information such as data distribution destination, data processing and the like under the rule, deleting the rule, enabling the rule in a state not to allow editing, and modifying and deleting the rule after the rule is disabled.
7. The multi-system multi-service visualization system access tool of claim 6, further comprising:
the data operation and maintenance module is used for checking the operation details of the data synchronization task, and monitoring the resources used for operating the data synchronization task, and comprises the steps of acquiring the overall operation condition of the data synchronization task and the change trend of resource scheduling during the operation of the data synchronization task;
the data operation and maintenance module comprises a real-time synchronous task operation and maintenance unit and an offline synchronous task operation and maintenance unit.
8. The multi-system multi-service visualization system access tool of claim 7, further comprising:
for a real-time synchronous task operation and maintenance unit, displaying running state distribution images of real-time synchronous tasks in a selected date, displaying the data synchronous speed of the real-time synchronous tasks, counting the sum of the speeds, and displaying the names of the tasks with the highest delay in a preset number;
for an offline synchronous task operation and maintenance unit, displaying an operation state distribution image of a real-time synchronous task in a selected date, and displaying a data synchronous quantity of the offline synchronous task in the selected date, wherein the data synchronous quantity comprises a total data synchronous quantity, a total public network flow and a total record quantity; displaying the names of the latest successful tasks of the preset number, and displaying the names of the latest failed tasks of the preset number.
CN202211688534.9A 2022-12-27 2022-12-27 Multi-system multi-service visual system access tool Pending CN116186132A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202211688534.9A CN116186132A (en) 2022-12-27 2022-12-27 Multi-system multi-service visual system access tool

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202211688534.9A CN116186132A (en) 2022-12-27 2022-12-27 Multi-system multi-service visual system access tool

Publications (1)

Publication Number Publication Date
CN116186132A true CN116186132A (en) 2023-05-30

Family

ID=86448062

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202211688534.9A Pending CN116186132A (en) 2022-12-27 2022-12-27 Multi-system multi-service visual system access tool

Country Status (1)

Country Link
CN (1) CN116186132A (en)

Similar Documents

Publication Publication Date Title
US20030037114A1 (en) System, method and apparatus for updating electronic mail recipient lists
US6275223B1 (en) Interactive on line code inspection process and tool
CN113204598B (en) Data synchronization method, system and storage medium
CN111736994B (en) Resource arranging method, system, storage medium and electronic equipment
CN112148788A (en) Data synchronization method and system for heterogeneous data source
CN111881037A (en) Test case management method and device and electronic equipment
CN115712623A (en) Batch data fault-tolerant acquisition method based on capture metadata change
CN110503408B (en) Business process management system based on custom configuration
CN115222077A (en) Work order operation and maintenance statistical analysis system
CN114372105A (en) ETL tool based method for realizing system automatic inspection
JP5989194B1 (en) Test management system and program
CN112541746A (en) Full-stack automatic arrangement method and system
CN113537943A (en) Universal workflow engine and construction method thereof
CN116186132A (en) Multi-system multi-service visual system access tool
CN112364106A (en) Data association visualization method and system for automatically generating service topological graph
CN115033646B (en) Method for constructing real-time warehouse system based on Flink and Doris
US20070033080A1 (en) Method and apparatus for process discovery related applications
CN115599524A (en) Data lake system based on cooperative scheduling processing of streaming data and batch data
CN115237989A (en) Mine data acquisition system
US20210142237A1 (en) Management system
CN113209629B (en) Method and device for converting sequence frames into GIF
CN116627392B (en) Model development method and system based on interactive IDE
CN115543969B (en) Data migration method, device, equipment and medium
CN117453558A (en) Software test information processing method
CN117633061A (en) Multimedia display method and device for changing data of data center

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination