CN111309790A - Automatic ETL configuration method and device based on maxcomputer database and control system - Google Patents
Automatic ETL configuration method and device based on maxcomputer database and control system Download PDFInfo
- Publication number
- CN111309790A CN111309790A CN201811432569.XA CN201811432569A CN111309790A CN 111309790 A CN111309790 A CN 111309790A CN 201811432569 A CN201811432569 A CN 201811432569A CN 111309790 A CN111309790 A CN 111309790A
- Authority
- CN
- China
- Prior art keywords
- database
- configuration information
- configuration
- automatically
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention is suitable for the technical field of computers, and provides an automatic ETL configuration method, a device and a control system based on a maxcomputer database, wherein the method comprises the following steps: acquiring configuration information of a user; automatically checking and configuring the source database and the target database based on the configuration information; and carrying out data synchronization based on the configuration information and the verification configuration result. In the invention, the source database and the target database are automatically checked and configured based on the configuration information of the user without repeated configuration, thereby simplifying the process of data extraction and improving the efficiency.
Description
Technical Field
The invention belongs to the technical field of computers, and particularly relates to an automatic ETL configuration method, an automatic ETL configuration device and an automatic ETL configuration control system based on a maxcomputer database.
Background
BI (business intelligence) extracts useful data from many data from different enterprise operating systems and cleans them to ensure the correctness of the data, then merges them into an enterprise-level data warehouse through Extraction (Extraction), Transformation (Transformation) and loading (Load), i.e. ETL (data warehouse technology) processes, so as to obtain a global view of the enterprise data, analyzes and processes it on the basis of this by using appropriate query and analysis tools, data mining tools, OLAP tools, etc., and finally presents knowledge to the manager to provide data support for the decision process of the manager.
In the prior art, the main characteristics of data extraction are large data volume and long processing time. The processing time period is mostly concentrated in the early morning with less database pressure, but the following problems exist: the running time of part of the drawing tasks is scheduled to the working time in the day due to the dependence on more tasks or error reporting and other reasons; the large data volume and long-time extraction is carried out on the database in working hours, and great influence is caused on the IO (read-write of database files, namely, retrieval data and insertion data) and the performance of the database; in addition, the extraction process of data requires each separate configuration script to complete synchronization, while some open source tools do not support some databases, such as: the keylet free sourcing tool does not support the maxcomputer database, so the extraction efficiency is low and the process is complicated.
Disclosure of Invention
The embodiment of the invention provides an automatic ETL configuration method, an automatic ETL configuration device and an automatic ETL configuration control system based on a maxcomputer database, and aims to solve the problem that in the prior art, the process is complex due to the fact that multiple configurations are required in the data extraction process.
An automatic ETL configuration method based on a maxcomputer database comprises the following steps:
acquiring configuration information of a user;
automatically checking and configuring the source database and the target database based on the configuration information;
and carrying out data synchronization based on the configuration information and the verification configuration result.
Preferably, automatically checking and configuring the source database and the target database based on the configuration information includes:
automatically detecting a source database and a target database based on the configuration information;
and checking and configuring the corresponding database based on the detection result.
Preferably, automatically detecting the source database and the target database based on the configuration information includes:
automatically detecting whether the source database is normal or not based on the configuration information, wherein the source database comprises a source data table;
automatically detecting whether a corresponding target database is normal or not based on the configuration information, wherein the target database comprises a target data table;
and detecting whether the source data table is consistent with the target data table or not based on the configuration information.
Preferably, after automatically detecting whether the source database is normal based on the configuration information, the method includes:
and when detecting that the source database is abnormal, sending an abnormal prompt.
Preferably, the automatically checking and configuring the source database and the target database based on the configuration information further includes:
detecting whether there is a field update based on the configuration information.
Preferably, the checking and configuring the corresponding database based on the detection result includes:
when detecting that the source database cannot be accessed, automatically updating the corresponding ETL synchronous switch; and/or
When field updating is detected, automatically adding the fields; and/or
And when detecting that the target database cannot be accessed, automatically updating the corresponding ETL synchronous switch.
Preferably, the data synchronization based on the configuration information and the verification configuration result includes:
generating a target data table of a target database based on the configuration information;
automatically generating corresponding field data for the target data table;
synchronizing data of the source database to a corresponding target database based on the generated field data. The invention also provides an automatic ETL configuration device based on the maxcomputer database, which comprises the following steps:
the acquisition unit is used for acquiring configuration information of a user;
the verification configuration unit is used for automatically verifying and configuring the source database and the target database based on the configuration information;
and the synchronization unit is used for carrying out data synchronization based on the configuration information and the verification configuration result.
The invention also provides a control system comprising an automatic ETL configuration device based on a maxcomputer database, wherein the device comprises:
the acquisition unit is used for acquiring configuration information of a user;
the verification configuration unit is used for automatically verifying and configuring the source database and the target database based on the configuration information;
and the synchronization unit is used for carrying out data synchronization based on the configuration information and the verification configuration result.
The invention also provides a memory storing a computer program executed by a processor to perform the steps of:
acquiring configuration information of a user;
automatically checking and configuring the source database and the target database based on the configuration information;
and carrying out data synchronization based on the configuration information and the verification configuration result.
The invention also provides a control terminal, which comprises a memory, a processor and a computer program which is stored in the memory and can run on the processor, wherein the processor executes the computer program to realize the following steps:
acquiring configuration information of a user;
automatically checking and configuring the source database and the target database based on the configuration information;
and carrying out data synchronization based on the configuration information and the verification configuration result.
In the embodiment of the invention, the source database and the target database are automatically checked and configured based on the configuration information of the user, and multiple configurations are not needed, so that the process of data extraction is simplified, and the efficiency is improved.
Drawings
Fig. 1 is a flowchart of an automatic ETL configuration method based on a maxcomputer database according to a first embodiment of the present invention;
fig. 2 is a detailed flowchart of step S2 of a method for automatic ETL configuration based on maxcomputer database according to a first embodiment of the present invention;
fig. 3 is a detailed flowchart of step S21 of a method for automatic ETL configuration based on maxcomputer database according to a first embodiment of the present invention;
fig. 4 is a block diagram of an automatic ETL configuring apparatus based on a maxcomputer database according to a second embodiment of the present invention;
fig. 5 is a structural diagram of a control terminal according to a third embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
In the embodiment of the invention, an automatic ETL configuration method based on a maxcomputer database comprises the following steps: acquiring configuration information of a user; automatically checking and configuring the source database and the target database based on the configuration information; and carrying out data synchronization based on the configuration information and the verification configuration result.
In order to explain the technical means of the present invention, the following description will be given by way of specific examples.
The first embodiment is as follows:
fig. 1 shows a flowchart of an automatic ETL configuration method based on a maxcomputer database according to a first embodiment of the present invention, where the method includes:
step S1, obtaining the configuration information of the user;
specifically, when a user needs to configure a source database and a target database that need to be synchronized, configuration information may be input manually or in other manners, such as bulk CSV file import. The configuration information includes related information of the source database and the target database, and the related information may include: the address of the source database, the current state information of the source database, etc., and the address of the target database, the current state information, etc. The configuration information may also include added fields, incremental tables, and the like.
Further, the configuration information further includes: the sync switch is configured to be preconfigured but temporarily not active, e.g. using field xtype to control whether or not to configure.
Further, the configuration information further includes: full or incremental synchronization is configured, for example, full synchronization is controlled by using a field isinc, incremental synchronization is controlled by using a field inc _ column, and when isinc is set to 1, the contents of inc _ column are sequentially set.
Further, the configuration information may further include: visualization is supported.
Step S2, automatically checking and configuring the source database and the target database based on the configuration information;
specifically, the source database and the target database are automatically checked and configured according to the configuration information, so that the source database and the target database can be normally accessed, and subsequent data synchronization is facilitated.
Step S3, data synchronization is carried out based on the configuration information and the verification configuration result;
specifically, data synchronization is performed according to the configuration information and the verification configuration result.
In this embodiment, the source database and the target database are automatically verified and configured based on the configuration information of the user, and multiple configurations are not required, so that the process of data extraction is simplified, and the efficiency is improved.
In a preferred embodiment of this embodiment, as shown in fig. 2, a specific flowchart of step S2 of an automatic ETL configuration method based on a maxcomputer database according to a first embodiment of the present invention is provided, where the step S2 specifically includes:
step S21, automatically detecting the source database and the target database based on the configuration information;
specifically, the source database and the target database are automatically detected based on the configuration information, so that the source database and the target database can be normally connected.
Step S22, checking and configuring the corresponding database based on the detection result;
specifically, the database that needs to be configured by checking is configured based on the foregoing detection result, for example, when the target database and/or the source database need to be configured by checking through detection, the configuration by checking is performed according to the detection result.
It should be noted that the detection result and the verification configuration result are both automatically written into the log database, so that the detection result and the verification configuration result can be obtained in time when the subsequent use is needed (for example, database maintenance is performed).
In a preferred embodiment of this embodiment, as shown in fig. 3, a specific flowchart of step S21 of an automatic ETL configuration method based on a maxcomputer database according to a first embodiment of the present invention is provided, where the step S21 specifically includes:
step S211, automatically detecting whether the source database is normal or not based on configuration information;
specifically, a background python script is called based on the configuration information to detect whether the database is normal, for example, whether the source database is configured correctly, whether the source database can be accessed in a normal connection mode, and the like; when the abnormity is detected, an abnormity prompt is sent out, the abnormity prompt can be sent out in a voice and mail mode, and the abnormity condition and the corresponding abnormity prompt are automatically written into a log database.
Step S212, whether the corresponding target database is normal or not is automatically detected based on the configuration information;
specifically, whether the target database is normal is detected based on the configuration information, for example, whether normal connection access can be performed, and when an abnormality occurs, an abnormality prompt is sent out, and the abnormality and the corresponding abnormality prompt are automatically written into the log database. In this embodiment, the exception prompt may be automatically sent in a voice or mail manner, or may be sent in another manner, which is not limited herein.
Step S213, detecting whether the source data table is consistent with the target data table based on the configuration information;
specifically, whether the source data table and the target data table are consistent or not is detected based on the configuration information.
In a preferable scheme of this embodiment, the step S21 further includes:
respectively detecting whether the fields of the source data table and the target data table are updated;
specifically, whether the configuration information has newly added fields of the target data table and the source data table is detected.
In a preferable scheme of this embodiment, the step S21 further includes:
detecting whether the configuration of the increment table in the configuration information is correct or not;
in this embodiment, the specific steps included in the step 21 are not strictly limited, and may include one or more than two of the specific steps, which is not limited herein.
When the steps S211 to S213 are included, the order of the steps S211 to S213 is not strictly limited, and three steps may be performed simultaneously, or two steps may be performed simultaneously, which is not limited herein.
In a preferable scheme of this embodiment, the step S22 specifically includes:
when detecting that the source database cannot be accessed, automatically updating the corresponding ELT synchronous switch;
specifically, when it is detected that a source data table of a source database cannot be accessed, a corresponding ETL synchronous switch is automatically added and a current ETL synchronous switch is updated;
in a preferable solution of this embodiment, the step S22 may further include:
when field updating is detected, automatically adding the updating field;
in a preferable solution of this embodiment, the step S22 may further include:
when the detection target database cannot be accessed, automatically adding a corresponding ETL synchronous switch and updating the current ETL synchronous switch;
in a preferable solution of this embodiment, the step S22 may further include:
and when the increment table is detected to be abnormal, updating the ETL synchronous switch.
In a preferable scheme of this embodiment, the step S3 specifically includes:
generating a target data table of a target database based on the configuration information;
automatically generating corresponding field data for the target data table;
specifically, corresponding field data is automatically generated for the target data table, and the field data can comprise field description, field type and the like;
synchronizing data of the source database to a corresponding target database based on the generated field data;
specifically, synchronizing data of the source database to the corresponding target database based on the generated field data; for example, a python script is used to invoke a datax tool to perform data multi-threaded synchronization, download data from a source database (either incrementally or fully), and then upload to a target database, preferably a maxcomputer database.
It should be noted that the downloaded data is also automatically written into the log database, so as to be conveniently obtained in time when the data needs to be used subsequently.
In this embodiment, the source database and the target database are automatically verified and configured based on the configuration information of the user, and multiple configurations are not required, so that the process of data extraction is simplified, and the efficiency is improved.
Secondly, the accuracy of configuration can be automatically detected, and the configuration efficiency is improved;
moreover, the detection, updating and operation data are automatically written into the log database for storage, so that the integrity of data storage can be provided, and convenience is brought to subsequent data tracking and maintenance.
Example two:
based on the first embodiment, as shown in fig. 4, a structure diagram of an automatic ETL configuration apparatus based on a maxcomputer database according to a second embodiment of the present invention is provided, where the apparatus includes: the system comprises an acquisition unit 1, a verification configuration unit 2 connected with the acquisition unit 1, and a synchronization unit 3 connected with the verification configuration unit 2, wherein:
an obtaining unit 1, configured to obtain configuration information of a user;
specifically, when a user needs to configure a source database and a target database that need to be synchronized, configuration information may be input manually or in other manners, such as bulk CSV file import. The configuration information includes related information of the source database and the target database, and the related information may include: the address of the source database, the current state information of the source database, etc., and the address of the target database, the current state information, etc. The configuration information may also include added fields, incremental tables, and the like.
Further, the configuration information further includes: the sync switch is configured to be preconfigured but temporarily not active, e.g. using field xtype to control whether or not to configure.
Further, the configuration information further includes: full or incremental synchronization is configured, for example, full synchronization is controlled by using a field isinc, incremental synchronization is controlled by using a field inc _ column, and when isinc is set to 1, the contents of inc _ column are sequentially set.
Further, the configuration information may further include: visualization is supported.
The verification configuration unit 2 is used for automatically verifying and configuring the source database and the target database based on the configuration information;
specifically, the source database and the target database are automatically checked and configured according to the configuration information, so that the source database and the target database can be normally accessed, and subsequent data synchronization is facilitated.
A synchronization unit 3, configured to perform data synchronization based on the configuration information and the verification configuration result;
specifically, data synchronization is performed according to the configuration information and the verification configuration result.
In this embodiment, the source database and the target database are automatically verified and configured based on the configuration information of the user, and multiple configurations are not required, so that the process of data extraction is simplified, and the efficiency is improved.
In a preferred embodiment of this embodiment, the verification configuration unit 2 specifically includes: detection subunit and check configuration subunit who connects with it, wherein:
the detection subunit is used for automatically detecting the source database and the target database based on the configuration information;
specifically, the source database and the target database are automatically detected based on the configuration information, so that the source database and the target database can be normally connected.
The verification configuration subunit is used for verifying and configuring the corresponding database based on the detection result;
specifically, the database that needs to be configured by checking is configured based on the foregoing detection result, for example, when the target database and/or the source database need to be configured by checking through detection, the configuration by checking is performed according to the detection result.
It should be noted that the detection result and the verification configuration result are both automatically written into the log database, so that the detection result and the verification configuration result can be obtained in time when the subsequent use is needed (for example, database maintenance is performed).
In a preferred embodiment of this embodiment, the detecting subunit is specifically configured to:
automatically detecting whether the source database is normal or not based on configuration information;
specifically, a background python script is called based on the configuration information to detect whether the database is normal, for example, whether the source database is configured correctly, whether the source database can be accessed in a normal connection mode, and the like; when the abnormity is detected, an abnormity prompt is sent out, the abnormity prompt can be sent out in a voice and mail mode, and the abnormity condition and the corresponding abnormity prompt are automatically written into a log database.
Automatically detecting whether the corresponding target database is normal or not based on the configuration information;
specifically, whether the target database is normal is detected based on the configuration information, for example, whether normal connection access can be performed, and when an abnormality occurs, an abnormality prompt is sent out, and the abnormality and the corresponding abnormality prompt are automatically written into the log database. In this embodiment, the exception prompt may be automatically sent in a voice or mail manner, or may be sent in another manner, which is not limited herein.
Detecting whether the source data table is consistent with the target data table based on the configuration information;
specifically, whether the source data table and the target data table are consistent or not is detected based on the configuration information.
In a preferred aspect of this embodiment, the detecting subunit is further configured to:
respectively detecting whether the fields of the source data table and the target data table are updated;
specifically, whether the configuration information has newly added fields of the target data table and the source data table is detected.
In a preferred aspect of this embodiment, the detecting subunit is further configured to:
detecting whether the configuration of the increment table in the configuration information is correct or not;
in a preferred embodiment of this embodiment, the verification configuration subunit is specifically configured to:
when detecting that the source database cannot be accessed, automatically updating the corresponding ELT synchronous switch;
specifically, when it is detected that a source data table of a source database cannot be accessed, a corresponding ETL synchronous switch is automatically added and a current ETL synchronous switch is updated;
in a preferred aspect of this embodiment, the verification configuration subunit is further configured to:
when field updating is detected, automatically adding the updating field;
in a preferred embodiment of this embodiment, the verification configuration subunit is specifically configured to:
when the detection target database cannot be accessed, automatically adding a corresponding ETL synchronous switch and updating the current ETL synchronous switch;
in a preferred embodiment of this embodiment, the verification configuration subunit is specifically configured to:
and when the increment table is detected to be abnormal, updating the ETL synchronous switch.
In a preferred embodiment of this embodiment, the synchronization unit 3 is specifically configured to:
generating a target data table of a target database based on the configuration information;
automatically generating corresponding field data for the target data table;
specifically, corresponding field data is automatically generated for the target data table, and the field data can comprise field description, field type and the like;
synchronizing data of the source database to a corresponding target database based on the generated field data;
specifically, synchronizing data of the source database to the corresponding target database based on the generated field data; for example, a python script is used to invoke a datax tool to perform data multi-threaded synchronization, download data from a source database (either incrementally or fully), and then upload to a target database, preferably a maxcomputer database.
It should be noted that the downloaded data is also automatically written into the log database, so as to be conveniently obtained in time when the data needs to be used subsequently.
In this embodiment, the source database and the target database are automatically verified and configured based on the configuration information of the user, and multiple configurations are not required, so that the process of data extraction is simplified, and the efficiency is improved.
Secondly, the accuracy of configuration can be automatically detected, and the configuration efficiency is improved;
moreover, the detection, updating and operation data are automatically written into the log database for storage, so that the integrity of data storage can be provided, and convenience is brought to subsequent data tracking and maintenance.
The invention further provides a control system, wherein the control system comprises the automatic ETL configuration device based on the maxcomputer database according to the second embodiment, and the specific structure, the working principle and the technical effect of the device are consistent with the description of the second embodiment, and are not repeated herein.
Example three:
fig. 5 shows a block diagram of a control terminal according to a third embodiment of the present invention, the control terminal including: a memory (memory)51, a processor (processor)52, a communication Interface (communication Interface)53 and a bus 54, wherein the processor 52, the memory 51 and the communication Interface 53 complete mutual communication through the bus 54.
A memory 51 for storing various data;
specifically, the memory 51 is used for storing various data, such as data in communication, received data, and the like, and is not limited herein, and the memory further includes a plurality of computer programs.
A communication interface 53 for information transmission between communication devices of the control terminal;
the processor 52 is configured to call various computer programs in the memory 51 to execute an automatic ETL configuration method based on the maxcomputer database provided in the first embodiment, for example:
acquiring configuration information of a user;
automatically checking and configuring the source database and the target database based on the configuration information;
and carrying out data synchronization based on the configuration information and the verification configuration result.
In the embodiment, the source database and the target database are automatically checked and configured based on the configuration information of the user, multiple configurations are not needed, the process of data extraction is simplified, and the efficiency is improved.
The invention also provides a memory, wherein the memory stores a plurality of computer programs, and the computer programs are called by a processor to execute the automatic FTL configuration method based on the maxcomputer database in the first embodiment.
In the invention, the source database and the target database are automatically checked and configured based on the configuration information of the user without repeated configuration, thereby simplifying the process of data extraction and improving the efficiency.
Secondly, the accuracy of configuration can be automatically detected, and the configuration efficiency is improved;
moreover, the detection, updating and operation data are automatically written into the log database for storage, so that the integrity of data storage can be provided, and convenience is brought to subsequent data tracking and maintenance.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation.
Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention. The above description is only for the specific embodiments of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present invention, and all the changes or substitutions should be covered within the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (11)
1. An automatic ETL configuration method based on a maxcomputer database is characterized by comprising the following steps:
acquiring configuration information of a user;
automatically checking and configuring the source database and the target database based on the configuration information;
and carrying out data synchronization based on the configuration information and the verification configuration result.
2. The automatic ETL configuration method of claim 1, wherein automatically verifying and configuring the source database and the target database based on the configuration information comprises:
automatically detecting a source database and a target database based on the configuration information;
and checking and configuring the corresponding database based on the detection result.
3. The automatic ETL configuration method of claim 2, wherein automatically detecting the source database and the target database based on the configuration information comprises:
automatically detecting whether the source database is normal or not based on the configuration information, wherein the source database comprises a source data table;
automatically detecting whether a corresponding target database is normal or not based on the configuration information, wherein the target database comprises a target data table;
and detecting whether the source data table is consistent with the target data table or not based on the configuration information.
4. The automatic ETL configuration method of claim 3, wherein automatically detecting whether the source database is normal based on the configuration information comprises:
and when detecting that the source database is abnormal, sending an abnormal prompt.
5. The automatic ETL configuration method of claim 4, wherein automatically verifying and configuring the source database and the target database based on the configuration information further comprises:
detecting whether there is a field update based on the configuration information.
6. The automatic ETL configuration method of claim 5, wherein verifying and configuring the corresponding database based on the detection result comprises:
when detecting that the source database cannot be accessed, automatically updating the corresponding ETL synchronous switch; and/or
When field updating is detected, automatically adding the fields; and/or
And when detecting that the target database cannot be accessed, automatically updating the corresponding ETL synchronous switch.
7. The automatic ETL configuration method of claim 6, wherein the data synchronization based on the configuration information and the verification configuration result comprises:
generating a target data table of a target database based on the configuration information;
automatically generating corresponding field data for the target data table;
synchronizing data of the source database to a corresponding target database based on the generated field data.
8. An automatic ETL configuration device based on a maxcomputer database, comprising:
the acquisition unit is used for acquiring configuration information of a user;
the verification configuration unit is used for automatically verifying and configuring the source database and the target database based on the configuration information;
and the synchronization unit is used for carrying out data synchronization based on the configuration information and the verification configuration result.
9. A control system comprising automatic ETL configuration means based on maxcomputer database according to claim 8.
10. A memory storing a computer program, the computer program being executable by a processor to perform the steps of:
acquiring configuration information of a user;
automatically checking and configuring the source database and the target database based on the configuration information;
and carrying out data synchronization based on the configuration information and the verification configuration result.
11. A control terminal comprising a memory, a processor and a computer program stored in said memory and executable on said processor, characterized in that said processor implements the steps of the maxcomputer database based automatic ETL configuration method according to any one of claims 1 to 7 when executing said computer program.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811432569.XA CN111309790A (en) | 2018-11-27 | 2018-11-27 | Automatic ETL configuration method and device based on maxcomputer database and control system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811432569.XA CN111309790A (en) | 2018-11-27 | 2018-11-27 | Automatic ETL configuration method and device based on maxcomputer database and control system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111309790A true CN111309790A (en) | 2020-06-19 |
Family
ID=71148514
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811432569.XA Pending CN111309790A (en) | 2018-11-27 | 2018-11-27 | Automatic ETL configuration method and device based on maxcomputer database and control system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111309790A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116361391A (en) * | 2023-03-30 | 2023-06-30 | 中电云数智科技有限公司 | Method and device for detecting and repairing structural abnormality of data synchronization table |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101706803A (en) * | 2009-11-27 | 2010-05-12 | 天津工业大学 | Method for migrating production data of weaving workshop |
| CN105005618A (en) * | 2015-07-21 | 2015-10-28 | 杭州合众数据技术有限公司 | Data synchronization method and system among heterogeneous databases |
| CN106302473A (en) * | 2016-08-18 | 2017-01-04 | 深圳市金证科技股份有限公司 | The duplication maintaining method of a kind of SQL Server data base and system |
| CN107423303A (en) * | 2016-05-24 | 2017-12-01 | 北京京东尚科信息技术有限公司 | The method and system of data syn-chronization |
| CN107888409A (en) * | 2017-10-23 | 2018-04-06 | 南京南瑞集团公司 | A kind of communication network configuration data automatic synchronous method with self-healing ability |
-
2018
- 2018-11-27 CN CN201811432569.XA patent/CN111309790A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101706803A (en) * | 2009-11-27 | 2010-05-12 | 天津工业大学 | Method for migrating production data of weaving workshop |
| CN105005618A (en) * | 2015-07-21 | 2015-10-28 | 杭州合众数据技术有限公司 | Data synchronization method and system among heterogeneous databases |
| CN107423303A (en) * | 2016-05-24 | 2017-12-01 | 北京京东尚科信息技术有限公司 | The method and system of data syn-chronization |
| CN106302473A (en) * | 2016-08-18 | 2017-01-04 | 深圳市金证科技股份有限公司 | The duplication maintaining method of a kind of SQL Server data base and system |
| CN107888409A (en) * | 2017-10-23 | 2018-04-06 | 南京南瑞集团公司 | A kind of communication network configuration data automatic synchronous method with self-healing ability |
Non-Patent Citations (1)
| Title |
|---|
| 苏亚涛: "《数字矿山建设理论与实践》", 合肥工业大学出版社, pages: 103 - 104 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116361391A (en) * | 2023-03-30 | 2023-06-30 | 中电云数智科技有限公司 | Method and device for detecting and repairing structural abnormality of data synchronization table |
| CN116361391B (en) * | 2023-03-30 | 2024-05-07 | 中电云计算技术有限公司 | Method and device for detecting and repairing structural abnormality of data synchronization table |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11163731B1 (en) | Autobuild log anomaly detection methods and systems | |
| US8538930B2 (en) | Method and system for database recovery | |
| CN104965754A (en) | Task scheduling method and task scheduling apparatus | |
| CN102736978A (en) | Method and device for detecting installation status of application program | |
| US8688795B2 (en) | GPS embedded interactive network interface | |
| CN112214411B (en) | Disaster recovery system testing method, device, equipment and storage medium | |
| US9304887B2 (en) | Method and system for operating system (OS) verification | |
| CN110278143B (en) | E-mail data processing method and device, computer equipment and storage medium | |
| CN110222119B (en) | Data conversion synchronization method, equipment and storage medium for heterogeneous database | |
| CN110543420B (en) | Software testing method, system, terminal and storage medium | |
| CN105512044A (en) | Method and system for updating object base used for keyword drive test | |
| US20170185505A1 (en) | Systems and methods for implementing an automated parallel deployment solution | |
| WO2017112127A1 (en) | Data flow analysis in processor trace logs using compiler-type information method and apparatus | |
| CN115454869A (en) | Interface automation test method, device, equipment and storage medium | |
| CN110673936A (en) | Breakpoint continuous operation method and device for arranging service, storage medium and electronic equipment | |
| CN111338834B (en) | Data storage method and device | |
| CN110555150A (en) | Data monitoring method, device, equipment and storage medium | |
| US9817731B2 (en) | Acquiring diagnostic data selectively | |
| US10001984B2 (en) | Identification of software updates using source code execution paths | |
| CN111309790A (en) | Automatic ETL configuration method and device based on maxcomputer database and control system | |
| CN109462661A (en) | Method of data synchronization, device, computer equipment and storage medium | |
| EP3639138B1 (en) | Action undo service based on cloud platform | |
| CN106204031A (en) | card application processing method and device | |
| CN116701053A (en) | Method, device, equipment and medium for restoring data backup of production environment database | |
| CN114253924A (en) | Synchronization method, synchronization equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |