CN110765196A - Method and equipment for generating and executing ETL task - Google Patents
Method and equipment for generating and executing ETL task Download PDFInfo
- Publication number
- CN110765196A CN110765196A CN201911023033.7A CN201911023033A CN110765196A CN 110765196 A CN110765196 A CN 110765196A CN 201911023033 A CN201911023033 A CN 201911023033A CN 110765196 A CN110765196 A CN 110765196A
- Authority
- CN
- China
- Prior art keywords
- etl
- data
- data processing
- metadata
- link
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/254—Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses
Abstract
The application relates to a method for generating and executing an ETL task, wherein a data processing process in an ETL requirement comprises a plurality of data processing links, the data processing process is converted into metadata, input data of each data processing link, output data and processing details of the current data processing link are described through the metadata. Generating an ETL task according to the metadata and executing the ETL task to obtain an ETL task execution result, wherein the ETL task execution result comprises: the input data quantity, the output data quantity and the data processing result of each data processing link. And determining whether each data processing link has errors according to the input data quantity and the output data quantity in the ETL task execution result and whether the data processing result succeeds or not, thereby improving the troubleshooting efficiency.
Description
Technical Field
The present application relates to the field of circuit control technologies, and in particular, to a method and an apparatus for generating and executing an ETL task.
Background
The ETL (Extract Transform Load) is a main method for establishing a data warehouse, a user extracts required data from a data source, and the required data is finally loaded into a target warehouse according to a rule defined in advance after cleaning and conversion, and the ETL is the basis for data analysis of an enterprise. When the ETL is developed, the complete ETL process needs to be known to configure, and if one link goes wrong in the configuration process, the whole ETL task link needs to be checked, which causes the problems of very high time cost and very low efficiency.
Disclosure of Invention
To overcome, at least in part, the problems in the related art, the present application provides a method and apparatus for generating and executing an ETL task.
The scheme of the application is as follows:
according to a first aspect of embodiments of the present application, there is provided a method for generating and executing an ETL task, including:
obtaining an ETL requirement, wherein the ETL requirement comprises: a data processing process comprising a plurality of data processing links;
converting the data processing process into metadata according to a preset rule, wherein the metadata is used for describing input data, output data and processing details of the current data processing link of each data processing link;
generating an ETL task according to the metadata, wherein the ETL task is configured to process data of a data source according to the data processing process;
executing the ETL task to obtain an ETL task execution result, wherein the ETL task execution result comprises: the input data quantity, the output data quantity and the data processing result of each data processing link are determined, and whether errors occur in each data processing link is determined according to the ETL task execution result.
Preferably, in an implementable manner of the present application, the converting the data processing procedure into metadata according to a preconfigured rule includes:
converting each data processing link of the data processing process into an ETL unit; the ETL unit comprises input data, output data and processing details of a corresponding data processing link;
arranging a plurality of ETL units in sequence to generate ETL process metadata; the sequence is the sequence of the data processing links corresponding to the ETL units in the data processing process.
Preferably, in an implementable manner of the present application, the converting the data processing procedure into metadata according to a preconfigured rule further includes:
and recording input data, output data and processing details included by the ETL unit in a table mode to generate ETL process detail metadata.
Preferably, in an implementable manner of the present application, the generating an ETL task according to the metadata includes:
processing the ETL process metadata; specifically, a unique identifier and an executed flow code are added to each ETL unit in the ETL process metadata;
inputting the processed ETL process metadata into an ETL metadata model to obtain an ETL configuration file, wherein the ETL configuration file is configured with the ETL task.
Preferably, in an implementable manner of the present application, the data processing procedure includes at least: a data extraction link, a code mapping link, a cleaning conversion link and a data loading link.
Preferably, in an implementable manner of the present application, the ETL process detail metadata includes at least: extracting the detail metadata from the data, mapping the detail metadata by codes, cleaning and converting the detail metadata and loading the data;
the data extraction detail metadata includes: extracting source description and rule;
the code mapping detail metadata includes: mapping mode, mapping field and mapping source field;
the cleansing conversion detail metadata includes: data cleaning conditions;
the data loading detail metadata includes: loading a pre-statement, loading a post-statement, loading a target, loading a mode and a target description.
Preferably, in an implementation manner of the present application, the input data is obtained from the data source or a previous link of a current data processing link, and the output data is obtained after being processed by the current data processing link.
Preferably, in an implementable manner of the present application, the output data comprises: output field, description and field type.
Preferably, in an implementation manner of the present application, the ETL requirement further includes: a data source, a data processing result pattern;
the method further comprises the following steps: and comparing the output data with the data processing result pattern to obtain the data processing result.
According to a second aspect of embodiments of the present application, there is provided an apparatus for generating and executing an ETL task, including:
a processor and a memory;
the processor and the memory are connected through a communication bus:
the processor is used for calling and executing the program stored in the memory;
the memory for storing a program for at least performing any of the methods of generating and executing an ETL task described above.
The technical scheme provided by the application can comprise the following beneficial effects:
because the data processing process in the ETL requirement comprises a plurality of data processing links, the data processing process is converted into metadata, the input data of each data processing link, the output data and the processing details of the current data processing link are described through the metadata. Generating an ETL task according to the metadata and executing the ETL task to obtain an ETL task execution result, wherein the ETL task execution result comprises: the input data quantity, the output data quantity and the data processing result of each data processing link. And determining whether each data processing link has errors according to the input data quantity and the output data quantity in the ETL task execution result and whether the data processing result succeeds or not, thereby improving the troubleshooting efficiency.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and together with the description, serve to explain the principles of the application.
FIG. 1 is a flow diagram of a method of generating and executing ETL tasks provided by one embodiment of the present application;
FIG. 2 is a flow diagram of converting a data processing procedure into metadata in a method of generating and executing an ETL task according to an embodiment of the present application;
fig. 3 is a block diagram of an apparatus for generating and executing an ETL task according to an embodiment of the present application.
Detailed Description
Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.
Fig. 1 is a flowchart of a method for generating and executing an ETL task according to an embodiment of the present application, and referring to fig. 1, a method for generating and executing an ETL task includes:
s11: obtaining an ETL requirement, wherein the ETL requirement comprises: a data processing process, wherein the data processing process comprises a plurality of data processing links;
the ETL is a main method for establishing a data warehouse, a user extracts required data from a data source, and the required data is loaded into a target warehouse according to a rule defined in advance after cleaning and conversion, and the ETL is a basis for data analysis of an enterprise.
The data processing process at least comprises the following steps: a data extraction link, a code mapping link, a cleaning conversion link and a data loading link.
And a data extraction link: the specified data is extracted from the specified data source.
And a code mapping link: and establishing a mapping relation table, and describing the association relation between the two tables, such as the association of the C field in the A table with the C field in the B table.
A cleaning conversion link: and filtering out required data according to certain conditions. In the data cleaning and converting link, due to the problem of a data production system, a plurality of repeated data can be generated, and the repeated data needs to be removed in the cleaning process.
And a data loading link: and loading the data after cleaning conversion into a storage table.
S12: converting the data processing process into metadata according to a preset rule, wherein the metadata is used for describing input data, output data and processing details of the current data processing link;
the metadata in the present embodiment includes two types, one is ETL process metadata, and one is ETL process detail metadata.
Referring to fig. 2, converting a data processing procedure into metadata according to a preconfigured rule includes:
s121: converting each data processing link in the data processing process into an ETL unit; the ETL unit comprises input data, output data and processing details of a corresponding data processing link;
s122: arranging a plurality of ETL units in sequence to generate ETL process metadata; the sequence is the sequence of the data processing links corresponding to the ETL units in the data processing process.
As shown in the following table, the ETL process metadata includes two parts: ETL unit and data processing procedure sequence.
Converting the data processing procedure into metadata according to the preconfigured rule, further comprising:
s123: and recording input data, output data and processing details included by the ETL unit in a table mode to generate ETL process detail metadata.
The ETL process detail metadata includes at least: extracting the detail metadata from the data, mapping the detail metadata by codes, cleaning and converting the detail metadata and loading the data;
the data extraction detail metadata includes: extracting source description and rule;
the code mapping detail metadata includes: mapping mode, mapping field and mapping source field;
the cleansing conversion detail metadata includes: data cleaning conditions;
the data loading detail metadata includes: loading a pre-statement, loading a post-statement, loading a target, loading a mode and a target description.
The input data is obtained from a data source or the previous link of the current data processing link, and the output data is obtained after the current data processing link.
The input data of the data extraction link is obtained from a data source, the input data of the code mapping link is obtained from the data extraction link, the input data of the cleaning conversion link is obtained from the code mapping link, and the input data of the data loading link is obtained from the cleaning conversion link.
And each data processing link processes the input data to obtain output data.
The output data includes: output field, description and field type.
S13: generating an ETL task according to the metadata, wherein the ETL task is configured to process data of the data source according to a data processing process;
generating an ETL job from the metadata, comprising:
processing the ETL process metadata; specifically, a unique identifier and an executed flow code are added to each ETL unit in the metadata of the ETL process;
inputting the processed ETL process metadata into an ETL metadata model to obtain an ETL configuration file, wherein the ETL configuration file is configured with an ETL task.
Adding a unique identifier and an executed flow code for each ETL unit in the ETL process metadata, refer to the following table:
the ETL name is the unique identification of each ETL unit, the ETL metadata code is the flow code executed by the ETL, and the sequence of the ETL unit and the data processing process is the ETL process metadata.
Inputting the processed ETL process metadata into an ETL metadata model to generate ETL configuration information. And generating an ETL configuration file according to the ETL configuration information through a special driver.
The configuration file is in xml or yml format.
And after the generated configuration file is compiled by C + +, an executable ETL task is generated.
S14: executing the ETL task to obtain an ETL task execution result, wherein the ETL task execution result comprises: the input data quantity, the output data quantity and the data processing result of each data processing link are determined, and whether errors occur in each data processing link is determined according to the ETL task execution result.
The metadata comprises input data and output data of each link, so that the input data quantity and the output data quantity of each data processing link can be obtained.
The ETL requirements also include: a data source, a data processing result pattern;
the method further comprises the following steps: and comparing the output data with the data processing result pattern to obtain a data processing result.
The input data quantity and the output data quantity of each data processing link have preset requirements, if the data processing links normally complete data processing tasks, the input data quantity and the output data quantity meet the preset requirements, if a certain data processing link fails to normally complete the data processing tasks, the output data quantity of the data processing link cannot meet the preset requirements, errors of the data processing link can be directly determined, and correction processing can be immediately carried out.
In the method for generating and executing an ETL task in this embodiment, a data processing process in an ETL requirement includes multiple data processing links, the data processing process is converted into metadata, input data of each data processing link is described by the metadata, and output data and processing details of a current data processing link are output. Generating an ETL task according to the metadata and executing the ETL task to obtain an ETL task execution result, wherein the ETL task execution result comprises: the input data quantity, the output data quantity and the data processing result of each data processing link. And determining whether each data processing link has errors according to the input data quantity and the output data quantity in the ETL task execution result and whether the data processing result succeeds or not, thereby improving the troubleshooting efficiency.
The ETL requirement is exemplified by vehicle dimension completion.
The data sources include the following vehicle record table and device table.
Vehicle RECORD table (T _ RECORD):
| column name | Description of the invention | Type of field |
| RECORD_ID | Record number | String |
| DEVICE_ID | Device numbering | String |
| RECORD_TIME | Recording time | bigint |
| PLATE_NUM | License plate | String |
| COLOR | Colour(s) | String |
The vehicle RECORD table is a Chinese description, and T _ RECORD is the name of the vehicle RECORD table, and in the database, the table name is generally English, and the same applies below.
DEVICE table (T _ DEVICE):
the data processing procedure is as follows:
the result after processing is a multi-bin RECORD table (DW _ RECORD):
| column name | Description of the invention | Type of field |
| RECORD_ID | Record number | String |
| DEVICE_ID | Device numbering | String |
| RECORD_TIME | Recording time | bigint |
| PLATE_NUM | License plate | String |
| COLOR | Colour(s) | String |
| DEVICE_NAME | Device name | String |
| LONGITUDE | Device longitude | double |
| LATITUDE | Device dimension | double |
| ALITUDE | Altitude of the equipment | double |
Digital storage table (DW _ RECORD)
All fields in the device table may be mapped to the database, and in this embodiment, the device installation date field is deleted as an exclusion item, and no mapping is performed.
In the data processing process, data are extracted from a data source, a vehicle RECORD table (T _ RECORD) is mapped to a DEVICE table (T _ DEVICE), DEVICEs corresponding to vehicle RECORDs are determined, data cleaning conversion is carried out, and all vehicle RECORDs after cleaning conversion are loaded into a multi-bin RECORD table (DW _ RECORD). According to this data processing procedure, the data processing procedure is divided into 4 ETL units: data extraction, code mapping, cleaning conversion and data loading.
Process metadata is formed from the ETL units.
| ETL unit | Data processing procedure sequence |
| Data extraction | 1 |
| Code mapping | 2 |
| Cleaning rotorChangeable pipe | 3 |
| Data loading | 4 |
And recording input data, output data and processing details included by the ETL unit in a table mode to generate ETL process detail metadata.
The data extraction detail metadata is as follows:
the output data of the data extraction link is as follows:
the code mapping detail metadata is as follows:
| mapping mode | Mapping fields | Mapping source fields |
| Leftjoin | DEVICE_ID | DEVICE_ID |
The code mapping detail metadata includes a mapping method, a mapping field, and a mapping source field.
The output data of the code mapping link is as follows:
| output field | Type of field |
| RECORD_ID | String |
| DEVICE_ID | String |
| RECORD_TIME | bigint |
| PLATE_NUM | String |
| COLOR | String |
| DEVICE_NAME | String |
| LONGITUDE | double |
| LATITUDE | double |
| ALITUDE | double |
The cleansing translation detail metadata is as follows:
| cleaning conditions |
| PLATE_NUM<>”AND ENABLE＝1 |
PLATE _ NUM < > "AND ENABLE ═ 1 indicates that the license PLATE number is not empty AND the device is valid.
The output data of the cleaning conversion link is as follows:
data load detail metadata is as follows:
the data loading mode comprises a statement before loading, a statement after loading, a loading target, a loading mode and a target description.
Fig. 3 is a block diagram of an apparatus for generating and executing an ETL task according to an embodiment of the present application, and referring to fig. 3, an apparatus for generating and executing an ETL task includes:
a processor 21 and a memory 22;
the processor 21 is connected to the memory 22 by a communication bus:
the processor 21 is configured to call and execute a program stored in the memory 22;
a memory 22 for storing a program for use at least in the method of generating and executing an ETL task in any of the above embodiments.
It is understood that the same or similar parts in the above embodiments may be mutually referred to, and the same or similar parts in other embodiments may be referred to for the content which is not described in detail in some embodiments.
It should be noted that, in the description of the present application, the terms "first", "second", etc. are used for descriptive purposes only and are not to be construed as indicating or implying relative importance. Further, in the description of the present application, the meaning of "a plurality" means at least two unless otherwise specified.
Any process or method descriptions in flow charts or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or steps of the process, and the scope of the preferred embodiments of the present application includes other implementations in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the present application.
It should be understood that portions of the present application may be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, the various steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any one or combination of the following techniques, which are known in the art, may be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application specific integrated circuit having an appropriate combinational logic gate circuit, a Programmable Gate Array (PGA), a Field Programmable Gate Array (FPGA), or the like.
It will be understood by those skilled in the art that all or part of the steps carried by the method for implementing the above embodiments may be implemented by hardware related to instructions of a program, which may be stored in a computer readable storage medium, and when the program is executed, the program includes one or a combination of the steps of the method embodiments.
In addition, functional units in the embodiments of the present application may be integrated into one processing module, or each unit may exist alone physically, or two or more units are integrated into one module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode. The integrated module, if implemented in the form of a software functional module and sold or used as a stand-alone product, may also be stored in a computer readable storage medium.
The storage medium mentioned above may be a read-only memory, a magnetic or optical disk, etc.
In the description herein, reference to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the application. In this specification, the schematic representations of the terms used above do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
Although embodiments of the present application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present application, and that variations, modifications, substitutions and alterations may be made to the above embodiments by those of ordinary skill in the art within the scope of the present application.
Claims (10)
1. A method of generating and executing an ETL task, comprising:
obtaining an ETL requirement, wherein the ETL requirement comprises: a data processing process comprising a plurality of data processing links;
converting the data processing process into metadata according to a preset rule, wherein the metadata is used for describing input data, output data and processing details of the current data processing link of each data processing link;
generating an ETL task according to the metadata, wherein the ETL task is configured to process data of a data source according to the data processing process;
executing the ETL task to obtain an ETL task execution result, wherein the ETL task execution result comprises: the input data quantity, the output data quantity and the data processing result of each data processing link are determined, and whether errors occur in each data processing link is determined according to the ETL task execution result.
2. The method of claim 1, wherein converting the data processing procedure into metadata according to preconfigured rules comprises:
converting each data processing link of the data processing process into an ETL unit; the ETL unit comprises input data, output data and processing details of a corresponding data processing link;
arranging a plurality of ETL units in sequence to generate ETL process metadata; the sequence is the sequence of the data processing links corresponding to the ETL units in the data processing process.
3. The method of claim 2, wherein converting the data processing procedure into metadata according to preconfigured rules further comprises:
and recording input data, output data and processing details included by the ETL unit in a table mode to generate ETL process detail metadata.
4. The method of claim 2, wherein generating an ETL task from the metadata comprises:
processing the ETL process metadata; specifically, a unique identifier and an executed flow code are added to each ETL unit in the ETL process metadata;
inputting the processed ETL process metadata into an ETL metadata model to obtain an ETL configuration file, wherein the ETL configuration file is configured with the ETL task.
5. The method according to claim 3, wherein the data processing procedure comprises at least: a data extraction link, a code mapping link, a cleaning conversion link and a data loading link.
6. The method of claim 5, wherein the ETL process detail metadata comprises at least: extracting the detail metadata from the data, mapping the detail metadata by codes, cleaning and converting the detail metadata and loading the data;
the data extraction detail metadata includes: extracting source description and rule;
the code mapping detail metadata includes: mapping mode, mapping field and mapping source field;
the cleansing conversion detail metadata includes: data cleaning conditions;
the data loading detail metadata includes: loading a pre-statement, loading a post-statement, loading a target, loading a mode and a target description.
7. The method of claim 2, wherein the input data is obtained from the data source or a previous stage of a current data processing stage, and the output data is obtained after processing of the current data processing stage.
8. The method of claim 2, wherein outputting the data comprises: output field, description and field type.
9. The method of claim 1, wherein the ETL requirement further comprises: a data source, a data processing result pattern;
the method further comprises the following steps: and comparing the output data with the data processing result pattern to obtain the data processing result.
10. An apparatus for generating and executing an ETL task, comprising:
a processor and a memory;
the processor and the memory are connected through a communication bus:
the processor is used for calling and executing the program stored in the memory;
the memory for storing a program for at least performing the method of generating and executing an ETL task according to any of claims 1-9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911023033.7A CN110765196A (en) | 2019-10-25 | 2019-10-25 | Method and equipment for generating and executing ETL task |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911023033.7A CN110765196A (en) | 2019-10-25 | 2019-10-25 | Method and equipment for generating and executing ETL task |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110765196A true CN110765196A (en) | 2020-02-07 |
Family
ID=69333772
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911023033.7A Pending CN110765196A (en) | 2019-10-25 | 2019-10-25 | Method and equipment for generating and executing ETL task |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110765196A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111881154A (en) * | 2020-07-29 | 2020-11-03 | 北京浪潮数据技术有限公司 | ETL task processing method, device and related equipment |
| CN114880385A (en) * | 2021-07-27 | 2022-08-09 | 云南省地质环境监测院(云南省环境地质研究院) | Method and device for accessing geological disaster data through automatic combined flow |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103455526A (en) * | 2012-06-05 | 2013-12-18 | 杭州勒卡斯广告策划有限公司 | ETL (extract-transform-load) data processing method, device and system |
| CN104778236A (en) * | 2015-04-02 | 2015-07-15 | 上海烟草集团有限责任公司 | ETL (Extract-Transform-Load) realization method and system based on metadata |
| US20170220654A1 (en) * | 2016-02-03 | 2017-08-03 | Wipro Limited | Method for automatically generating extract transform load (etl) codes using a code generation device |
| CN109739893A (en) * | 2018-12-28 | 2019-05-10 | 上海连尚网络科技有限公司 | A kind of metadata management method, equipment and computer-readable medium |
-
2019
- 2019-10-25 CN CN201911023033.7A patent/CN110765196A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103455526A (en) * | 2012-06-05 | 2013-12-18 | 杭州勒卡斯广告策划有限公司 | ETL (extract-transform-load) data processing method, device and system |
| CN104778236A (en) * | 2015-04-02 | 2015-07-15 | 上海烟草集团有限责任公司 | ETL (Extract-Transform-Load) realization method and system based on metadata |
| US20170220654A1 (en) * | 2016-02-03 | 2017-08-03 | Wipro Limited | Method for automatically generating extract transform load (etl) codes using a code generation device |
| CN109739893A (en) * | 2018-12-28 | 2019-05-10 | 上海连尚网络科技有限公司 | A kind of metadata management method, equipment and computer-readable medium |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111881154A (en) * | 2020-07-29 | 2020-11-03 | 北京浪潮数据技术有限公司 | ETL task processing method, device and related equipment |
| CN114880385A (en) * | 2021-07-27 | 2022-08-09 | 云南省地质环境监测院(云南省环境地质研究院) | Method and device for accessing geological disaster data through automatic combined flow |
| CN114880385B (en) * | 2021-07-27 | 2022-11-22 | 云南省地质环境监测院(云南省环境地质研究院) | Method and device for accessing geological disaster data through automatic combination process |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107958057B (en) | Code generation method and device for data migration in heterogeneous database | |
| CN107807982B (en) | Consistency checking method and device for heterogeneous database | |
| US8615526B2 (en) | Markup language based query and file generation | |
| CN110781231B (en) | Database-based batch import method, device, equipment and storage medium | |
| US11481440B2 (en) | System and method for processing metadata to determine an object sequence | |
| CN108334609B (en) | Method, device, equipment and storage medium for realizing JSON format data access in Oracle | |
| CN106557307B (en) | Service data processing method and system | |
| CN110765196A (en) | Method and equipment for generating and executing ETL task | |
| CN112685117A (en) | System language internationalization maintenance method, device and computer readable storage medium | |
| US10776313B2 (en) | Converting source objects to target objects | |
| US20110264703A1 (en) | Importing Tree Structure | |
| CN111124892B (en) | Automatic testing method and system based on command line | |
| CN112732763A (en) | Data aggregation method and device, electronic equipment and medium | |
| WO2018066661A1 (en) | Log analysis method, system, and recording medium | |
| CN111581183A (en) | Data migration method and device based on data model | |
| CN107168866B (en) | Parameter analysis method and device for configuration file | |
| CN106383914B (en) | Method and system for realizing configuration of multiple data sources based on cloud call platform | |
| CN112818129B (en) | Atlas modeling system and method | |
| CN111666278A (en) | Data storage method, data retrieval method, electronic device and storage medium | |
| US20160335335A1 (en) | Intermediate data format for database population | |
| CN111142965A (en) | Language configuration method and device, electronic equipment and storage medium | |
| CN110309211B (en) | Method for positioning ETL process problem and related equipment | |
| CN111352824B (en) | Test method and device and computer equipment | |
| CN109284278B (en) | Calculation logic migration method based on data analysis technology and terminal equipment | |
| CN113377672B (en) | Test method, device and equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200207 |