CN113934786B - Implementation method for constructing unified ETL - Google Patents
Implementation method for constructing unified ETL Download PDFInfo
- Publication number
- CN113934786B CN113934786B CN202111147981.9A CN202111147981A CN113934786B CN 113934786 B CN113934786 B CN 113934786B CN 202111147981 A CN202111147981 A CN 202111147981A CN 113934786 B CN113934786 B CN 113934786B
- Authority
- CN
- China
- Prior art keywords
- data
- etl
- page
- rules
- processing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/254—Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3684—Test management for test design, e.g. generating new test cases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3688—Test management for test execution, e.g. scheduling of test suites
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/252—Integrating or interfacing systems involving database management systems between a Database Management System and a front-end application
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/283—Multi-dimensional databases or data warehouses, e.g. MOLAP or ROLAP
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/20—Software design
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- General Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Computer Hardware Design (AREA)
- Quality & Reliability (AREA)
- Software Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses an implementation method for constructing unified ETL, which relates to the construction field of decision analysis systems, analyzes the data internal logic structure of a data source, organizes business rules according to business targets and processing rules of data processing, analyzes and summarizes data processing algorithms according to the business rules and the data source structure, designs data mapping, wherein the data processing source table and target table basic metadata information related to the initialization of basic mapping are consulted, corresponding data processing algorithms are selected, data rules are developed, the business rules are converted into executable code expressions according to data mapping contents and the processing rules, an ETL script to be executed is generated by utilizing a script generator according to the data mapping contents and the data processing algorithms, the script is tested, the processing rules are adjusted, and the construction of the ETL is completed.
Description
Technical Field
The invention discloses a method, relates to the field of construction of decision analysis systems, and particularly relates to an implementation method for constructing unified ETL.
Background
Along with the vigorous development of the big data age, the construction quality requirement of the decision analysis system is continuously improved, and the ETL system with high efficiency and high quality is directly related to the success or failure of the construction of the decision analysis system. The data demand index is flexible and changeable in the implementation process of the system, the business logic rule is complex, the data processing rule is fragmented and concrete, but the data developer lacks unified technical specifications and frequent personnel flow, so that the decision analysis system is low in efficiency due to the lack of quality control in ETL implementation, and the accuracy cannot be ensured.
Disclosure of Invention
Aiming at the problems in the prior art, the invention provides an implementation method for constructing the unified ETL, which can uniformly design and realize ETL data processing in the construction implementation of an OLAP system, saves the manpower development cost, improves the development quality, unifies the development specifications, and ensures that the data processing process is more standardized and more efficient and unified.
The specific scheme provided by the invention is as follows:
an implementation method for constructing unified ETL, analyzing the data internal logic structure of a data source,
the business rules are combed according to the business targets and the processing rules of the data processing, and the data processing algorithm is analyzed and summarized according to the business rules and the data source structure,
designing a data map, wherein the data processing source table and the target table base metadata information involved in the initialization of the base map are referenced, selecting a corresponding data processing algorithm,
developing data rules, wherein business rules are converted into executable code expressions according to data mapping content and processing rules,
the ETL script to be executed is generated with a script generator according to the data mapping content and the data processing algorithm,
testing the script and adjusting the processing rule to complete the construction of the ETL.
Further, the method for constructing the unified ETL includes, before the analyzing the data intrinsic logic structure of the data source:
the crawling data source is used for collecting only list page data of a webpage or collecting list page and page turning data or collecting list page, page turning and detail page data.
Further, in the implementation method for constructing the unified ETL, the analyzing the data internal logic structure of the data source includes:
determining whether the logical structures of the list page and the page turning data are consistent,
judging whether the repeated condition exists after the data are combined,
and analyzing the association relation between the carding list page and the detail page or between the page turning and the detail page.
Further, the carding business rule in the implementation method for constructing the unified ETL comprises the following steps:
the unique key of the data service is determined according to the unique requirements on the logic structure and service in the data of the source data,
the integration mechanism of the unified view of the data is completed,
and ensuring the logic relevance among the business data.
Further, in the implementation method for constructing the unified ETL, according to the test result of the test script, the processing rule is adjusted, and according to the adjusted processing rule, an updated ETL script is generated, and the ETL is optimized iteratively.
An implementation system for constructing unified ETL comprises an analysis and arrangement module and a generation module,
the analysis and sorting module analyzes the data internal logic structure of the data source,
the business rules are combed according to the business targets and the processing rules of the data processing, and the data processing algorithm is analyzed and summarized according to the business rules and the data source structure,
designing a data map, wherein the data processing source table and the target table base metadata information involved in the initialization of the base map are referenced, selecting a corresponding data processing algorithm,
developing data rules, wherein business rules are converted into executable code expressions according to data mapping content and processing rules,
the generation module generates ETL scripts to be executed by using a script generator according to the data mapping content and the data processing algorithm,
testing the script and adjusting the processing rule to complete the construction of the ETL.
Further, the implementation system for constructing the unified ETL further comprises a crawling module, wherein the crawling module crawls the data source before the analysis of the logic structure in the data of the data source, and the crawling data source is used for only collecting list page data of the webpage or collecting list page and page turning data or collecting list page, page turning and detail page data.
Further, the analysis and arrangement module in the implementation system for constructing the unified ETL analyzes the data internal logic structure of the data source, and the analysis and arrangement module comprises:
determining whether the logical structures of the list page and the page turning data are consistent,
judging whether the repeated condition exists after the data are combined,
and analyzing the association relation between the carding list page and the detail page or between the page turning and the detail page.
Further, the analysis and arrangement module in the implementation system for constructing the unified ETL combs the business rule, which comprises the following steps:
the unique key of the data service is determined according to the unique requirements on the logic structure and service in the data of the source data,
the integration mechanism of the unified view of the data is completed,
and ensuring the logic relevance among the business data.
Further, the generation module in the implementation system for constructing the unified ETL adjusts the processing rule according to the test result of the test script, generates an updated ETL script according to the adjusted processing rule, and iteratively optimizes the ETL.
The invention has the advantages that:
the invention provides an implementation method for constructing unified ETL, which ensures unified carding of business logic rules, unified design of data mapping, unified development of processing rules and unified generation of ETL script in the implementation process; ensuring unification of the ground steps from business requirements to implementation, quality inspection and development specifications, thereby completing establishment of a decision analysis system with high quality.
Drawings
FIG. 1 is a schematic flow chart of the method of the invention.
Detailed Description
The present invention will be further described with reference to the accompanying drawings and specific examples, which are not intended to be limiting, so that those skilled in the art will better understand the invention and practice it.
The invention provides an implementation method for constructing unified ETL, analyzing the data internal logic structure of a data source,
the business rules are combed according to the business targets and the processing rules of the data processing, and the data processing algorithm is analyzed and summarized according to the business rules and the data source structure,
designing a data map, wherein the data processing source table and the target table base metadata information involved in the initialization of the base map are referenced, selecting a corresponding data processing algorithm,
developing data rules, wherein business rules are converted into executable code expressions according to data mapping content and processing rules,
the ETL script to be executed is generated with a script generator according to the data mapping content and the data processing algorithm,
testing the script and adjusting the processing rule to complete the construction of the ETL.
In the construction implementation of the OLAP system, ETL data processing can be designed and realized uniformly, the development quality is improved while the manpower development cost is saved, the development specification is unified, and the data processing process is more standard, efficient and unified.
In specific applications, when the implementation of the unified ETL is constructed in some embodiments of the present invention, the specific procedures are as follows:
analyzing the data internal logic structure of the data source, wherein before analyzing the data internal logic structure of the data source, the data of the source is crawled by taking the data of the web crawler of the sky eye examination website of the enterprise data as an example, and the data mainly comprises three situations; firstly, the data only collect the list page data of the web page, secondly, the data collect the list page and page turning data, thirdly, the data collect the list page/page turning/detail page data,
in analyzing the data inherent logic structure of a data source, comprising:
judging whether the logic structures of the list page and the page turning data are consistent, if so, indicating that the two parts of data can logically do the collective UNION operation,
judging whether the repeated condition exists after the data are combined, properly deleting the repeated data,
analyzing the association relation between the carding list page and the detail page or between the page turning and the detail page, and carding and analyzing various relations among clear data;
the business rules are combed, taking the data of the sky eye checking enterprise as an example,
business uniqueness processing is carried out on the data, namely, the unique key of the data business is determined according to the internal logic structure of the data of the source data and the uniqueness requirement on the business, the duplication removing operation is carried out on the data,
the integration mechanism of the unified view of the data is completed, that is, each business module completes the integration of the unified view of the data, for example, list page data is to be integrated with detail page data,
ensuring the logic relevance among the business data, and if the business module data of the enterprise data need to be checked by the sky and the eyes, the main external key of the unified coding rule needs to be provided;
according to business rules and data source structure analysis summarization processing algorithm, taking the sky eye checking enterprise data as an example, the data processing algorithm summarization P0 can be that only list page- > according to business logic main KEY deduplication- > company name standardization- > society unified credit code/key_ID information supplement- > current date data are inserted into a target table; p1, having list pages and page turning, wherein the page turning has company name- > page turning and list page merging- > company name standardization- > social unified credit code/KEY_ID information supplement- > current date data are inserted into a target table according to business logic main KEY duplication removal-; p3, list page/page turning/detail page- > list page and page turning and detail page association- > merging- > standardized- > social unified credit code/KEY_ID information supplementary- > current date data are inserted into the target table according to the business logic main KEY;
designing data mapping, referring to the basic metadata information of the data processing source table and the target table related to the initialization of basic mapping, selecting a corresponding data processing algorithm,
taking the data of the eye-checking enterprise as an example to summarize and induce three data processing algorithms, the data processing algorithm selection information is also considered to be stored in the data mapping design process,
and mapping field correspondence of data processing, most of the field correspondence of data does not require special processing,
adding customized information such as sky and eye checking enterprise data according to business rules, performing duplication removal operation on the data according to the business unique key, and marking the unique key in the mapping;
developing a data rule, wherein the business rule is converted into an executable code expression according to data mapping content and processing rules, for example, a logic rule indicates that a plurality of field codes are needed to generate a new ID, then the data rule is developed, a specific SQL sentence is written as input of a script generator, and finally an executable ETL script is generated;
according to the data mapping content and the selection of the algorithm, the script generator is used as input to conveniently generate an ETL script to be executed, mapping information of the data, a data processing algorithm flow, a data duplication removal rule and the like are stored as knowledge base content, the script generator takes the knowledge base as input and then outputs the executable ETL script, the system can be repeatedly iterated and optimized, and algorithm induction, data mapping design and processing rule development and optimization are more conveniently generated to be in line with the actual program script of the service;
script test, adjusting processing rules, and continuously modifying the development content of the processing rules in the script test to finally obtain a standard, accurate and unified program to complete ETL construction.
The invention also provides an implementation system for constructing the unified ETL, which comprises an analysis and arrangement module and a generation module,
the analysis and sorting module analyzes the data internal logic structure of the data source,
the business rules are combed according to the business targets and the processing rules of the data processing, and the data processing algorithm is analyzed and summarized according to the business rules and the data source structure,
designing a data map, wherein the data processing source table and the target table base metadata information involved in the initialization of the base map are referenced, selecting a corresponding data processing algorithm,
developing data rules, wherein business rules are converted into executable code expressions according to data mapping content and processing rules,
the generation module generates ETL scripts to be executed by using a script generator according to the data mapping content and the data processing algorithm,
testing the script and adjusting the processing rule to complete the construction of the ETL.
The content of information interaction and execution process between the modules in the system is based on the same concept as the method embodiment of the present invention, and specific content can be referred to the description in the method embodiment of the present invention, which is not repeated here.
Similarly, the system can uniformly design and realize ETL data processing in the construction and implementation of the OLAP system, save the manpower development cost, improve the development quality, unify the development specifications, and ensure that the data processing process is more standardized, efficient and unified.
It should be noted that, in the above preferred embodiments, not all steps and modules in each flow and each system structure are necessary, and some steps or modules may be omitted according to actual needs. The execution sequence of the steps is not fixed and can be adjusted as required. The system structure described in the above embodiments may be a physical structure or a logical structure, that is, some modules may be implemented by the same physical entity, or some modules may be implemented by multiple physical entities, or may be implemented jointly by some components in multiple independent devices.
The above-described embodiments are merely preferred embodiments for fully explaining the present invention, and the scope of the present invention is not limited thereto. Equivalent substitutions and modifications will occur to those skilled in the art based on the present invention, and are intended to be within the scope of the present invention. The protection scope of the invention is subject to the claims.
Claims (2)
1. An implementation method for constructing unified ETL is characterized by crawling data sources, wherein the crawling data sources are used for only collecting list page data of webpages or collecting list page and page turning data or collecting list page, page turning and detail page data;
analyzing the data internal logic structure of the data source, comprising: judging whether the logic structures of the list page and the page turning data are consistent, judging whether repeated conditions exist after the data are combined, and analyzing the association relation between the list page and the detail page or between the page turning data and the detail page;
carding the business rules according to the business targets and the processing rules of data processing, comprising: determining a unique key of the data service according to the data internal logic structure of the source data and the uniqueness requirement on the service, completing an integration mechanism of a unified view of the data, and ensuring the logic relevance among the service data;
summarizing the data processing algorithm based on the business rules and the data source structure analysis,
designing a data map, wherein the data processing source table and the target table base metadata information involved in the initialization of the base map are referenced, selecting a corresponding data processing algorithm,
developing data rules, wherein business rules are converted into executable code expressions according to data mapping content and processing rules,
the ETL script to be executed is generated with a script generator according to the data mapping content and the data processing algorithm,
and testing the script and adjusting the processing rule to complete the construction of the ETL, wherein the processing rule is adjusted according to the test result of the test script, the updated ETL script is generated according to the adjusted processing rule, and the ETL is optimized iteratively.
2. An implementation system for constructing unified ETL is characterized by comprising a crawling module, an analysis and arrangement module and a generation module,
the crawling module crawls the data source, wherein the crawling data source only collects the list page data of the web page or collects the list page and page turning data or collects the list page, page turning data and detail page data,
the analysis and arrangement module analyzes the data internal logic structure of the data source, and comprises the following steps: judging whether the logic structures of the list page and the page turning data are consistent, judging whether repeated conditions exist after the data are combined, and analyzing the association relation between the list page and the detail page or between the page turning data and the detail page;
the analysis and arrangement module is used for combing the business rules according to the business targets and the processing rules of the data processing, and comprises the following steps: determining a unique key of the data service according to the data internal logic structure of the source data and the uniqueness requirement on the service, completing an integration mechanism of a unified view of the data, and ensuring the logic relevance among the service data;
summarizing the data processing algorithm based on the business rules and the data source structure analysis,
designing a data map, wherein the data processing source table and the target table base metadata information involved in the initialization of the base map are referenced, selecting a corresponding data processing algorithm,
developing data rules, wherein business rules are converted into executable code expressions according to data mapping content and processing rules,
the generation module generates ETL scripts to be executed by using a script generator according to the data mapping content and the data processing algorithm,
and testing the script and adjusting the processing rule to complete the construction of the ETL, wherein the generating module adjusts the processing rule according to the test result of the test script, generates an updated ETL script according to the adjusted processing rule, and iteratively optimizes the ETL.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111147981.9A CN113934786B (en) | 2021-09-29 | 2021-09-29 | Implementation method for constructing unified ETL |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111147981.9A CN113934786B (en) | 2021-09-29 | 2021-09-29 | Implementation method for constructing unified ETL |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113934786A CN113934786A (en) | 2022-01-14 |
| CN113934786B true CN113934786B (en) | 2023-09-08 |
Family
ID=79277360
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111147981.9A Active CN113934786B (en) | 2021-09-29 | 2021-09-29 | Implementation method for constructing unified ETL |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113934786B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115080653B (en) * | 2022-08-23 | 2022-11-11 | 北京华御数观科技有限公司 | Universal method for data processing |
| CN115858622B (en) * | 2022-12-12 | 2023-08-04 | 浙江大学 | Automatic generation method of business data checking script |
Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102915303A (en) * | 2011-08-01 | 2013-02-06 | 阿里巴巴集团控股有限公司 | Method and device for ETL (extract-transform-load) tests |
| CN104915341A (en) * | 2014-03-10 | 2015-09-16 | 中国科学院沈阳自动化研究所 | Visual multi-database ETL integration method and system |
| US9158827B1 (en) * | 2012-02-10 | 2015-10-13 | Analytix Data Services, L.L.C. | Enterprise grade metadata and data mapping management application |
| CN105359141A (en) * | 2013-05-17 | 2016-02-24 | 甲骨文国际公司 | Supporting combination of flow based ETL and entity relationship based ETL |
| CN107038177A (en) * | 2016-02-03 | 2017-08-11 | 维布络有限公司 | The method and apparatus for automatically generating extraction-conversion-loading code |
| CN108959564A (en) * | 2018-07-04 | 2018-12-07 | 玖富金科控股集团有限责任公司 | Data warehouse metadata management method, readable storage medium storing program for executing and computer equipment |
| CN109669983A (en) * | 2018-12-27 | 2019-04-23 | 杭州火树科技有限公司 | Visualize multi-data source ETL tool |
| CN110019551A (en) * | 2017-12-19 | 2019-07-16 | 阿里巴巴集团控股有限公司 | A kind of Building Method of Data Warehouse and device |
| CN111159266A (en) * | 2019-12-05 | 2020-05-15 | 江苏艾佳家居用品有限公司 | ETL task batch generation method based on metadata |
| CN111324647A (en) * | 2020-01-21 | 2020-06-23 | 北京东方金信科技有限公司 | Method and device for generating ETL code |
| CN111930819A (en) * | 2020-08-14 | 2020-11-13 | 工银科技有限公司 | ETL script generation method and device |
| CN112817971A (en) * | 2021-01-21 | 2021-05-18 | 于克干 | Data processing method and system based on two-dimensional mapping table |
| CN113051263A (en) * | 2019-12-26 | 2021-06-29 | 上海科技发展有限公司 | Metadata-based big data platform construction method, system, equipment and medium |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8510270B2 (en) * | 2010-07-27 | 2013-08-13 | Oracle International Corporation | MYSQL database heterogeneous log based replication |
-
2021
- 2021-09-29 CN CN202111147981.9A patent/CN113934786B/en active Active
Patent Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102915303A (en) * | 2011-08-01 | 2013-02-06 | 阿里巴巴集团控股有限公司 | Method and device for ETL (extract-transform-load) tests |
| US9158827B1 (en) * | 2012-02-10 | 2015-10-13 | Analytix Data Services, L.L.C. | Enterprise grade metadata and data mapping management application |
| CN105359141A (en) * | 2013-05-17 | 2016-02-24 | 甲骨文国际公司 | Supporting combination of flow based ETL and entity relationship based ETL |
| CN104915341A (en) * | 2014-03-10 | 2015-09-16 | 中国科学院沈阳自动化研究所 | Visual multi-database ETL integration method and system |
| CN107038177A (en) * | 2016-02-03 | 2017-08-11 | 维布络有限公司 | The method and apparatus for automatically generating extraction-conversion-loading code |
| CN110019551A (en) * | 2017-12-19 | 2019-07-16 | 阿里巴巴集团控股有限公司 | A kind of Building Method of Data Warehouse and device |
| CN108959564A (en) * | 2018-07-04 | 2018-12-07 | 玖富金科控股集团有限责任公司 | Data warehouse metadata management method, readable storage medium storing program for executing and computer equipment |
| CN109669983A (en) * | 2018-12-27 | 2019-04-23 | 杭州火树科技有限公司 | Visualize multi-data source ETL tool |
| CN111159266A (en) * | 2019-12-05 | 2020-05-15 | 江苏艾佳家居用品有限公司 | ETL task batch generation method based on metadata |
| CN113051263A (en) * | 2019-12-26 | 2021-06-29 | 上海科技发展有限公司 | Metadata-based big data platform construction method, system, equipment and medium |
| CN111324647A (en) * | 2020-01-21 | 2020-06-23 | 北京东方金信科技有限公司 | Method and device for generating ETL code |
| CN111930819A (en) * | 2020-08-14 | 2020-11-13 | 工银科技有限公司 | ETL script generation method and device |
| CN112817971A (en) * | 2021-01-21 | 2021-05-18 | 于克干 | Data processing method and system based on two-dimensional mapping table |
Non-Patent Citations (1)
| Title |
|---|
| 基于ETL的金融数据集成过程模型;苌程等;《计算机工程与设计》(第09期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113934786A (en) | 2022-01-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113934786B (en) | Implementation method for constructing unified ETL | |
| US9558230B2 (en) | Data quality assessment | |
| US9606980B2 (en) | Generating natural language text sentences as test cases for NLP annotators with combinatorial test design | |
| US9195952B2 (en) | Systems and methods for contextual mapping utilized in business process controls | |
| CN111258966A (en) | Data deduplication method, device, equipment and storage medium | |
| US9727319B1 (en) | Reducing compilation time using profile-directed feedback | |
| US9928288B2 (en) | Automatic modeling of column and pivot table layout tabular data | |
| US10628584B1 (en) | Functional language source code vulnerability scanner | |
| Wang et al. | Enhancing spatial and textual analysis with EUPEG: An extensible and unified platform for evaluating geoparsers | |
| CN110674360B (en) | Tracing method and system for data | |
| US11327720B2 (en) | Automated generation of software bindings | |
| Fan et al. | A differential testing approach for evaluating abstract syntax tree mapping algorithms | |
| CN112783867A (en) | Database optimization method for meeting real-time big data service requirements and cloud server | |
| EP4258107A1 (en) | Method and system for automated discovery of artificial intelligence and machine learning assets in an enterprise | |
| CN111400369A (en) | Big data analysis-based policy information service system and method | |
| Davis et al. | The supertree tool kit | |
| Tan et al. | Detecting outdated code element references in software repository documentation | |
| CN116257876A (en) | Data hierarchical access processing method, device, equipment and storage medium | |
| US20180336242A1 (en) | Apparatus and method for generating a multiple-event pattern query | |
| CN113672512A (en) | Code inspection rule generating method, code inspection method, device and medium | |
| Nguyen et al. | Using topic model to suggest fine-grained source code changes | |
| Yong et al. | Data change analysis based on function call path | |
| Odoni et al. | Introducing orbis: An extendable evaluation pipeline for named entity linking performance drill‐down analyses | |
| US20230342558A1 (en) | Systems and methods for generalized entity matching | |
| CN114254081B (en) | Enterprise big data search system, method and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |