CN110413669A - A kind of data processing method based on ETL, device, terminal device and storage medium - Google Patents
A kind of data processing method based on ETL, device, terminal device and storage medium Download PDFInfo
- Publication number
- CN110413669A CN110413669A CN201910542628.7A CN201910542628A CN110413669A CN 110413669 A CN110413669 A CN 110413669A CN 201910542628 A CN201910542628 A CN 201910542628A CN 110413669 A CN110413669 A CN 110413669A
- Authority
- CN
- China
- Prior art keywords
- data
- etl
- source
- configuration parameter
- change data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003860 storage Methods 0.000 title claims abstract description 22
- 238000003672 processing method Methods 0.000 title claims abstract description 19
- 230000008859 change Effects 0.000 claims abstract description 86
- 238000006243 chemical reaction Methods 0.000 claims abstract description 46
- 238000000034 method Methods 0.000 claims description 19
- 238000013506 data mapping Methods 0.000 claims description 18
- 238000012545 processing Methods 0.000 claims description 13
- 238000013507 mapping Methods 0.000 claims description 7
- 238000000605 extraction Methods 0.000 claims description 6
- 230000006870 function Effects 0.000 claims description 5
- 230000008569 process Effects 0.000 description 4
- 241000208340 Araliaceae Species 0.000 description 2
- 235000005035 Panax pseudoginseng ssp. pseudoginseng Nutrition 0.000 description 2
- 235000003140 Panax quinquefolius Nutrition 0.000 description 2
- 238000013523 data management Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 239000000284 extract Substances 0.000 description 2
- 235000008434 ginseng Nutrition 0.000 description 2
- 238000007726 management method Methods 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 210000004556 brain Anatomy 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000005611 electricity Effects 0.000 description 1
- 230000014759 maintenance of location Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000001737 promoting effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/254—Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a kind of data processing method based on ETL, device, terminal device and storage mediums, wherein should be applied to terminal device based on the data processing method of ETL, which comprises obtain the configuration parameter of source database；Source data is extracted from corresponding source database according to the configuration parameter；Obtain the ETL conversion parameter of the source data；The source data is converted to obtain change data according to the ETL conversion parameter；The output configuration parameter of the change data is obtained, the output configuration parameter includes exporting first address information in warehouse；The change data is exported to the output warehouse according to first address information.Data processing method provided by the present invention based on ETL can be realized the automatic management of change data, reduce artificial intervention, improve change data treatment effeciency.
Description
Technical field
The present invention relates to data warehouse technology field more particularly to a kind of data processing methods based on ETL, device, end
End equipment and storage medium.
Background technique
ETL (Extract-Transform-Load is extracted, conversion, loaded) is a kind of data warehouse technology, and user passes through
ETL task can extract required data from data source, by data cleansing, the mutually processes such as conversion, load data into
Destination.
Currently, handling mostly by the way of user intervention for ETL source data, i.e. user are at configuration change data
While managing logic rules, it is necessary to the input data format and output data format of each step ETL operation be manually specified.It is this
The intelligence for the source data management method explicitly intervened based on user is poor, operates relatively complicated, is unfavorable for promoting and applying.
Summary of the invention
The main purpose of the present invention is to provide a kind of data processing method based on ETL, device, terminal device and storages
Medium, it is intended to realize the automation of source data management, reduce user intervention.
To achieve the above object, the present invention provides a kind of data processing method based on ETL, is applied to terminal device, institute
The method of stating includes:
Obtain the configuration parameter of source database；
Source data is extracted from corresponding source database according to the configuration parameter；
Obtain the ETL conversion parameter of the source data；
The source data is converted to obtain change data according to the ETL conversion parameter；
The output configuration parameter of the change data is obtained, the output configuration parameter includes exporting first address in warehouse
Information；
The change data is exported to the output warehouse according to first address information.
Preferably, the configuration parameter includes the second address information of source database, it is described according to the configuration parameter from
Source data is obtained in corresponding source database, comprising:
Request of data is sent to corresponding source database according to second address information, the request of data includes output
The source data information that warehouse need to be sent to terminal device；
Source data is received, the source data is that the corresponding source database responds the number sent after the request of data
According to.
Preferably, the ETL conversion parameter includes data mapping relations, it is described according to the ETL conversion parameter to described
Source data is converted to obtain change data, comprising:
The source data is converted into change data according to the data mapping relations, the data mapping relations are described
The mapping relations of source data and the change data.
Preferably, the method also includes: judge whether the change data converts success；
If the change data is not converted into function, obtains change data and convert failed error message, and will be described
Error message is written in recording documents.
Preferably, the error message includes at least one type of error, the corresponding recording documents of each error category, institute
It states the error message to be written in recording documents and includes:
Identify the error category of the error message；
According to the error category, the error message is written in recording documents corresponding with the error category.
Preferably, if the change data is converted successfully, prompt information is generated.
To achieve the above object, the present invention also provides a kind of data processing equipment based on ETL, the numbers based on ETL
Include: according to processing unit
First obtains module, for obtaining the configuration parameter of source database；
Extraction module, for extracting source data from corresponding source database according to the configuration parameter；
Second obtains module, for obtaining the ETL conversion parameter of the source data；
Conversion module, for being converted the source data to obtain change data according to the ETL conversion parameter；
Third obtains module, obtains the output configuration parameter of the change data, and the output configuration parameter includes output
First address information in warehouse；
Output module, for being exported the change data to the output warehouse according to first address information.
Preferably, extraction module is also used to send data to corresponding source database according to the source database address information
Request；And source data is received, the source data is sent to institute after responding the request of data for the corresponding source database
State the data of terminal device.
To achieve the above object, the present invention also provides a kind of terminal device, the terminal device includes:
Memory, the memory are used to store the executable data processor based on ETL of computer；And
Processor when the processor calls the data processor based on ETL, executes above-mentioned based on ETL's
Data processing method.
To achieve the above object, it the present invention also provides a kind of storage medium, is stored on the storage medium based on ETL's
Data processor, the data processor based on ETL can be executed by one or more processor, above-mentioned to realize
Data processing method based on ETL.
Compared with existing design, the data processing method based on ETL, device, terminal device and storage provided by the invention
Medium, by the configuration parameter for obtaining source database；Source data is extracted from corresponding source database according to the configuration parameter；
Obtain the ETL conversion parameter of the source data；The source data is converted according to the ETL conversion parameter to obtain conversion
Data；The output configuration parameter of the change data is obtained, the output configuration parameter includes the first address letter for exporting warehouse
Breath；The change data is exported to the output warehouse according to first address information.So as to realize change data
Automatic management, reduce artificial intervention, improve change data treatment effeciency.
Detailed description of the invention
Fig. 1 is a kind of step flow chart for data processing method based on ETL that first embodiment of the invention provides.
Fig. 2 is a kind of specific embodiment flow chart of step S12 in Fig. 1.
Fig. 3 is a kind of specific embodiment flow chart of step S14 in Fig. 1.
Fig. 4 is a kind of structural block diagram of the data processing equipment based on ETL provided by second embodiment of the invention.
Fig. 5 is a kind of structural block diagram of terminal device provided by third embodiment of the invention.
Specific embodiment
In order to make the objectives, technical solutions, and advantages of the present invention clearer, with reference to the accompanying drawings and embodiments, right
The present invention is further elaborated.It should be appreciated that described herein, specific examples are only used to explain the present invention, not
For limiting the present invention.Based on the embodiments of the present invention, those of ordinary skill in the art are not before making creative work
Every other embodiment obtained is put, shall fall within the protection scope of the present invention.
The description and claims of this application and term " first ", " second ", " third ", " in above-mentioned attached drawing
The (if present)s such as four " are to be used to distinguish similar objects, without being used to describe a particular order or precedence order.It should manage
The data that solution uses in this way are interchangeable under appropriate circumstances, so that the embodiments described herein can be in addition to illustrating herein
Or the sequence other than the content of description is implemented.In addition, term " includes " and " having " and their any deformation, it is intended that
Cover it is non-exclusive include, for example, containing the process, method, system, product or equipment of a series of steps or units need not limit
In step or unit those of is clearly listed, but may include be not clearly listed or for these process, methods, produce
The other step or units of product or equipment inherently.
It should be noted that the description for being related to " first ", " second " etc. in the present invention is used for description purposes only, and cannot
It is interpreted as its relative importance of indication or suggestion or implicitly indicates the quantity of indicated technical characteristic.Define as a result, " the
One ", the feature of " second " can explicitly or implicitly include at least one of the features.In addition, the skill between each embodiment
Art scheme can be combined with each other, but must be based on can be realized by those of ordinary skill in the art, when technical solution
Will be understood that the combination of this technical solution is not present in conjunction with there is conflicting or cannot achieve when, also not the present invention claims
Protection scope within.
A kind of data processing method based on ETL of the present invention, device, terminal device and storage medium, by obtaining source number
According to the configuration parameter in library；Source data is extracted from corresponding source database according to the configuration parameter；Obtain the source data
ETL conversion parameter；The source data is converted to obtain change data according to the ETL conversion parameter；Obtain described turn
The output configuration parameter of data is changed, the output configuration parameter includes exporting first address information in warehouse；According to described first
Address information exports the change data to the output warehouse.To realize the automatic management of change data, reduce
Artificial intervention, improves change data treatment effeciency.
Referring to Fig. 1, Fig. 1 is a kind of data processing method based on ETL provided by the invention, it should the data based on ETL
Processing method is applied to terminal device, which can be server or PC machine etc., should the data processing method based on ETL
Including step S11-S16.
Wherein, step S11 obtains the configuration parameter of source database.
The configuration parameter of source database is for allowing the terminal device according to the configuration parameter to corresponding source data
Library obtains corresponding source data.The configuration parameter is user by the input equipments such as keyboard, mouse to being configured or select.
Step S12: source data is extracted from corresponding source database according to the configuration parameter.
Terminal device extracts corresponding source data according to the configuration parameter from corresponding source database, wherein configuration ginseng
Number includes one or more of source database title, IP address, login user name or password.
Referring to Fig. 2, in some embodiments, configuration parameter includes the second address information of source database, step S12 packet
It includes: step S121-S122.
Step S121: request of data is sent to corresponding source database according to second address information, the data are asked
Seek the source data information that need to be sent to terminal device including output warehouse；
Step S122: receiving source data, and the source data is after the corresponding source database responds the request of data
It is sent to the data of the terminal device.
Terminal device operation has an operating platform, can be with when needing corresponding source data being added to the operating platform
According to the second address information that the source data saves, the i.e. address information of source database, request of data is sent to the source database,
It include the source data information that the database need to be sent in the request of data, so that the corresponding source database response request of data will
Corresponding source data is sent to the terminal device, so that the source data is imported into the operating platform by terminal device.
Step S13: the ETL conversion parameter of the source data is obtained.
The ETL conversion parameter is that user is configured as desired by input equipments such as keyboard, mouses, so that terminal is set
It is standby that corresponding source data is converted according to the ETL conversion parameter.
Step S14: the source data is converted to obtain change data according to the ETL conversion parameter.
Referring to Fig. 3, in some embodiments, the ETL conversion parameter includes data mapping relations, step S14 includes:
Step S141: the source data is converted into change data, the data mapping according to the data mapping relations
Relationship is the mapping relations of the source data and the change data.
When needing to convert source data, user can be arranged ETL by input units such as keyboard, mouses and convert ginseng
Number, the ETL conversion parameter include data mapping relations, and data mapping relations are the source data and the change data is reflected
Penetrate relationship.Corresponding source data is converted into change data according to the data mapping relations by terminal device.
In some embodiments, step S14 further include:
Step S142: judge whether the change data converts success.
Terminal device real-time monitoring and the change over condition data for obtaining the change data, to be sentenced according to the change over condition data
It is disconnected whether by the source data successful conversion to be converted at change data.
Step S143: if the change data is not converted into function, obtaining change data and convert failed error message,
And the error message is written in recording documents.
In change data conversion process, terminal device obtains the change over condition data of the change data and generation pair in real time
The recording documents answered, such as log journal file.If change data convert failed, the error message that convert failed is generated is recorded
Into the recording documents, so as to staff according to the recording documents can with the reason of timely learning convert failed to carry out in time
Debugging amendment.
In some embodiments, error message includes multiple type of errors, and recording documents include at least one and wrong class
The corresponding recording documents of type, the described error message is written in recording documents include:
Identify the error category of the error message；
According to the error category, the error message is written in recording documents corresponding with the error category.
Each error message is corresponding with unique mark, by the error category of mark identification error message, and according to mistake
Corresponding record is written in corresponding error message by the accidentally mapping relations of classification and preset error category and corresponding record document
In document, so that staff can quickly identify error category according to corresponding recording documents.
Step S144: if the change data is converted successfully, prompt information is generated.
The prompt information can be text prompt information, in auditory tone cues information or picture cues information any one,
Both any or three combination.
Step S15: obtaining the output configuration parameter of the change data, and the output configuration parameter includes output warehouse
First address information.
Step S16: the change data is exported to the output warehouse according to first address information.
User is configured by output configuration parameter of the input equipments such as keyboard, mouse to change data, which matches
Setting parameter includes the output title of change data, the output type of change data, the access mode of change data, output warehouse
First address information, the title for exporting warehouse export the login user title in warehouse, any one or more persons in password.
Terminal device exports change data to preset output warehouse according to first address information.
Referring to Fig. 4, Fig. 4 is a kind of data processing equipment 10 based on ETL provided by the invention, it should the number based on ETL
Include: according to processing unit 10
First obtains module 101, for obtaining the configuration parameter of source database；
Extraction module 102, for extracting source data from corresponding source database according to the configuration parameter；
Second obtains module 103, for obtaining the ETL conversion parameter of the source data；
Conversion module 104, for being converted to the source data according to the ETL conversion parameter to obtain conversion number
According to；
Third obtains module 105, obtains the output configuration parameter of the change data, the output configuration parameter includes defeated
First address information in warehouse out；And
Output module 106, for being exported the change data to the output warehouse according to first address information.
In some embodiments, the configuration parameter includes the second address information of source database, and extraction module 102 is also used
In sending request of data to corresponding source database according to second address information, the request of data includes that output warehouse needs
The source data information sent to terminal device；Source data is received, the source data is described in the corresponding source database response
The data sent after request of data.
In some embodiments, the ETL conversion parameter includes source data information to be converted, and conversion module 104 is also used
In obtaining source data to be converted from the source data according to the source data information to be converted；According to preset relation by institute
It states source data to be converted and is converted into change data, the preset relation is source data to be converted and the change data is reflected
Penetrate relationship.
In some embodiments, the ETL conversion parameter includes data mapping relations, and conversion module 104 is also used to basis
The source data is converted into change data by the data mapping relations, the data mapping relations be the source data with it is described
The mapping relations of change data.
In some embodiments, conversion module 104 is also used to judge whether the change data converts success；
If the change data is not converted into function, obtains change data and convert failed error message, and will be described
Error message is written in recording documents.
In some embodiments, the error message includes multiple type of errors, the corresponding record text of each error category
Shelves, conversion module 104 are also used to identify the error category of the error message；According to the error category, the mistake is believed
Breath is written in recording documents corresponding with the error category.
In some embodiments, if conversion module 104 is also used to the change data and converts successfully, prompt letter is generated
Breath.
Referring to Fig. 5, in some embodiments, terminal device 20 can be PC (Personal Computer, personal electricity
Brain), it is also possible to tablet computer, palm PC, portable computer, mine machine or server.
The terminal device 20 includes memory 201, processor 201 and bus 203, and memory 201 passes through bus 203 and place
Device 201 is managed to be electrically connected.
Wherein, memory 201 includes at least a type of readable storage medium storing program for executing, and the readable storage medium storing program for executing includes dodging
It deposits, hard disk, multimedia card, card-type memory (for example, SD or DX memory etc.), magnetic storage, disk, CD etc..Storage
Device 201 can be the internal storage unit of terminal device 20, such as the hard disk of the terminal device 20 in some embodiments.Storage
What device 201 was also possible to be equipped on the External memory equipment of terminal device 20, such as terminal device 20 in further embodiments
Plug-in type hard disk, intelligent memory card (Smart Media Card, SMC), secure digital (Secure Digital, SD) card dodge
Deposit card (Flash Card) etc..Further, memory 201 can also both include the internal storage unit of terminal device 20 or wrap
Include External memory equipment.Memory 201 can be not only used for the application software and Various types of data that storage is installed on terminal device 20,
Such as data processor based on ETL of computer-readable program etc. namely the memory 201 can be used as storage medium.
Processor 201 can be in some embodiments a central processing unit (Central Processing Unit,
CPU), controller, microcontroller, microprocessor or other data processing chips, the program for being stored in run memory 201
Code or processing data, to realize following steps:
Obtain the configuration parameter of source database；
Source data is extracted from corresponding source database according to the configuration parameter；
Obtain the ETL conversion parameter of the source data；According to the ETL conversion parameter to the source data converted with
Obtain change data；
The output configuration parameter of the change data is obtained, the output configuration parameter includes exporting first address in warehouse
Information；
The change data is exported to the output warehouse according to first address information.
In some embodiments, the configuration parameter includes the second address information of source database, and processor 201 is also used to
It performs the steps of
Request of data is sent to corresponding source database according to second address information, the request of data includes output
The source data information that warehouse need to be sent to terminal device；
Source data is received, the source data is that the corresponding source database responds the number sent after the request of data
According to.
In some embodiments, the ETL conversion parameter includes source data information to be converted, and processor 201 is also used to
It performs the steps of
Source data to be converted is obtained from the source data according to the source data information to be converted；
The source data to be converted is converted into change data according to preset relation, the preset relation is to be converted
The mapping relations of source data and the change data.
In some embodiments, the ETL conversion parameter includes data mapping relations, processor 201 be also used to realize with
Lower step: the source data is converted into change data according to the data mapping relations, the data mapping relations are described
The mapping relations of source data and the change data.
In some embodiments, processor 201 is also used to perform the steps of
Judge whether the change data converts success；
If the change data is not converted into function, obtains change data and convert failed error message, and will be described
Error message is written in recording documents.
In some embodiments, the error message includes at least one type of error, the corresponding note of each error category
Document is recorded, processor 201 is also used to perform the steps of
Identify the error category of the error message；
According to the error category, the error message is written in recording documents corresponding with the error category.
In some embodiments, processor 201 is also used to perform the steps of
If the change data is converted successfully, prompt information is generated.
The present invention also provides a kind of storage medium, the data processor based on ETL, institute are stored on the storage medium
Stating the data processor based on ETL can be executed by one or more processor, to realize at the data above-mentioned based on ETL
Reason method.
The above is only a preferred embodiment of the present invention, is not intended to limit the scope of the invention, all to utilize this hair
Equivalent structure or equivalent flow shift made by bright specification and accompanying drawing content is applied directly or indirectly in other relevant skills
Art field, is included within the scope of the present invention.
Claims (10)
1. a kind of data processing method based on ETL is applied to terminal device, which is characterized in that the described method includes:
Obtain the configuration parameter of source database；
Source data is extracted from corresponding source database according to the configuration parameter；
Obtain the ETL conversion parameter of the source data；
The source data is converted to obtain change data according to the ETL conversion parameter；
The output configuration parameter of the change data is obtained, the output configuration parameter includes the first address letter for exporting warehouse
Breath；
The change data is exported to the output warehouse according to first address information.
2. the method as described in claim 1, which is characterized in that the configuration parameter includes the second address letter of source database
Breath, it is described that source data is obtained from corresponding source database according to the configuration parameter, comprising:
Request of data is sent to corresponding source database according to second address information, the request of data includes output warehouse
The source data information that need to be sent to terminal device；
Source data is received, the source data is that the corresponding source database responds the data sent after the request of data.
3. the method as described in claim 1, which is characterized in that the ETL conversion parameter includes data mapping relations, and described
The source data is converted to obtain change data according to the ETL conversion parameter, comprising:
The source data is converted into change data according to the data mapping relations, the data mapping relations are the source number
According to the mapping relations with the change data.
4. method as claimed in claim 3, which is characterized in that the method also includes:
Judge whether the change data converts success；
If the change data is not converted into function, obtains change data and convert failed error message, and by the mistake
Information is written in recording documents.
5. method as claimed in claim 4, which is characterized in that the error message includes at least one type of error, each
Error category corresponds to a recording documents, and the described error message is written in recording documents includes:
Identify the error category of the error message；
According to the error category, the error message is written in recording documents corresponding with the error category.
6. method as claimed in claim 4, which is characterized in that the method also includes: if the change data is converted successfully,
Then generate prompt information.
7. a kind of data processing equipment based on ETL, which is characterized in that the data processing equipment based on ETL includes:
First obtains module, for obtaining the configuration parameter of source database；
Extraction module, for extracting source data from corresponding source database according to the configuration parameter；
Second obtains module, for obtaining the ETL conversion parameter of the source data；
Conversion module, for being converted the source data to obtain change data according to the ETL conversion parameter；
Third obtains module, obtains the output configuration parameter of the change data, and the output configuration parameter includes output warehouse
The first address information；
Output module, for being exported the change data to the output warehouse according to first address information.
8. data processing equipment as claimed in claim 7, it is characterised in that: extraction module is also used to according to second address
Information sends request of data to corresponding source database；Source data is received, the source data is that the corresponding source database is rung
Answer the data sent after the request of data.
9. a kind of terminal device, which is characterized in that the terminal device includes:
Memory, the memory are used to store the executable data processor based on ETL of computer；And
Processor when the processor calls the data processor based on ETL, is executed as claim 1-6 is any one
The data processing method based on ETL described in.
10. a kind of storage medium, it is characterised in that: change data processing routine is stored on the storage medium, it is described to be based on
The data processor of ETL can be executed by one or more processor, to realize as of any of claims 1-6
Data processing method based on ETL.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910542628.7A CN110413669A (en) | 2019-06-21 | 2019-06-21 | A kind of data processing method based on ETL, device, terminal device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910542628.7A CN110413669A (en) | 2019-06-21 | 2019-06-21 | A kind of data processing method based on ETL, device, terminal device and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110413669A true CN110413669A (en) | 2019-11-05 |
Family
ID=68359540
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910542628.7A Pending CN110413669A (en) | 2019-06-21 | 2019-06-21 | A kind of data processing method based on ETL, device, terminal device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110413669A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112966031A (en) * | 2019-12-12 | 2021-06-15 | 北京奇艺世纪科技有限公司 | Data processing method and device, electronic equipment and computer readable storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101364240A (en) * | 2008-10-14 | 2009-02-11 | 杭州华三通信技术有限公司 | Metadata management method and device |
| CN102508912A (en) * | 2011-11-09 | 2012-06-20 | 深圳市同洲电子股份有限公司 | Method and system for data extracting, converting and loading |
| US20130073515A1 (en) * | 2011-09-21 | 2013-03-21 | International Business Machines Corporation | Column based data transfer in extract transform and load (etl) systems |
| CN107135198A (en) * | 2017-03-28 | 2017-09-05 | 深圳市卓讯信息技术有限公司 | A kind of method for interchanging data and device based on ETL technology synergy multi-communication agreements |
| CN107463709A (en) * | 2017-08-21 | 2017-12-12 | 北京奇艺世纪科技有限公司 | A kind of ETL processing method and processing devices based on multi-data source |
| CN107784026A (en) * | 2016-08-31 | 2018-03-09 | 杭州海康威视数字技术股份有限公司 | A kind of ETL data processing methods and device |
-
2019
- 2019-06-21 CN CN201910542628.7A patent/CN110413669A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101364240A (en) * | 2008-10-14 | 2009-02-11 | 杭州华三通信技术有限公司 | Metadata management method and device |
| US20130073515A1 (en) * | 2011-09-21 | 2013-03-21 | International Business Machines Corporation | Column based data transfer in extract transform and load (etl) systems |
| CN102508912A (en) * | 2011-11-09 | 2012-06-20 | 深圳市同洲电子股份有限公司 | Method and system for data extracting, converting and loading |
| CN107784026A (en) * | 2016-08-31 | 2018-03-09 | 杭州海康威视数字技术股份有限公司 | A kind of ETL data processing methods and device |
| CN107135198A (en) * | 2017-03-28 | 2017-09-05 | 深圳市卓讯信息技术有限公司 | A kind of method for interchanging data and device based on ETL technology synergy multi-communication agreements |
| CN107463709A (en) * | 2017-08-21 | 2017-12-12 | 北京奇艺世纪科技有限公司 | A kind of ETL processing method and processing devices based on multi-data source |
Non-Patent Citations (1)
| Title |
|---|
| 周茂伟;邓苏;黄宏斌;: "基于元数据的ETL工具设计与实现" * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112966031A (en) * | 2019-12-12 | 2021-06-15 | 北京奇艺世纪科技有限公司 | Data processing method and device, electronic equipment and computer readable storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110908837B (en) | Application program exception handling method and device, electronic equipment and storage medium | |
| CN110334144A (en) | Message methods of exhibiting, device, computer equipment and storage medium | |
| CN111930489A (en) | Task scheduling method, device, equipment and storage medium | |
| CN109902073A (en) | Log processing method, device, computer equipment and computer readable storage medium | |
| CN109146095A (en) | The treating method and apparatus of multiservice system problem | |
| CN108197309A (en) | A kind of method, system and the readable storage medium storing program for executing of the storage of server assets information | |
| CN112769802A (en) | Access verification method and device based on server, electronic equipment and storage medium | |
| US10754748B2 (en) | System and method for constructing extensible event log with javascript object notation (JSON) encoded payload data | |
| CN113157315A (en) | Method, device, equipment and medium for identifying difference information of different versions of software | |
| CN110413669A (en) | A kind of data processing method based on ETL, device, terminal device and storage medium | |
| CN109446077A (en) | A kind of data library test method and device | |
| CN110019485A (en) | A kind of product data storage method, terminal device and storage medium | |
| CN108595924A (en) | Business authority management method and device, computer equipment and storage medium | |
| CN114186976A (en) | Workflow transfer method and device, computer equipment and storage medium | |
| CN109560964B (en) | Equipment compliance checking method and device | |
| CN114416560A (en) | Program crash analysis aggregation method and system | |
| CN109669811A (en) | A kind of data processing method and terminal for capableing of reliability access | |
| CN108268662B (en) | Social graph generation method based on H5 page, electronic device and storage medium | |
| CN113569538A (en) | Document generation method and device, storage medium and electronic equipment | |
| CN110413196A (en) | Drive distribution method, system, electronic equipment and the medium of external storage equipment | |
| CN110337017A (en) | Television set and its log processing method, log processing device and readable storage medium storing program for executing | |
| CN109324951A (en) | The acquisition methods and device of hard disk information in server | |
| CN109547290B (en) | Cloud platform garbage data detection processing method, device, equipment and storage medium | |
| CN116910050A (en) | Data processing method, device, system and storage medium | |
| CN113849858A (en) | Method, device, computer storage medium and terminal for realizing file supervision |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20191105 |