CN108959411A - Processing method, device and the equipment of ETL task - Google Patents
Processing method, device and the equipment of ETL task Download PDFInfo
- Publication number
- CN108959411A CN108959411A CN201810578640.9A CN201810578640A CN108959411A CN 108959411 A CN108959411 A CN 108959411A CN 201810578640 A CN201810578640 A CN 201810578640A CN 108959411 A CN108959411 A CN 108959411A
- Authority
- CN
- China
- Prior art keywords
- xml file
- character string
- etl
- serial type
- type character
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Abstract
Present disclose provides processing method, device and the equipment of a kind of ETL task；Wherein, this method is applied to the equipment that operation has ETL tool, comprising: is requested to generate XML file according to the ETL task creation of user；Compression processing is carried out to the XML file, obtains serial type character string；The serial type character string is stored in the specific field of specified database.The disclosure stores ETL task data in such a way that XML file and database combine, and also can guarantee that ETL task data is saved with security and stability under high concurrent scene, improves the reliability of ETL task storage.
Description
Technical field
This disclosure relates to data warehouse technology field, more particularly, to the processing method of ETL task, device and equipment.
Background technique
ETL (Extract-Transform-Load is extracted, conversion, loaded) is a kind of data warehouse technology, and user passes through
ETL task can extract required data from data source, by data cleansing, the mutually processes such as conversion, load data into
Destination (data warehouse such as pre-defined).
Engineer needs to save designed ETL task during designing ETL task；A kind of preserving type
It is to establish multiple tables of data in the database, the information such as line, component, data source in ETL task is stored respectively to correspondence
Tables of data in；Another preserving type is to save ETL task hereof.But the preserving type of file be easy to cause mistake
It deletes, safety is low；And for the preservation form of above-mentioned database, if multiple engineers are saved using same database simultaneously
ETL task, database may save request only in response to one of, this will cause another ETL task and saves failure or data
Abnormal situation is stored, the reliability for causing ETL task to store is lower.
Summary of the invention
In view of this, processing method, device and the equipment for being designed to provide a kind of ETL task of the disclosure, to improve
The reliability of ETL task storage.
To achieve the goals above, the disclosure the technical solution adopted is as follows:
In a first aspect, this method, which is applied to operation, ETL tool present disclose provides a kind of processing method of ETL task
Equipment, comprising: according to the ETL task creation of user request generate XML file；Compression processing is carried out to XML file, is gone here and there
Ocra font ocr string；The serial type character string is stored in the specific field of specified database.
Second aspect, present disclose provides a kind of processing unit of ETL task, which, which is applied to operation, ETL tool
Equipment, comprising: file generating module, for according to the ETL task creation of user request generate XML file；Compression module is used
In carrying out compression processing to the XML file, serial type character string is obtained；Memory module, for the serial type character string to be stored in finger
Determine in the specific field of database.
The third aspect, disclosure embodiment provides a kind of processing equipment of ETL task, including processor and machine can
Storage medium is read, the machine readable storage medium is stored with the machine-executable instruction that can be executed by the processor, institute
It states processor and executes the machine-executable instruction to realize the processing method of above-mentioned ETL task.
Fourth aspect, disclosure embodiment provide a kind of machine readable storage medium, and the machine readable storage is situated between
Matter is stored with machine-executable instruction, and when being called and being executed by processor, the machine can be held the machine-executable instruction
Row instructs the processing method for promoting the processor to realize above-mentioned ETL task.
Processing method, device, equipment and the machine readable storage medium of above-mentioned ETL task, firstly generate ETL task creation
Corresponding XML file is requested, is stored in after XML file progress compression processing, then by the serial type character string obtained after compression
In the specific field of specified database.Which stores ETL task data in such a way that XML file and database combine,
It also can guarantee that ETL task data is saved with security and stability under high concurrent scene, improve the reliability of ETL task storage.
Other feature and advantage of the disclosure will illustrate in the following description, alternatively, Partial Feature and advantage can be with
Deduce from specification or unambiguously determine, or by implement the disclosure above-mentioned technology it can be learnt that.
To enable the above objects, features and advantages of the present invention to be clearer and more comprehensible, better embodiment is cited below particularly, and match
Appended attached drawing is closed, is described in detail below.
Detailed description of the invention
It, below will be to specific in order to illustrate more clearly of disclosure specific embodiment or technical solution in the prior art
Embodiment or attached drawing needed to be used in the description of the prior art be briefly described, it should be apparent that, it is described below
Attached drawing is some embodiments of the disclosure, for those of ordinary skill in the art, before not making the creative labor
It puts, is also possible to obtain other drawings based on these drawings.
Fig. 1 is a kind of design diagram for ETL task that disclosure embodiment provides；
Fig. 2 is the schematic diagram of the tables of data for the preservation ETL task that disclosure embodiment provides；
Fig. 3 is the database structure schematic diagram for the storage ETL task that disclosure embodiment provides；
Fig. 4 is a kind of flow chart of the processing method for ETL task that disclosure embodiment provides；
Fig. 5 is the flow chart of the processing method for another ETL task that disclosure embodiment provides；
Fig. 6 is a kind of structural schematic diagram for ETL task list that disclosure embodiment provides；
Fig. 7 is the flow chart of the processing method for another ETL task that disclosure embodiment provides；
Fig. 8 is a kind of structural schematic diagram of the processing unit for ETL task that disclosure embodiment provides；
Fig. 9 is a kind of structural schematic diagram of the processing equipment for ETL task that disclosure embodiment provides.
Specific embodiment
To keep the purposes, technical schemes and advantages of disclosure embodiment clearer, below in conjunction with attached drawing to this public affairs
The technical solution opened is clearly and completely described, it is clear that and described embodiment is a part of embodiment of the disclosure,
Rather than whole embodiment.Based on the embodiment in the disclosure, those of ordinary skill in the art are not making creation
Property labour under the premise of every other embodiment obtained, belong to the disclosure protection range.
The technical solution of the disclosure in order to better understand is first below briefly described ETL task, this embodiment party
In formula, ETL task is referred to as ETL process.Data source, component and link information etc. are generally comprised in one ETL task
Element；The link information includes the connection relationship between data source and component or between each component；Fig. 1 show ETL a kind of
The design diagram of business；In the ETL task, from two data sources, data are extracted in data source A and data source B；The number extracted
After being converged component progress convergence processing, then is acted executive module (may include multiple assembly) and cleaned, converted
Processing, finally loads into data warehouse.
After the completion of above-mentioned ETL task design, saved；Since ETL task may need Reusability, save
Form needs higher safety；And the safety coefficient of database is high, user can just operate after needing to carry out authentication
Therefore data usually save each element in ETL into database.
Specifically, an ETL task can be saved by multiple tables of data, tables of data as shown in Figure 2 is saved
The all information of one ETL task；Wherein, the number in ETL task is typically used to hold with the tables of data that r_database starts
According to source information, for example, mark, the path etc. of data source A and data source B in above-mentioned Fig. 1；It is logical with the tables of data of r_step beginning
It is usually used in saving the module information in ETL task, such as the information of above-mentioned convergence component, each movement executive module；r_
Trans_lock tables of data is for saving link information, the connection between connection relationship, each component including data source and component
Relationship and the connection relationship of component and data warehouse etc..When the ETL task of preservation in the database reuses, pass through this
The information recorded in a little tables of data can be assembled into complete ETL task.
Fig. 3 show the database structure schematic diagram of storage ETL task；The database structure includes service database and deposits
Store up the database of ETL task data；Wherein, ETL task list is preserved in service database, is preserved in the ETL task list more
A list item, each list item can correspond to an ETL task；It include ETL task identification (id), title in each list item
(name), the fields such as user (user), key element (key element)；The key element can be used to save associated storage
The path of the database of ETL task data specifically includes the path of the corresponding multiple ETL tables of data of the ETL task；In general, one
Corresponding more than ten tables of data of a ETL task；Tables of data shown in Fig. 2 can be according to the data for storing ETL task data in Fig. 3
Mode in library stores.
The storage mode of above-mentioned this ETL task is relatively more suitable for single machine user's scene, i.e. engineer pacifies in local computing
ETL developing instrument is filled, independent design develops ETL task；After the completion of ETL task exploitation, then save into database.But if
Engineer designs ETL task using the ETL developing instrument of webpage version, might have multiple engineers and uses the ETL developing instrument
Design ETL task, the good ETL task of accesses database design Storage；At this point, influenced by database storage mechanism, if
Two engineers send preservation request to database simultaneously, and database is likely to save request distribution similarly for the two
ID, and one of save of normal response is requested；Preservation request for another same ID, corresponding ETL task
Meeting data insertion failure, causes not save normally, but engineer can obtain preservation and successfully feed back.
Since the storage logic of database is complex, partial data can not carry out rollback when storage is abnormal
(when i.e. data are abnormal, being restored to the behavior of last correct status), the ETL task data saved extremely may remain
Partial redundance data are in the database；These redundant datas probably will lead to the data normally saved and also become problematic
Data, and then a series of more serious problems can be caused.
It can be seen from the above, the reliability of database preserving type obviously drops in the case where the ETL task of high concurrent stores scene
It is low.Based on this, disclosure embodiment provides processing method, device and the equipment of a kind of ETL task；The technology can be applied
In storage ETL task or the storage of the data of other data mining processes, it is specifically described below.
Firstly, a kind of flow chart of the processing method of ETL task shown in Figure 4；This method, which is applied to operation, ETL
The equipment of tool；The ETL tool is usually the software that can design ETL task, for example, datastage, informatica,
Kettle etc.；Operation ETL tool equipment be usually server, PC (personal computer, personal computer) machine or its
His dedicated human-computer interaction device etc..The ETL tool may be mounted in equipment, can also pass through browser in the form of a web page
Run the ETL tool.
The above method includes the following steps:
Step 402, it is requested to generate XML file according to the ETL task creation of user；
XML (Extensible Markup Language, the extensible markup language) file is a kind of for marking electricity
Subfile makes it have structural markup language.The data of ETL task can be stored in XML file.Include in XML file
Many labels, the label are used for the attribute of flag data；It can parse the data in XML file by way of parsing label.
Wherein, the label in XML file is all closure, for example,<connection>label,<connection></connection
The content that the centering of > label saves is the link information of the database in the ETL task.
Step 404, compression processing is carried out to above-mentioned XML file, obtains serial type character string；
When specific implementation, squeeze operation can directly be carried out to above-mentioned XML file, it can also be advanced to above-mentioned XML file
Then row encoding operation carries out squeeze operation, or first carry out squeeze operation, then perform the encoding operation；These operating procedures are equal
Available above-mentioned serial type character string, and realize the purpose for reducing XML file.It can be first to the XML file according to business demand
It performs the encoding operation, for example, the non-ascii character in XML file is converted to ascii character；If necessary to the XML file
It is encoded, JDK API (Java Development Kit Application can be used
ProgrammingInterface, Java developing instrument collection Application Program Interface) in primary addressable port
XML file is carried out Base64 coding by Base64.encodeBase64；In general, Base64 coding after XML file have can not
The property read needs to read after decoding.Furthermore it is also possible to using relevant encoding tool to XML file carry out Base32 or
Base16 coding or other coding forms.
It, can be using the primary compression interface in JDK API when being compressed to the XML file
GZIPOutputStream compresses the XML file, obtains a serial type character string.Above by processing such as coding, compressions to contract
Small XML file, can the less XML file store pressure caused by memory space, while the process encoded can also improve
The reliability of data storage.
Step 406, serial type character string is stored in the specific field of specified database.
If serial type character string is directly stored in the storage resource in local computing or cloud, it is possible to cause accidentally to delete；
In order to guarantee the safety of file, serial type character string can be saved in the database.
In the embodiment, the corresponding serial type character string of an ETL task is saved in specific field in the database, phase
For in the prior art, requesting directly to save ETL task to the form of multiple tables of data of database after distributing ID to save,
The storage organization of which is simpler, high concurrent ETL task store scene under, due to ETL task information via
XML file is converted into serial type character string, so database no longer needs to request distribution ID for each preservation, to the XML file received
After being converted, serial type character string is successively saved.
The processing method of above-mentioned ETL task firstly generates ETL task creation and requests corresponding XML file, to XML text
After part carries out compression processing, then obtained serial type character string is stored in the specific field of specified database.Which uses
The mode that XML file and database combine stores ETL task data, also can guarantee ETL task data under high concurrent scene
It is saved with security and stability, improves the reliability of ETL task storage.
In another embodiment, the ETL tool in above-mentioned high concurrent scene is usually webpage version tool, is supported more
User, which accesses, to be used, such as the software service mode of SaaS (software is to service).ETL can be used by opening browser in engineer
Tool design ETL task；The interface of access database is further typically provided in the ETL tool, by the interface, engineer can be with
The ETL task handled well is saved to database.
Fig. 5 show the flow chart of the processing method of another ETL task；In this method, XML file is compiled first
Code, then the XML file after coding is compressed, to advanced optimize the data volume and structure of XML file, and improves data and deposit
The reliability of storage；This method specifically comprises the following steps:
Step S502 requests to generate XML file according to the ETL task creation of user；
Step S504 is encoded according to byte stream of the preset coding rule to XML file；
The preset coding rule can encode for the above-mentioned Base64 coding referred to, Base32 coding or Base16,
It can be other coding rules.
Step S506 carries out compression processing to the XML file after coding, obtains serial type character string；
In actual implementation, XML file first can be carried out by Base64 coding using Base64.encodeBase64；Again
Using the XML file after GZIPOutputStream method compressed encoding, serial type character string is obtained.
Pair in another embodiment, the execution of above-mentioned steps S504 and step S506 sequence can also be exchanged, i.e.,
XML file carries out compression processing；Compressed XML file is encoded according to preset coding rule, obtains serial type character
String.
Step S508 establishes task index mark in the ETL task list of service database, above-mentioned serial type character string is deposited
Task index is stored in identify in corresponding specific field.
The structure of the ETL task list can be as shown in fig. 6, can wrap containing multiple ETL tasks, often in the ETL task list
The corresponding list item of a ETL task, each list item generally include following field: ETL task identification (id), is used title (name)
Family (user), key element (key element) etc.；Wherein, id field can be used for storing the task index mark of foundation,
Key element field can be used as above-mentioned specific field, for saving above-mentioned serial type character string；Certainly, the word of above-mentioned list item
Duan Tongchang is not limited and above-mentioned listed field；The effect of each field, and the specific field for saving serial type character string are equal
User's self-setting can be passed through.
When engineer, which needs to execute again, saves ETL task in the database, can be obtained by mode shown in Fig. 7
ETL task is taken and is restored, is specifically included:
Step S702 extracts the corresponding serial type character string of ETL task from the specific field in specified database；
Engineer can send data retrieval request to database, and the task rope of ETL task can be carried in the request
Tendering knowledge, title or other can be with the identification information of unique identification ETL task；After database receives the request, according to wherein
Identification information corresponding serial type character string can be found from specified database.
Specifically, when the corresponding serial type character string of XML file saves in specific field in the database, Ke Yitong
It crosses above-mentioned identification information and searches corresponding list item from ETL task list, then from the corresponding field of the list item, such as above-mentioned key
Element field extracts the serial type character string.
Step S704 unzips it operation to the serial type character string extracted, obtains XML file；Wherein, the decompression
Operation is reciprocal relation with squeeze operation；
Step S706 executes ETL task using XML file.
If directly encoded to XML file during preservation, then serial type character string is unziped it at this time
?；If first encoded to XML file, then compression processing is carried out to the XML file after coding, has then needed elder generation at this time
The serial type character string is decompressed, then is decoded processing, obtains original XML file.If first right during preservation
XML file is compressed, then is encoded to compressed XML file, then is needed at this time first to the serial type character string solution
Code, then processing is unziped it, obtain original XML file.The original XML file includes that the execution tool of ETL can identify
The elements such as data source, component and link information, and then execute ETL task according to these elements.
In aforesaid way, XML is encoded and is compressed, XML file format, and compressed file can be advanced optimized.
Which can reduce XML file to the storage pressure of database, solve setting in the storage ETL conversion of a large number of users high concurrent
Occur the abnormal problem as caused by database performance when meter process, has ensured the accuracy of ETL task storage.
It should be noted that above-mentioned each method embodiment is described in a progressive manner, each embodiment emphasis
What is illustrated is all the difference with other embodiments, and same and similar part refers to each other i.e. between each embodiment
It can.
Corresponding to above method embodiment, a kind of structural representation of the processing unit of ETL task shown in Figure 8
Figure；The device is applied to the equipment that operation has ETL tool, comprising:
File generating module 80 generates XML file for requesting according to the ETL task creation of user；
Compression module 81 obtains serial type character string for carrying out compression processing to XML file；
Memory module 82, for the serial type character string to be stored in the specific field of specified database.
Above-mentioned ETL tool is webpage version tool, and multi-user access is supported to use.
Above-mentioned compression module is also used to execute following one: carrying out compression processing to the XML file, obtains serial type character
String；Compression processing is carried out to the XML file；The compressed XML file is encoded according to preset coding rule,
Obtain serial type character string；The XML file is encoded according to preset coding rule；To the XML file after coding
Compression processing is carried out, serial type character string is obtained.
Above-mentioned memory module is also used to: task index mark is established in the ETL task list of service database, by serial type word
Symbol string is stored in task index and identifies in corresponding specific field.
Present embodiments provide for a kind of processing equipments of ETL task corresponding with above method embodiment.Fig. 9 is
The structural schematic diagram of the equipment, as shown in figure 9, the equipment includes processor 901 and memory 902；Wherein, memory 902 is used
In storing one or more computer instruction, one or more computer instruction is executed by processor, to realize above-mentioned ETL task
Processing method.
Equipment shown in Fig. 9 further includes bus 903 and forwarding chip 904, processor 901, forwarding chip 904 and memory
902 are connected by bus 903.The processing equipment of the ETL task can be network edge device.
Wherein, memory 902 may include high-speed random access memory (RAM, Random Access Memory),
It may further include non-labile memory (non-volatile memory), for example, at least a magnetic disk storage.Bus
903 can be isa bus, pci bus or eisa bus etc..The bus can be divided into address bus, data/address bus, control always
Line etc..Only to be indicated with a four-headed arrow in Fig. 9, it is not intended that an only bus or a type of convenient for indicating
Bus.
Forwarding chip 904 is used to connect by network interface at least one user terminal and other network units, will seal
The IPv4 message or IPv6 message installed is sent to the user terminal by network interface.
Processor 901 may be a kind of IC chip, the processing capacity with signal.It is above-mentioned during realization
Each step of method can be completed by the integrated logic circuit of the hardware in processor 901 or the instruction of software form.On
The processor 901 stated can be general processor, including central processing unit (Central Processing Unit, abbreviation
CPU), network processing unit (Network Processor, abbreviation NP) etc.；It can also be digital signal processor (Digital
Signal Processing, abbreviation DSP), specific integrated circuit (Application Specific Integrated
Circuit, abbreviation ASIC), ready-made programmable gate array (Field-Programmable Gate Array, abbreviation FPGA) or
Person other programmable logic device, discrete gate or transistor logic, discrete hardware components.It may be implemented or execute sheet
Disclosed each method, step and logic diagram in invention embodiment.General processor can be microprocessor or this at
Reason device is also possible to any conventional processor etc..The step of method in conjunction with disclosed in embodiment of the present invention, can direct body
Now executes completion for hardware decoding processor, or in decoding processor hardware and software module combine and execute completion.It is soft
Part module can be located at random access memory, and flash memory, read-only memory, programmable read only memory or electrically erasable programmable are deposited
In the storage medium of this fields such as reservoir, register maturation.The storage medium is located at memory 902, and processor 901 reads storage
Information in device 902, in conjunction with its hardware complete aforementioned embodiments method the step of.
Embodiment of the present invention additionally provides a kind of machine readable storage medium, and machine readable storage medium storage is organic
Device executable instruction, for the machine-executable instruction when being called and being executed by processor, machine-executable instruction promotes processor
Realize the processing method of above-mentioned ETL task, specific implementation can be found in method implementation, and details are not described herein.
The technology of the processing unit and equipment of ETL task provided by embodiment of the present invention, realization principle and generation
Effect is identical with preceding method embodiment, and to briefly describe, device embodiments part does not refer to place, can refer to aforementioned side
Corresponding contents in method embodiment.
In several embodiments provided herein, it should be understood that disclosed device and method can also lead to
Other modes are crossed to realize.Device embodiments described above are only schematical, for example, the flow chart in attached drawing and
Block diagram shows the system in the cards of the device of multiple embodiments according to the present invention, method and computer program product
Framework, function and operation.In this regard, each box in flowchart or block diagram can represent a module, program segment or generation
A part of code, a part of the module, section or code include one or more for realizing defined logic function
Executable instruction.It should also be noted that function marked in the box can also be in some implementations as replacement
Occur different from the sequence marked in attached drawing.For example, two continuous boxes can actually be basically executed in parallel, they
Sometimes it can also execute in the opposite order, this depends on the function involved.It is also noted that block diagram and or flow chart
In each box and the box in block diagram and or flow chart combination, can function or movement as defined in executing it is special
Hardware based system is realized, or can be realized using a combination of dedicated hardware and computer instructions.
Finally, it should be noted that embodiment described above, the only specific embodiment of the disclosure, to illustrate this public affairs
The technical solution opened, rather than its limitations, the protection scope of the disclosure are not limited thereto, although referring to aforementioned embodiments pair
The disclosure is described in detail, those skilled in the art should understand that: any technology for being familiar with the art
Personnel can still modify to technical solution documented by aforementioned embodiments in the technical scope that the disclosure discloses
Or variation or equivalent replacement of some of the technical features can be readily occurred in；And these modifications, variation or replacement,
The spirit and scope for disclosure embodiment technical solution that it does not separate the essence of the corresponding technical solution, should all cover in this public affairs
Within the protection scope opened.Therefore, the protection scope of the disclosure shall be subject to the protection scope of the claim.
Claims (10)
1. a kind of processing method of ETL task, which is characterized in that the method is applied to the equipment that operation has ETL tool, packet
It includes:
It is requested to generate XML file according to the ETL task creation of user；
Compression processing is carried out to the XML file, obtains serial type character string；
The serial type character string is stored in the specific field of specified database.
2. the method according to claim 1, wherein the ETL tool is webpage version tool.
3. being gone here and there the method according to claim 1, wherein described carry out compression processing to the XML file
The step of ocra font ocr string includes following one:
Compression processing is carried out to the XML file, obtains serial type character string；
Compression processing is carried out to the XML file, and the compressed XML file is compiled according to preset coding rule
Code, obtains serial type character string；
The XML file is encoded according to preset coding rule, and the XML file after coding is carried out at compression
Reason, obtains serial type character string.
4. method according to claim 1 or 3, which is characterized in that described that the serial type character string is stored in specified number
According to the step in the specific field in library, comprising:
Task index mark is established in the ETL task list of service database, and the serial type character string is stored in the task
Index identifies in corresponding specific field.
5. the method according to claim 1, wherein the method also includes:
If necessary to execute ETL task, the serial type character string is extracted from the specific field of the specified database；
Operation is unziped it to the serial type character string extracted, obtains XML file；Wherein, the decompression operation with
The squeeze operation is reciprocal relation；
The ETL task is executed using the XML file.
6. a kind of processing unit of ETL task, which is characterized in that described device is applied to the equipment that operation has ETL tool, packet
It includes:
File generating module generates XML file for requesting according to the ETL task creation of user；
Compression module obtains serial type character string for carrying out compression processing to the XML file；
Memory module, for the serial type character string to be stored in the specific field of specified database.
7. device according to claim 6, which is characterized in that the ETL tool is webpage version tool, and multi-user is supported to visit
Ask use.
8. device according to claim 6, which is characterized in that the compression module is also used to execute following one:
Compression processing is carried out to the XML file, obtains serial type character string；
Compression processing is carried out to the XML file；The compressed XML file is compiled according to preset coding rule
Code, obtains serial type character string；
The XML file is encoded according to preset coding rule；The XML file after coding is carried out at compression
Reason, obtains serial type character string.
9. the device according to claim 6 or 8, which is characterized in that the memory module is also used to:
Task index mark is established in the ETL task list of service database, and the serial type character string is stored in the task
Index identifies in corresponding specific field.
10. a kind of processing equipment of ETL task, which is characterized in that including processor and machine readable storage medium, the machine
Readable storage medium storing program for executing is stored with the machine-executable instruction that can be executed by the processor, and the processor executes the machine
Executable instruction is to realize method described in any one of claim 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810578640.9A CN108959411B (en) | 2018-06-07 | 2018-06-07 | Processing method, device and equipment of ETL (extract transform and load) task |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810578640.9A CN108959411B (en) | 2018-06-07 | 2018-06-07 | Processing method, device and equipment of ETL (extract transform and load) task |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108959411A true CN108959411A (en) | 2018-12-07 |
| CN108959411B CN108959411B (en) | 2020-12-08 |
Family
ID=64493921
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810578640.9A Active CN108959411B (en) | 2018-06-07 | 2018-06-07 | Processing method, device and equipment of ETL (extract transform and load) task |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108959411B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112232816A (en) * | 2020-10-15 | 2021-01-15 | 北京新创智链科技有限公司 | Payment transaction data processing method, device, equipment and storage medium |
| CN112291312A (en) * | 2020-10-15 | 2021-01-29 | 杭州数梦工场科技有限公司 | ETL data synchronization method and device, electronic equipment and storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102480335A (en) * | 2010-11-30 | 2012-05-30 | 金蝶软件（中国）有限公司 | Method and system for transmitting business data |
| CN103348598A (en) * | 2011-01-28 | 2013-10-09 | 起元科技有限公司 | Generating data pattern information |
| CN103399934A (en) * | 2013-08-09 | 2013-11-20 | 神州数码(中国)有限公司 | Form data transmission and storage method |
| US9519695B2 (en) * | 2013-04-16 | 2016-12-13 | Cognizant Technology Solutions India Pvt. Ltd. | System and method for automating data warehousing processes |
-
2018
- 2018-06-07 CN CN201810578640.9A patent/CN108959411B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102480335A (en) * | 2010-11-30 | 2012-05-30 | 金蝶软件（中国）有限公司 | Method and system for transmitting business data |
| CN103348598A (en) * | 2011-01-28 | 2013-10-09 | 起元科技有限公司 | Generating data pattern information |
| US9519695B2 (en) * | 2013-04-16 | 2016-12-13 | Cognizant Technology Solutions India Pvt. Ltd. | System and method for automating data warehousing processes |
| CN103399934A (en) * | 2013-08-09 | 2013-11-20 | 神州数码(中国)有限公司 | Form data transmission and storage method |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112232816A (en) * | 2020-10-15 | 2021-01-15 | 北京新创智链科技有限公司 | Payment transaction data processing method, device, equipment and storage medium |
| CN112291312A (en) * | 2020-10-15 | 2021-01-29 | 杭州数梦工场科技有限公司 | ETL data synchronization method and device, electronic equipment and storage medium |
| CN112291312B (en) * | 2020-10-15 | 2022-07-01 | 杭州数梦工场科技有限公司 | ETL data synchronization method and device, electronic equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108959411B (en) | 2020-12-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US8990148B1 (en) | System and method for dynamic hierarchical data parsing | |
| CN112015430A (en) | JavaScript code translation method and device, computer equipment and storage medium | |
| CN110597814B (en) | Structured data serialization and deserialization method and device | |
| EP1999699A1 (en) | Edi instance based transaction set definition | |
| WO2021175053A1 (en) | Method and apparatus for executing functional module in virtual machine | |
| CN114500690A (en) | Interface data processing method and device, electronic equipment and storage medium | |
| CN108959411A (en) | Processing method, device and the equipment of ETL task | |
| CN108205482B (en) | File mount restoration methods | |
| CN105094959B (en) | It is a kind of that the method and apparatus of reconciliation file are provided | |
| CN112671734A (en) | Message processing method facing multiple data sources and related equipment thereof | |
| CN109413131A (en) | A kind of method and device of log parsing | |
| CN107977344A (en) | Date storage method, acquisition methods and server | |
| US20030131014A1 (en) | Processing data objects | |
| CN114201756A (en) | Vulnerability detection method and related device for intelligent contract code segment | |
| CN111737621B (en) | WASM-based OFD rendering method, system and device | |
| CN115982675A (en) | Document processing method, device, electronic equipment and storage medium | |
| CN111324645A (en) | Data processing method and device for block chain | |
| CN101553800B (en) | Migration apparatus which convert SAM/VSAM files of mainframe system into SAM/VSAM files of open system and method for thereof | |
| KR20080100344A (en) | Scalable algorithm for sharing edi schemas | |
| CN109857807A (en) | Transaction data synchronization method, apparatus, equipment and medium based on block chain | |
| CN102799423A (en) | Method and device for implementing dynamic method in JSF (java service face) | |
| CN115604343A (en) | Data transmission method, system, electronic equipment and storage medium | |
| CN114978649A (en) | Information security protection method, device, equipment and medium based on big data | |
| CN106156069B (en) | Log system and log recording method | |
| CN114168581A (en) | Data cleaning method and device, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |