CN106991183A - A kind of business intelligence ETL method for packing and system - Google Patents

A kind of business intelligence ETL method for packing and system Download PDF

Info

Publication number
CN106991183A
CN106991183A CN201710227461.6A CN201710227461A CN106991183A CN 106991183 A CN106991183 A CN 106991183A CN 201710227461 A CN201710227461 A CN 201710227461A CN 106991183 A CN106991183 A CN 106991183A
Authority
CN
China
Prior art keywords
data
dimension
etl
erp
analysis
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201710227461.6A
Other languages
Chinese (zh)
Other versions
CN106991183B (en
Inventor
郑政涛
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Fujian Forest Mdt Infotech Ltd
Original Assignee
Fujian Forest Mdt Infotech Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Fujian Forest Mdt Infotech Ltd filed Critical Fujian Forest Mdt Infotech Ltd
Publication of CN106991183A publication Critical patent/CN106991183A/en
Application granted granted Critical
Publication of CN106991183B publication Critical patent/CN106991183B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/25Integrating or interfacing systems involving database management systems
    • G06F16/254Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses

Landscapes

  • Engineering & Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The present invention relates to the method for packing of business intelligence ETL a kind of and system.This method can be based on several ERP data sources, build the data cube customized by ERP user.Because the industry of each enterprise, accounting system, management system, ERP suppliers, data source quantity etc. are not quite similar, Current commercial intelligence ETL fields present situation is to carry out standard packaging for specific ERP system common segment data, or all must be by professional IT personnel in the customized ETL logics of database code rank, so as to cause ETL to realize and safeguard or produce little effect or cost height enterprise, popularization of the business intelligence in medium-sized and small enterprises ERP fields is significantly constrained.The present invention makes common ERP user depart from specialty IT personnel assistance, can also complete the foundation in business intelligence data warehouse, configure, builds and update the data cube, in case Intellectual analysis is used.And pertinent literature is not found in the OLAP financial statements Chinese and English retrieval in the present invention.

Description

A kind of business intelligence ETL method for packing and system
Technical field
The present invention relates to data processing field, specially a kind of business intelligence ETL method for packing and system.
Background technology
ETL (Extraction-Transformation-Loading, extraction-conversion-loading) is BI (Business Intelligence, business intelligence) important component, because BI shows instrument (for example:Tableau、Microsoft BI, Qlikview etc.) it is highly developed and perfect, the performance period of a business intelligence project largely determines with effect Design cycle and effect in ETL module, if can not carry out accurately extracting and storing to the data that enterprise needs on demand, So business intelligence project turns into mirage unavoidably.
Inventor has found business intelligence in existing practice, and there are the following problems:Industry, accounting core due to each enterprise The phase not to the utmost such as calculation system, management system, ERP system supplier, focal point, the analytical model of data source quantity and client Together, business intelligence project be generally all project type deliver, i.e., according to client analysis purpose, design BI analysis models, according to divide Analysis model writes code by professional IT personnel and realizes ETL, also needs to realize by code when safeguarding.Certainly, also there are some ERP supplies Business is directed to the standardization of business intelligence product always, but ETL encapsulates general execution logic according to the absolute general character of enterprise, Customization is not supported either to configure or need also exist for be adjusted by professional IT personnel modification program code, SQL code completely ETL execution logic, and default ETL logics only support the entry level of all enterprise's general character to analyze.Sum it up, customization ETL logic means can not be general, and general ETL logic means can only support coarse and crude entry level to analyze, to most enterprises Break-up value is very limited, by taking financial analysis as an example:Accounting item set under several levels, some accounting item whether configuration item Mesh is adjusted, is configured with several Project Financial Analysis, is specifically which Project Financial Analysis, whether the enterprise is self-defined for Project Financial Analysis.Collection is not said Different enterprises can be different under group, also can be different in the different operating cycles of same enterprise, and more leisure opinion enterprise is customized Why individuation data can be present, just as these data have management value, the value of analysis is self-evident.
And in ERP Intellectual analysis field, professional due to the especially financial field in ERP fields, IT personnel are usual It is difficult to understand ERP professional knowledge, and ERP user can not generally also grasp the IT code languages required for ETL, so as to cause Technology barriers and communication disorders, cause the ETL parts in business intelligence project long performance period, and data check and correction is difficult, and effect is not Ideal, business intelligence system is also difficult to adjust and safeguard after reaching the standard grade, in addition ERP system data change, solidification may be caused ETL module produces indiscoverable mistake, often needs to change ETL codes, can correctly perform ETL logics.Above-mentioned ETL's Present situation result in business intelligence project implementation cost height enterprise, and popularization difficulty is heavy.
The content of the invention
It is an object of the invention to provide the method for packing of business intelligence ETL a kind of and system, it can be compatible common ERP system is (for example:Yongyou Software, JinDie Software, smooth prompt logical etc.), and as the adjustment of ERP system data source is regenerated point Dimension type or analysis dimension data are analysed, the ERP data of isomery can be merged together, and can be by ERP user according to enterprise Industry needs directly configuration ETL schemes and generation data cube (the true table of customization).
To achieve the above object, the technical scheme is that:A kind of business intelligence ETL method for packing, including it is as follows Step,
S1, the selected ERP data sources of analysis and extraction and analysis dimension, if multiple ERP data sources, go to analysis dimension Overlap simultaneously;The analysis dimension, according to actual conditions, in analysis dimension type or analysis both dimension datas at least one It is individual;
Alternatively parameter selects to user and saves as ETL schemes for S2, the analysis dimension for generating S1, point provided according to S1 Analyse dimensional properties different, that is, at least one in analysis dimension type or analysis both dimension datas is provided;According to actual conditions, If it also has multiple factual data types, factual data type can alternatively parameter be supplied to user's selection simultaneously simultaneously ETL schemes are saved in together;
S3, the analysis dimension and factual data type of ETL schemes preserved (if ETL schemes are not wrapped based on user in step S2 Type containing factual data, then be the acquiescence factual data type of encapsulation) generation data cube is built, preserve to data warehouse In, each ETL scheme have and only correspond to data cube (while comprising dimension and the fact, can be T-SQL sentences, The object such as view or database table), so that BI demonstration tools, report tool etc. are called.
In an embodiment of the present invention, in the step S2,
User except select it is described analysis dimension, factual data type parameter, may also provide factual data calculating standard (i.e. plus Carry parameter) selected as parameter to user;
User can change above-mentioned ETL schemes at any time, when new departure is preserved or when scheduled, will reconstruct and update according to new departure Data cube.
In an embodiment of the present invention, in the step S3,
Preserved in data cube to data warehouse, according to the characteristic of ERP data sources, creation analysis dimension and factual data Storing framework is in order to data storage, specifically:
The ERP data sources refer to the database of the various information management system of enterprises and institutions, described information management system bag Include financial system, supply chain, human resources, the manufacturing, COST system etc.;
The memory cell of equivalent amount, Mei Gecun are generated according to the corresponding organization's number of ERP data sources (or data source number) Storage unit includes several consistent dimension tables of storage organization and true table, that is, the table of the same property of all memory cell Number of columns and attribute be just as;
User can change ERP data sources at any time, after the adjustment of ERP data sources or when ETL schemes are scheduled, and the storing framework is by basis New ERP data sources are rebuild.
In an embodiment of the present invention, after in the step S3, in addition to a data conversion step S4:Dimension is virtually weighed Group, for the code recombination analysis dimension of user zero, i.e. user can carry out classification dimension it is self-defined, and by configured in S2 several The detailed dimension that ETL schemes are mapped in self-defined classification dimension, will be self-defined according to multistage when scheme is preserved or is scheduled Classification dimension and correspondence ETL schemes in dimension build and update the data cube, called for BI demonstration tools.
In an embodiment of the present invention, for financial analysis, in dimension map, simultaneously each can be specified self-defined Whether detailed dimension is deducted item, to support financial statement olap analysis, and the scheme that dimension is virtually recombinated is supported according to financial statement Report item set, one form of a scheme correspondence is (for example:Balance sheet, profit and loss statement etc.) or form Partial Elements are (for example:Assets class is total, be in debt and owner's equity etc.), specifically include:
Being directly generated based on the virtual reorganization scheme of the dimension can support the financial statement data cube of olap analysis (for example: Assets add up to-current assets-money-capital-cash in banks-XX banks, support collects, drilled through step by step step by step, stripping and slicing of cutting into slices etc.), The financial statement is individual statements, the consolidated statement of several enterprises of individual enterprise, or loads counteracting data, is reported as merging Table.
Present invention also offers a kind of business intelligence ETL package system, including,
ETL components are pre-processed, for analyzing the N number of ERP data sources specified, and analysis dimension type or analysis number of dimensions are generated According at least one in the two, if during multiple ERP data sources, should be carried out to the analysis dimension type or analysis dimension data Duplicate removal merges;Wherein, N is the integer more than or equal to 1;
ETL layout structures, for showing that the pretreatment ETL components merge analysis dimension type (the analysis dimension selection of generation Device) or analysis dimension data (loading environment selector), encapsulate and support the selector of user configuring above-mentioned parameter to increase newly, repair Change and store customization ETL schemes;According to actual conditions, if it also has multiple factual data types, factual data class Type (factual data selector) can alternatively parameter be supplied to user to select and be saved in customization ETL schemes together simultaneously;
The true table layout structure of customization, according to the customization ETL schemes of ERP user configurings (if do not included in such ETL scheme Factual data type, then the fact that encapsulated for program data type) build generation data cube, ETL schemes and data cube Body is corresponded;This sentences database table that is, the true table of customization is replaced as data cube, or with T-SQL sentences, view etc. Generation database table is used as data cube;
Customization ETL components, according to the true table of the customization ETL scheme constructses of ERP user configurings generation customization, or T-SQL languages Sentence, view etc..
In an embodiment of the present invention, the ETL layout structures, may also display loading parameter and are selected to user;
ERP user can change the ETL schemes (or the virtual reorganization scheme of dimension) used, and preserve the program, with after an action of the bowels Continuous ETL layout structures are called and changed and customize true table replacement component and the true table of corresponding customization is rebuild, and are adjusted With customization ETL components.
In an embodiment of the present invention, in addition to dimension virtually recombinates component, enable a user to carry out self-defined It is classified dimension to set, and several ETL schemes virtual maps is given to the self-defined detailed dimension configured, concrete function includes:
User can distinguish several specified ETL schemes virtual map to wherein the one of user configuring according to analysis demand In self-defined detailed dimension in the virtual reorganization scheme of dimension, the cured dimension membership of restructuring ERP system is reconstructed " many This brand-new dimension of the self-defined dimension-ETL Scheme Choices dimension of level ", and the condition and ginseng loaded with reference to data in ETL schemes Number, complete self-defined analysis dimension;
For financial analysis, by by financial statement (for example:Asset-liabilities, owner's equity) each form element set It is set to the classification dimension of different levels, and sets each self-defined detailed dimension whether to deduct item, realizes olap analysis finance report Table.
In an embodiment of the present invention, in addition to a data warehouse framework layout structure, a data warehouse framework reset structure Part, the true table of a customization reset component and a data permission control member,
The data warehouse framework layout structure, the data structure characteristics for analyzing selected ERP data sources, and according to characteristic Creation analysis dimension is (for example:Accounting item, Project Financial Analysis, true type etc.) storing framework, according to corresponding group of ERP data sources Loom structure number (or data source number) generates the memory cell of equivalent amount, if each memory cell is comprising storage organization consistent Dry dimension table and true table;
The data warehouse framework resets component, when being changed for ERP data sources, reconstructs data warehouse storage framework;
The true table of customization resets component, for when user ETL schemes or the virtual reorganization scheme of dimension change, reconstruct to be fixed The true table of system;
The data permission control member, during for exporting the true table of customization to BI demonstration tools, authorizing according to user, only exporting The fact that authorized data.
In an embodiment of the present invention, the data warehouse framework layout structure, also with following function:
The database table structure of the database on specified database server is assessed, according to the storage organization feature recognition ERP of database Systems provider and product type, it is automatic to build ERP data source lists, in case ERP user configurings;
The ERP data sources refer to the various information Management System Data storehouse of enterprises and institutions, and above- mentioned information management system includes Financial system, supply chain, human resources, the manufacturing, COST system etc.;
For the different ERP system products of each ERP system supplier, the data of the supporting product in pretreatment ETL components are called ETL methods, carry out data prediction.
Compared to prior art, the present invention can support the ERP system number of several isomeries (or isomorphism) of several enterprises According to source, multiple or single ERP data sources of some enterprise can be also supported, by pre-processing ETL components transmission analysis dimension type With analysis dimension data to data warehouse subdivision update with storage, user by ETL layout structure rapid configuration ETL schemes, Customization ETL is by user's scheme constructses and updates the true table of customization;
It is emphasized that the present invention is supported the isomate of different suppliers, data are merged with identical storage architecture Same data warehouse is drawn into, so as to realize that the data of isomery ERP data sources merge, makes the ERP systems using multiple suppliers The conglomerate of system, or changed ERP system and while there is the enterprise of heterogeneous data source, can quickly realize many of isomery ERP data sources merge, and provide support for follow-up customization ETL;
It is clear that this method can also be supported to possess the conglomerate of different industries subordinate enterprise simultaneously, either it is multiple still Individual data source, and Intellectual analysis the fact that call tables of data, all pass through ETL layout structures by user or dimension are empty Intend the restructuring code configuration of component zero, ERP fields business intelligence project ETL cost of implementation is greatly reduced, while relative to general Standardize for ETL encapsulation, data analysis effect is greatly improved;
Especially it is emphasized that OLAP financial statements signified in the present invention, refer in particular to OLAP balance sheets and (support all Collecting step by step for such as " assets adding up to-current assets-money-capital-cash in banks-XX banks ", is drilled through step by step, stripping and slicing of cutting into slices) (remittance step by step of such as " total profit-operating profit-income from main operation-operation cost-travel charge " is supported with OLAP profit and loss statements Always, drill through step by step, stripping and slicing of cutting into slices) it is the customized brand-new things of an inventor, it can not find correlation with Chinese and English retrieval at present Any document.
Brief description of the drawings
The current ETL method configuration diagrams of Fig. 1.
The ETL system configuration diagram of Fig. 2 one embodiment of the present of invention.
The ETL system schematic flow sheet of Fig. 3 one embodiment of the present of invention.
The ETL configuration selector schematic diagrames of Fig. 4 one embodiment of the present of invention.
Fig. 5 lists dimension and virtually recombinates component operation principle schematic diagram exemplified by reconstructing subject analysis dimension.
Fig. 6 the present invention another be applied to forms data source embodiment ETL method flow schematic diagrams.
Embodiment
In order to be better understood from business intelligence ETL of the present invention method for packing and system, below in conjunction with Fig. 2-figure 5 pairs of of the invention specific embodiments carry out the description of system.Each component involved in the present invention and selector are (included but is not limited to by programming:Java, H5, PHP, C#, VB) method encapsulation realize, and provide operation interface to user (especially ERP user) carries out operational control.
As shown in Figure 3:
Step 301:First there is provided interface is by user configuring and connects ERP data source servers, data warehouse framework layout structure The storage organization of database on server is analyzed, and feature and database table are stored according to the system of ERP suppliers Structure recognition ERP supplier, the classification of system, the version of system.
If ERP data sources support a set of account of group (for example:NC, U9 of Yongyou Software, EAS, K/3 of JinDie Software Cloud), then organizational chart is built according to the organizational chart of ERP data sources.
If ERP data sources do not provide the function of institutional framework, one data source of an enterprise is (for example:Yongyou Software U8, smooth prompt logical T series of products, K/3 WISE, KIS series of products of Kingdee), then carried by data warehouse framework layout structure ERP data sources are mapped to the organization specified for an interface for users maintenance organization structural table, and by user.Due to one Individual enterprise there may be multiple ERP data sources (one data source of a fiscal year of such as UFSOFT U8 early productions, Huo Zhegai Become base profile and build account again), so the same enterprise that multiple ERP data sources are mapped in organization by user must be supported Industry.
Step 302:Preferably, in order that ETL whole system more efficiently, should will analyze dimension type, analyze dimension Data, factual data etc. first carries out an ETL and is saved in data warehouse.It is true in order to build rationally applicable storing framework Table divides memory cell storage with dimension table according to organization or data source, is each organization (or ERP data sources) Virtual one single memory cell, the unit includes storage organization consistent correlation analysis dimension table and underlying fact tables, visitor When family ETL components are called, according to ETL schemes or the virtual reorganization scheme aggregate statistics preprocessed data of dimension, customization is updated True table.
Analyze dimension table to refer to specifically analyze dimension data, such as:Operating Chart of Accounts, Project Financial Analysis table are (when it is implemented, regarding pre- It is removable to be divided into if data volume is larger depending on the amount of counting:Commodity list, customer table, supplier's table, department table, staff table etc.), By taking subject dimension table as an example, data warehouse framework reset component to the subject levels of each ERP data sources, whether configure project core The characteristics such as calculation are analyzed, and create compatible all data source section purpose storing frameworks.
Step 303:When ERP data sources are come into force by modification preservation, then data warehouse framework is called to reset component.
Step 304:The loading range that pretreatment ETL components are specified according to timestamp or user, by specified ERP data sources Analysis dimension type update arrive dimension type list, or by analyze dimension data update to correspondence memory cell analysis dimension Table, or said two devices are handled simultaneously, factual data cleans and is re-loaded to the underlying fact tables of correspondence memory cell.
Whether pretreatment ETL components can be already present on data warehouse according to the data characteristics identification data of analysis dimension, If there is no then synchronous analysis dimension data, with bound data integrality.
Step 305:If data source increases or adjusted, data warehouse framework, which resets component, can clean all dimension tables, and Call data warehouse framework layout structure rebuild data warehouse storage framework (or according to circumstances while clean true table, and Reset time is stabbed), the component also can as needed be called by user.
Step 306:User can be by the 1- in selector selection analysis dimension, factual data, loading environment, loading parameter 4 projects, specific embodiment is follow-up 4 carefully to be stated referring to the drawings.
Step 307:User can by configuring the virtual reorganization scheme of dimension, and ETL layout structures have been configured several ETL schemes (including analysis dimension, factual data, loading environment, loading parameter) are mapped to the self-defined detailed dimension specified.Tool The embodiment of body is follow-up 5 carefully to be stated referring to the drawings.
Step 308:Check that the true table of the corresponding customization of ETL schemes whether there is, the fact is customized if there is no then calling Table layout structure.Otherwise, turn to and check whether ETL schemes adjust.
Step 309:The true table layout structure of customization is verified to allocation plan type, if ETL layout structures are protected The scheme deposited, then the analysis dimension specified according to user and factual data build the true table of customization.
If dimension virtually recombinates the scheme of component preservation, then according to the multistage self-defined dimension of user configuring and reservation ETL scheme dimensions, and factual data builds the true table of customization corresponding to the virtual reorganization scheme of dimension.
Step 310:When ETL schemes are come into force by modification preservation, component then calls the true table of customization to reset component.And reset The timestamp of self-defined detailed dimension is mapped in the virtual reorganization scheme of dimension of association.
Step 311:Customization ETL components are verified to allocation plan type, if the side that ETL layout structures are preserved Case, then the analysis dimension specified according to user and factual data extract data from subdivision data warehouse, and carry out aggregate statistics. If dimension virtually recombinates the scheme of component preservation, then tieed up according to the self-defined dimension of user configuring and the ETL schemes of reservation Spend, data are extracted from corresponding true tables of several customizations, and carry out aggregate statistics, build and update the corresponding customization fact Table.
One ETL scheme or the virtual reorganization scheme of dimension, will build one and only build a data cube, preferably Ground, the data cube is the true table of customization, as described above, customization fact table data or from subdivision data warehouse, Or source and the true table of other customizations, the star-like storing framework of classics is not used, but the true table of customization is included one The data cube (DataCube) included entirely, that is, selected analysis dimension level and factual data comprising all simultaneously.Citing For, for financial analysis, such as comprising having selected:It is all kinds of times, each level institutional framework, each level subject, each Level Project Financial Analysis, remaining sum and amount etc. are arranged, to support to be directly used in Intellectual analysis;The data cube included entirely Benefit essentially consists in the complicated incidence relation that avoids between star-like storing framework database table, and (the Left join Left-wing Federations connect, Right Couple in the right connections of join, Inner join, Full join couple entirely, Cross join cross products, Primary key masters Key, Foreign key external keys, Join field join field etc.) etc. the amateur IT personnel that cause can not be by true table and dimension Spend table turns into olap analysis data source by query designer association, and core purpose of the invention is in order that common after all ERP user can configure ETL schemes completely and build the fact that oneself is expected custom table, to be directly used in Intellectual analysis, institute Not select traditional star-like storing framework.
Alternately select, can also be not based on physics mode generation tables of data, and only build one section of T-SQL sentence using as Business intelligence data source, but this method is selected, when data reach certain magnitude, performance will be difficult to ensure, it is contemplated that final The user of business intelligence is usually enterprise owner or decision-making level, so the present invention is preferred by dimension and the fact to be uniformly stored in one The storing framework of individual physical table.
Step 312:After ETL schemes change, the corresponding true table of customization can not be true by customization only according to update of time stamp Table resets the true table of component refactoring customization, and calls customization ETL components.The true table of customization resets component to needing reconstruct to determine The true table of system is cleaned, and the true true table of table layout structure reconstruct customization of customization is called after the completion of cleaning.Preferably, thing is customized Real table, which resets component, can also check whether the ETL schemes are associated by the virtual reorganization scheme of dimension, if there is association scheme, then Customization ETL components are called also to carry out clearly the corresponding data of the true table of customization by the virtual reorganization scheme generation of dimension of association Wash and heavy duty.
As described in step 306, the design of selector in ETL layout structures, configuration ETL side that can be flexible and efficient with user For the purpose of case, the various controls commonly used from ERP user are as interface operation carrier.User can increase newly and preserve new ETL Scheme, can also preset to component or the scheme of user's history configuration is adjusted and preserved.Preferably, in order in ETL side The operating efficiency of user is improved during case substantial amounts, ETL layout structures allow user to the self-defined classification of ETL schemes, and by ETL The classification that scheme is specified by user is deposited, it is allowed to user search and selection ETL schemes.
As shown in figure 4, the selector in ETL layout structures includes following part, wherein module 403 and 405 should be deposited at least At one:
Module 401:It is main to include analysis dimension selector and factual data selector, the data row of the true table of customization will be influenceed.
Module 402:Mainly comprising loading environment selector and loading parameter selector, the number of the true table of customization will be influenceed According to row, or influence the data value of the fact that in data row.
Module 403:Analyze dimension selector to arrange the dimension data for influenceing the true table of customization, the data row of the selector The dimension type that table is generated from pretreatment ETL components, for selecting which dimension to participate in analysis.Pass through combo box, list The selection of the controls such as choice box will be analyzed which dimension, for financial analysis, for example:The true table of current customization Theme is selling charges analysis, can combine dimension with selection analysis " department+item sale ", can also a selection analysis " office worker " Dimension.
Module 404:The fact that factual data selector will influence customization fact table data are arranged.Pass through list choice box etc. Control selection will be analyzed which factual data, for financial analysis, for example:Beginning balance, debit is initial remaining Volume, credit side's beginning balance, debit's amount, credit amount, ending balance, debit's ending balance, credit side's ending balance, the profit and loss Volume etc..The fact that will be different data several operation interfaces, such as profit and loss class ETL arrangements forms can also be directly packaged into, Directly encapsulation amount of excess and deficit access, without by user's select facts data.
Module 405:Loading environment selector will influence the data row of the true table of customization, and the data list of the selector comes Come from the dimension data of pretreatment ETL component generations, the data for screening specified dimension.Selected by tree-shaped choice box, date Select the controls such as frame and specify the data area currently to be analyzed.For example:By date choice box selection time scope, pass through tree-shaped choosing Select frame specifies which mechanism of particular organization, or which specific accounting item etc. only counted.Certainly, in order to controlling to own The loading environment of company's (or data source), dimension here, it may be that the merging dimension of the whole group after duplicate removal merging.With meeting Count exemplified by subject, using subject code as key column, the ERP data sources to whole group carry out duplicate removal merging, form group's subject Table, so that the tree-shaped selector of subject is selected.
Module 406:Load the fact that parameter selector is by influenceing in the data row of the true table of customization, or influence data row Data value.Designate whether to load specific data by controls such as list choice boxs, for example:The state of voucher is (unexamined Core, audit, transferred items), and by the controls such as check box selection loading parameter, for financial analysis, for example:Whether examine Consider the combined offset fact, whether calculate percentage of shares etc..
The virtual restructuring component of 5 pairs of dimensions is carefully stated below in conjunction with the accompanying drawings, by taking financial analysis as an example, on the one hand, enterprise is More efficient management data, it will usually reduce the level of subject or Project Financial Analysis, and quantity more huge one-level dimension is set Spend (subject, Project Financial Analysis), by taking subject as an example, one-level subject generally has more than hundred, and object is too many, unsuitable data comparison point Analysis;On the other hand, the base profile classification of ERP system is typically based on daily management needs, except the level of this solidification is subordinate to Category relation, enterprise is generally also needed to according to other statistical method analyzing financial datas, for can by user configuring ETL methods For, user can be supported separately to configure multistage self-defined dimension, and several specified ETL schemes (are contained by virtual map Analyze dimension, factual data, loading environment and load parameter) virtual map is to the self-defined detailed dimension specified, and generation is new to divide It is vital to analyse dimension.
Dimension virtually recombinates component, supports the virtual reorganization scheme of User Defined dimension, and by user's virtual map ETL side Case, expands analysis dimension, customization ETL components expanded again according to virtual map after multistage self-defined dimension and ETL schemes in Dimension, and the corresponding true table of customization of ETL schemes builds and updates the true table of customization, so that by customization ETL effect Lifted to a New step.
Do not have tangible membership between self-defined detailed dimension and analysis dimension, component is virtually recombinated according only to dimension In the association specified, form virtual map relation, an ETL scheme can be mapped to the virtual reorganization scheme of multiple dimensions, and be somebody's turn to do The common practice in field is:Directly writing code specifies loading environment to collect dimension data upwards, under the pattern, and analysis dimension is not Expanded, do not support user flexibility to configure, less support olap analysis (collect step by step, drill through step by step, stripping and slicing of cut into slices etc.), and Invention realizes the restructuring and expansion of analysis dimension by the virtual reorganization scheme of dimension of user configuring.Both easily realized by dimension The virtual restructuring of degree carries out various collect statistics analyses to data, also supports " multistage self-defined dimension-multistage in olap analysis ETL schemes dimension " is completely through the organic whole of one complete analysis chain of formation supports olap analysis.
For financial statement analysis, the virtual reorganization scheme of dimension is supported to set according to the report item of financial statement, and one Individual scheme one form of correspondence or Partial Elements, and allow user to set each self-defined detailed dimension whether to deduct item, structure Part can be directly generated based on the virtual reorganization scheme of the dimension can support the financial statement of olap analysis (for example:Balance sheet, Profit and loss statement etc.) data cube, these financial statements can be the individual statements of monomer enterprise or some subordinate enterprise of group, Can also be the consolidated statement of several enterprises, or " consideration combined offset " in selection loading parameter, loading counteracting data, As consolidated accounts.
As described in step 307, dimension, which virtually recombinates component, can recombinate the dimension of customer analysis.Dimension virtually recombinates component It is main to include two cores:Dimension virtually recombinates configurator, it is allowed to which user is newly-increased or configures the virtual reorganization scheme of dimension, often Individual dimension virtually recombinates the self-defined dimension for supporting several ranks.Preferably, user can classify to dimension scheme, and Search function is provided simultaneously.To improve the efficiency of user search and selection scheme.
Dimension virtually recombinates mapper, it is allowed to which the ETL schemes preserved are mapped in the virtual reorganization scheme of dimension by user Most detailed self-defined dimension, so as to form virtual associated relation, it is preferable that configurator support user selection original ETL schemes in Analysis dimension whether continue retain, and retain dimension level so that user can with it is more targeted customization and deposit The true table of storage customization, dimension virtually recombinates component and can tieed up when stored according to self-defined dimension, the ETL schemes for being arranged to retain Degree and corresponding dimension level, expand and form new self-defined analysis dimension, and customization ETL components call dimension virtually to recombinate During scheme, it will call the self-defined analysis dimension, the corresponding true table of several customizations of aggregate statistics original ETL schemes.Generation The true table of new customization.
Preferably, component supports dimension virtual reorganization scheme by each self-defined detailed dimension storage time stamp, by than The virtual reorganization scheme of dimension is verified and updates to the renewal time stamp and the self-defined detailed dimensions updating timestamp of ETL schemes The data of the corresponding true table of customization.If user reconfigures ETL schemes simultaneously, ETL layout structures will also reset corresponding Self-defined detailed dimension timestamp, so that the virtual reorganization scheme of the dimension that have modified ETL scheme sections can completely update.
In order to further illustrate how dimension virtually maps between restructuring and ETL schemes, Fig. 5 illustrates mapping It is to list assets in the self-defined dimension of one group of three level, figure to add up to-current assets-in the rough schematic of expansion, figure Accounts receivable, wherein accounts receivable are self-defined detailed dimensions, one ETL scheme of virtual map, due to needing according to subject and Client carries out reclassification to remaining sum, so analysis dimension selector have selected subject and client (if receivable class subject does not have item Mesh is adjusted, only scarabaeidae mesh, then only selects subject), also since it is desired that reclassification, balance type have selected debit's ending balance, And accounts receivable and the balance due of Deposit received belong to accounts receivable, so subject have selected the two one-level subjects, and Loading parameter have selected consideration percentage of shares and consider combined offset.The OLAP balance sheets namely provided are big shareholder's power The consolidated accounts of beneficial part.
The BI products of main flow have considerable part not support data permission to control, or the authority of user accesses data is needed Develop and realize in script level, the Qlikview of such as one of BI leading manufacturers is accomplished by defining user-accessible by script Data permission.Based on this present situation, it is preferable that the present invention can provide data output by modes such as API or Web Service Interface, to export the true table of customization.
On the one hand, as shown in Fig. 2 the system can provide data permission control member, user is only exported authorized addressable Data:Authority data model, organization, analysis dimension (such as client, supplier, department), BI systems pass through interface User name is returned, interface routine is according to the user data authority of passback before output, and only exporting the user and having authorized allows to access Data.
What is more important, BI calls data access interface, before output data, the interface can verify ETL schemes with Whether the renewal time stamp of the virtual true table of reorganization scheme correspondence customization of dimension is consistent with pretreatment ETL timestamps, if inconsistent, Customization ETL components are called to update the true table of customization.To avoid planned dispatching or the slow caused business of renewal be manually specified The non-current data shown in intelligence.If it is clear that BI and ETL exists as the product of an entirety, only providing interior Portion's access interface or method.
Fig. 6 is a kind of second embodiment of business intelligence ETL methods, and this method can be used for the ETL of list ERP data sources:
Step 601:Obtain ERP data sources analysis dimension (analysis dimension type or some analysis dimension real data or Both persons generate simultaneously), in case the ETL layout structures selection in step 602.
Step 602:User can be by the 1- in selector selection analysis dimension, factual data, loading environment, loading parameter 4 projects, specific embodiment is shown in that one embodiment is stated for the thin of accompanying drawing 4.
Step 603:The true table of the corresponding customization of inspection ETL configurations whether there is, and the fact is customized if there is no then calling Table layout structure.
Step 604:The analysis dimension and factual data that the true table layout structure of customization is specified according to user build customization thing Real table.
Step 605:When ETL schemes are come into force by modification preservation, component then calls the true table of customization to reset component.
Step 606:The analysis dimension and factual data that customization ETL components are specified according to user are extracted from ERP data sources Data, and data are updated to the true table of customization.
Step 607:After ETL schemes change, the corresponding true table of customization can not be true by customization only according to update of time stamp Table resets the true table of component refactoring customization, and calls customization ETL components.
In order that description of the invention is concisely, embodiments of the invention are only to most common financial mould in ERP products Block is described and illustrated, it should be apparent that, method for packing (in addition to OLAP financial statements) described in the invention is same suitable For the other systems module of ERP products, include but is not limited to:Supply chain, production system, COST system, budgeting system, manpower Resource system.
By above-mentioned two embodiment, above description is embodiments of the invention and the technical principle used, Those skilled in the art is clearly understood that the present invention, and can be by way of programming encapsulation, with software product, interface Or the mode of service realizes the present invention, or modified or modification based on the present invention.So, those skilled in the art incite somebody to action this The described method and system of invention carry out formal modification, or conception under this invention makes a change, the work(produced by it When can act on the spirit still covered without departing from specification and accompanying drawing, protection scope of the present invention should be belonged to.

Claims (10)

1. a kind of business intelligence ETL method for packing, it is characterised in that:Comprise the following steps,
S1, the selected ERP data sources of analysis and extraction and analysis dimension, if multiple ERP data sources, go to analysis dimension Overlap simultaneously;The analysis dimension, according to actual conditions, in analysis dimension type or analysis both dimension datas at least one It is individual;
Alternatively parameter selects to user and saves as ETL schemes for S2, the analysis dimension for generating S1, point provided according to S1 Analyse dimensional properties different, that is, at least one in analysis dimension type or analysis both dimension datas is provided;According to actual conditions, If it also has multiple factual data types, factual data type can alternatively parameter be supplied to user's selection simultaneously simultaneously ETL schemes are saved in together;
S3, the analysis dimension and factual data type for preserving based on user in step S2 ETL schemes build generation data cube Body, and preserve into data warehouse, each ETL scheme has and only corresponds to a data cube, for subsequent calls.
2. method according to claim 1, it is characterised in that:In the step S2,
User may also provide the calculating standard conduct of factual data except selecting the analysis dimension, factual data type parameter Parameter is selected to user;
User can change above-mentioned ETL schemes at any time, when new departure is preserved or when scheduled, will reconstruct and update according to new departure Data cube.
3. method according to claim 1, it is characterised in that:In the step S3,
Preserved in data cube to data warehouse, according to the characteristic of ERP data sources, creation analysis dimension and factual data Storing framework is in order to data storage, specifically:
The ERP data sources refer to the database of the various information management system of enterprises and institutions, described information management system bag Include financial system, supply chain, human resources, the manufacturing, COST system;
The memory cell of equivalent amount is generated according to the corresponding organization's number of ERP data sources, each memory cell includes storage Several consistent dimension tables of structure and true table, that is, the number of columns and attribute of the table of the same property of all memory cell It is just as;
User can change ERP data sources at any time, after the adjustment of ERP data sources or when ETL schemes are scheduled, and the storing framework is by basis New ERP data sources are rebuild.
4. method according to claim 1, it is characterised in that:After in the step S3, in addition to a data conversion step S4:Dimension is virtually recombinated, and it is self-defined to carry out classification dimension for the code recombination analysis dimension of user zero, i.e. user, and by S2 The detailed dimension that several ETL schemes of middle configuration are mapped in self-defined classification dimension, will when scheme is preserved or is scheduled Dimension in multistage customized classification dimension and correspondence ETL schemes is built and updates the data cube, and work is shown for BI Tool is called.
5. method according to claim 4, it is characterised in that:For financial analysis, in dimension map, it can refer to simultaneously Whether each fixed self-defined detailed dimension is deducted item, and to support financial statement olap analysis, the scheme that dimension is virtually recombinated is supported Set according to the report item of financial statement, the Partial Elements of scheme one form of correspondence or a form, specifically Including:
The financial statement data cube of olap analysis, the finance can be supported by being directly generated based on the virtual reorganization scheme of the dimension Form is individual statements, the consolidated statement of several enterprises of individual enterprise, or loads counteracting data, as consolidated accounts.
6. a kind of business intelligence ETL package system, it is characterised in that:Including,
ETL components are pre-processed, for analyzing the N number of ERP data sources specified, and analysis dimension type or analysis number of dimensions are generated According at least one in the two, if during multiple ERP data sources, should be carried out to the analysis dimension type or analysis dimension data Duplicate removal merges;Wherein, N is the integer more than or equal to 1;
ETL layout structures, for showing that the pretreatment ETL components merge the analysis dimension type or analysis number of dimensions of generation According to encapsulation supports the selector of user configuring above-mentioned parameter to increase newly, change and store customization ETL schemes;According to actual feelings Condition, if it also has multiple factual data types, factual data type can alternatively parameter be supplied to user to select simultaneously Select and be saved in customization ETL schemes together;
The true table layout structure of customization, data cube, ETL side are generated according to the customization ETL scheme constructses of ERP user configurings Case and data cube are corresponded;This sentences database table that is, the true table of customization is as data cube, or with T-SQL languages Sentence, view alternate data storehouse table are used as data cube;
Customization ETL components, according to the true table of the customization ETL scheme constructses of ERP user configurings generation customization, or T-SQL languages Sentence, view.
7. system according to claim 6, it is characterised in that:
The ETL layout structures, may also display loading parameter and are selected to user;
ERP user can change the ETL schemes used, and preserve the program, so that follow-up ETL layout structures are called and changed And the true table of customization resets component and the true table of corresponding customization is rebuild, and call customization ETL components.
8. system according to claim 6, it is characterised in that:Component is virtually also recombinated including dimension, so as to obtain user Multistage self-defined dimension can be carried out to set, and several ETL schemes virtual maps are given to the detailed dimension configured, specific work( It can include:
User can distinguish several specified ETL schemes virtual map to wherein the one of user configuring according to analysis demand In self-defined detailed dimension in the virtual reorganization scheme of dimension, the cured dimension membership of restructuring ERP system, the new dimension of reconstruct Degree, and the condition and parameter loaded with reference to data in ETL schemes, complete self-defined analysis dimension;
For financial analysis, by the way that each form element of financial statement to be set to the classification dimension of different levels, and Set each self-defined detailed dimension whether to deduct item, realize olap analysis financial statement.
9. the system according to claim 6 or 8, it is characterised in that:Also include a data warehouse framework layout structure, a number Component, the true table of a customization, which are reset, according to warehouse framework resets component and a data permission control member,
The data warehouse framework layout structure, the data structure characteristics for analyzing selected ERP data sources, and according to characteristic Creation analysis dimension storing framework, the memory cell of equivalent amount is generated according to the corresponding organization's number of ERP data sources, each Memory cell includes several consistent dimension tables of storage organization and true table;
The data warehouse framework resets component, when being changed for ERP data sources, reconstructs data warehouse storage framework;
The true table of customization resets component, for when user ETL schemes or the virtual reorganization scheme of dimension change, reconstruct to be fixed The true table of system;
The data permission control member, during for exporting the true table of customization to BI demonstration tools, authorizing according to user, only exporting The fact that authorized data.
10. system according to claim 9, it is characterised in that:The data warehouse framework layout structure, also with as follows Function:
The database table structure of the database on specified database server is assessed, according to the storage organization feature recognition ERP of database Systems provider and product type, it is automatic to build ERP data source lists, in case ERP user configurings;
The ERP data sources refer to the various information Management System Data storehouse of enterprises and institutions, and above- mentioned information management system includes Financial system, supply chain, human resources, the manufacturing, COST system;
For the different ERP system products of each ERP system supplier, the data of the supporting product in pretreatment ETL components are called ETL methods, carry out data prediction.
CN201710227461.6A 2017-03-27 2017-04-10 A kind of packaging method and system of business intelligence ETL Active CN106991183B (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN2017101892391 2017-03-27
CN201710189239 2017-03-27

Publications (2)

Publication Number Publication Date
CN106991183A true CN106991183A (en) 2017-07-28
CN106991183B CN106991183B (en) 2019-09-06

Family

ID=59415907

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201710227461.6A Active CN106991183B (en) 2017-03-27 2017-04-10 A kind of packaging method and system of business intelligence ETL

Country Status (1)

Country Link
CN (1) CN106991183B (en)

Cited By (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107358376A (en) * 2017-08-25 2017-11-17 袁也 Corporation finance system and method based on performance management and financial analysis
CN107633094A (en) * 2017-10-11 2018-01-26 江苏神州信源系统工程有限公司 The method and apparatus of data retrieval in a kind of cluster environment
CN107943863A (en) * 2017-11-09 2018-04-20 北京许继电气有限公司 Agile Modeling method and system based on business intelligence
CN110069561A (en) * 2019-04-29 2019-07-30 金瓜子科技发展(北京)有限公司 Account acquisition methods, system, electronic equipment and computer-readable medium
CN110209422A (en) * 2018-05-09 2019-09-06 腾讯科技(深圳)有限公司 A kind of method for processing business, computer equipment and client
CN110347992A (en) * 2019-07-10 2019-10-18 成都函夏科技有限公司 Data analysing method and system based on electronic report forms
CN111295651A (en) * 2017-09-29 2020-06-16 甲骨文国际公司 System and method for weighting multiple parents in a multidimensional database environment
CN112488580A (en) * 2020-12-18 2021-03-12 江苏苏宁云计算有限公司 Intelligent dynamic dimension ordering method and system based on multi-dimensional characteristic parameters
CN112764788A (en) * 2021-01-19 2021-05-07 南京大学 Software dynamic update hot patch synthesis method based on program source code slice recombination
CN113254546A (en) * 2021-05-24 2021-08-13 北京无线电测量研究所 Supply chain management and control system and method and electronic equipment
CN113361126A (en) * 2021-06-24 2021-09-07 浪潮软件科技有限公司 Business modeling analysis method based on big data
CN117273400A (en) * 2023-11-21 2023-12-22 领先未来科技集团有限公司 Enterprise resource planning intelligent merging and upgrading early warning method and system
CN117785984A (en) * 2024-02-28 2024-03-29 广州思迈特软件有限公司 Data extraction method, device, electronic equipment and storage medium

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101710280A (en) * 2009-11-05 2010-05-19 金蝶软件(中国)有限公司 Packaging method and devices of BI product members
CN103093322A (en) * 2013-02-21 2013-05-08 用友软件股份有限公司 System and method for impromptu analyzing business data
CN105787059A (en) * 2016-02-29 2016-07-20 四川长虹电器股份有限公司 Data warehouse based financial data integration method

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101710280A (en) * 2009-11-05 2010-05-19 金蝶软件(中国)有限公司 Packaging method and devices of BI product members
CN103093322A (en) * 2013-02-21 2013-05-08 用友软件股份有限公司 System and method for impromptu analyzing business data
CN105787059A (en) * 2016-02-29 2016-07-20 四川长虹电器股份有限公司 Data warehouse based financial data integration method

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
冯瑞芳: "ERP与商业智能整合应用的研究", 《中国优秀硕士学位论文全文数据库 信息科技辑》 *

Cited By (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107358376A (en) * 2017-08-25 2017-11-17 袁也 Corporation finance system and method based on performance management and financial analysis
CN111295651B (en) * 2017-09-29 2023-09-26 甲骨文国际公司 System and method for weighting multiple parents in a multidimensional database environment
CN111295651A (en) * 2017-09-29 2020-06-16 甲骨文国际公司 System and method for weighting multiple parents in a multidimensional database environment
CN107633094A (en) * 2017-10-11 2018-01-26 江苏神州信源系统工程有限公司 The method and apparatus of data retrieval in a kind of cluster environment
CN107943863A (en) * 2017-11-09 2018-04-20 北京许继电气有限公司 Agile Modeling method and system based on business intelligence
CN110209422A (en) * 2018-05-09 2019-09-06 腾讯科技(深圳)有限公司 A kind of method for processing business, computer equipment and client
CN110209422B (en) * 2018-05-09 2021-08-27 腾讯科技(深圳)有限公司 Service processing method, computer equipment and client
CN110069561A (en) * 2019-04-29 2019-07-30 金瓜子科技发展(北京)有限公司 Account acquisition methods, system, electronic equipment and computer-readable medium
CN110347992A (en) * 2019-07-10 2019-10-18 成都函夏科技有限公司 Data analysing method and system based on electronic report forms
CN110347992B (en) * 2019-07-10 2024-05-14 成都函夏科技有限公司 Data analysis method and system based on electronic report
CN112488580A (en) * 2020-12-18 2021-03-12 江苏苏宁云计算有限公司 Intelligent dynamic dimension ordering method and system based on multi-dimensional characteristic parameters
CN112764788A (en) * 2021-01-19 2021-05-07 南京大学 Software dynamic update hot patch synthesis method based on program source code slice recombination
CN112764788B (en) * 2021-01-19 2023-06-16 南京大学 Software dynamic update hot patch synthesis method based on program source code slice recombination
CN113254546A (en) * 2021-05-24 2021-08-13 北京无线电测量研究所 Supply chain management and control system and method and electronic equipment
CN113361126A (en) * 2021-06-24 2021-09-07 浪潮软件科技有限公司 Business modeling analysis method based on big data
CN117273400A (en) * 2023-11-21 2023-12-22 领先未来科技集团有限公司 Enterprise resource planning intelligent merging and upgrading early warning method and system
CN117785984A (en) * 2024-02-28 2024-03-29 广州思迈特软件有限公司 Data extraction method, device, electronic equipment and storage medium

Also Published As

Publication number Publication date
CN106991183B (en) 2019-09-06

Similar Documents

Publication Publication Date Title
CN106991183B (en) A kind of packaging method and system of business intelligence ETL
US10853387B2 (en) Data retrieval apparatus, program and recording medium
US20070276755A1 (en) Systems and methods for assignment generation in a value flow environment
Tannock et al. Data-driven simulation of the supply-chain—Insights from the aerospace sector
US20120150820A1 (en) System and method for testing data at a data warehouse
CN106095942B (en) Strong variable extracting method and device
CN107729252A (en) For reducing instable method and system when upgrading software
WO2007005949A2 (en) Dispute resolution processing method and system
CN106204284A (en) The implementation method of the future payment product of a kind of pre-core insurance system and device
US20130124265A1 (en) Enterprise System/Process Modeling System and Method
Tirkel Cycle time prediction in wafer fabrication line by applying data mining methods
CN109101296B (en) Distributed automatic deduction method and system
Danilczuk et al. Computer-aided material demand planning using ERP systems and business intelligence technology
CN106779240A (en) The Forecasting Methodology and system of civil aviaton's market macroscopic view index
CN101739454A (en) Data processing system
CN113723822A (en) Power supply service data management system
Bruzzone et al. An application methodology for logistics and transportation scenarios analysis and comparison within the retail supply chain
Tsai et al. A cost-based module mining method for the assemble-to-order strategy
CN107451846A (en) The management method and managing device of business rule
CN112001539B (en) High-precision passenger transport prediction method and passenger transport prediction system
CN101546273B (en) Method for forecasting execution time of software process
US20150120397A1 (en) General enterprise modeling system and methodology for constructing enterprise applications
Singh et al. Network design for cylinder gas distribution
Insani et al. Business intelligence for profiling of telecommunication customers
KR101903530B1 (en) Optimization diagnostic system of business and IT system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant