CN201522706U - Metadata management system - Google Patents
Metadata management system Download PDFInfo
- Publication number
- CN201522706U CN201522706U CN2009201730913U CN200920173091U CN201522706U CN 201522706 U CN201522706 U CN 201522706U CN 2009201730913 U CN2009201730913 U CN 2009201730913U CN 200920173091 U CN200920173091 U CN 200920173091U CN 201522706 U CN201522706 U CN 201522706U
- Authority
- CN
- China
- Prior art keywords
- metadata
- relation
- data
- library facilities
- data library
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Lifetime
Links
Images
Abstract
The utility model provides a metadata management system which comprises a relation data base device which stores metadata from a data source and mutually exchanges data with the data source and an external operation system through a relation data base interface, a bridge device and an object data base device, wherein the bridge device obtains the synchronous mapping relation of the relation data base device and the object data base device, and synchronously stores metadata stored in the relation data base device to the object data base device according to the synchronous mapping relation, the object data base device stores metadata of the bridge device after synchronized according to the synchronous mapping relation, and mutually exchanges data with a WEB server through an object data base interface. The metadata management system can realize the compatibility with an existing international standard interface through object models, further can realize the rapid response of a data base during operation through the interface of a relational model, provides necessary metadata information during the operation of the base, and guarantees the highly-effective and stable operation of database.
Description
Technical field
The utility model is about the metadata management technology, especially in regard to a kind of metadata management system.
Background technology
Metadata management system has become the requisite ingredient of current industry enterprise-level data warehouse (Enterprise DataWarehouse EDW), can provide enough information for exploitation, operation and the management of EDW, promote the production efficiency of EDW, guarantee the stable operation of EDW.
The metadata management system that makes up data warehouse need be considered two aspects: how to realize metadata in each subsystem of data warehouse synchronously, and utilize the standard of industry and the management that instrument is carried out metadata and the analysis of metadata; How simple and high-efficient operation access interface is provided.
At present, though industry has the instrument that much is used for metadata management, the instrument of these metadata managements can provide metadata history management, metadata analysis and OO metadata access interface, to support the structure and the visit of metadatabase.Yet the metadata instrument based on industrywide standard realizes aspect the terseness of access performance and interface, can't satisfy and support that data warehouse moves this ageing very high requirement; Based on the relational model that the bivariate table of database is set up, can't carry out complicated metadata analysis, and can not with the interface compatibility of international standard.
Summary of the invention
The purpose of the utility model embodiment is to provide a kind of metadata management system, by the metadata synchronization between the dual model in the metadata management system, quick response during with realization metadata management system and existing international standard interface compatibility and data warehouse operation provides the warehouse to move necessary metadata information.
To achieve these goals, the utility model embodiment provides a kind of metadata management system, described system comprises: object data library facilities, relation data library facilities and bridge device, described bridge device connects and is connected with described object data library facilities and relation data library facilities respectively, described relation data library facilities is connected with data source and outside operational system by the relation data bank interface, and described object data library facilities is connected with the WEB server by the object data bank interface; Described relation data library facilities, the metadata that storage is obtained from data source, and carry out data interaction by relation data bank interface and data source and outside operational system; Described bridge device, the synchronization map of obtaining relation data library facilities and object data library facilities concern, and according to described synchronization map relation the metadata synchronization of storing in the described relation data library facilities are stored in the described object data library facilities; Described object data library facilities is stored the metadata after described bridge device concerns synchronously according to described synchronization map, and carries out data interaction by described object data bank interface and WEB server.
The useful technique effect of the utility model embodiment: the metadata synchronization in the metadata management system between the dual model, can realize metadata management system and existing international standard interface compatibility by object model; Quick response in the time of can also realizing the data warehouse operation by the interface of relational model provides the warehouse to move necessary metadata information, guarantees the operation of data warehouse efficient stable.
Description of drawings
In order to be illustrated more clearly in the utility model embodiment or technical scheme of the prior art, to do one to the accompanying drawing of required use in embodiment or the description of the Prior Art below introduces simply, apparently, accompanying drawing in describing below only is embodiment more of the present utility model, for those of ordinary skills, under the prerequisite of not paying creative work, can also obtain other accompanying drawing according to these accompanying drawings.
Figure 1A is the application synoptic diagram of prior art metadata management system;
Figure 1B is the application synoptic diagram of the utility model embodiment metadata management system;
Fig. 2 A is the structural representation of the utility model embodiment metadata management system 102;
Fig. 2 B is the structural representation of the utility model embodiment bridge device 203;
Fig. 3 is the metadata management method process flow diagram of the utility model embodiment metadata management system;
Fig. 4 is the utility model embodiment metadata management system and the structural representation that is connected of external system;
Fig. 5 is the functional schematic of the utility model embodiment metadata Runtime Library device 401;
Fig. 6 is the functional schematic of the utility model embodiment metadata knowledge base device 401;
Fig. 7 is the structural representation of the utility model embodiment metadata bridge device 403;
Fig. 8 is a synchronous overview flow chart between two models of the utility model embodiment;
Fig. 9 is the synchronous process flow diagram of the isolated object of the utility model embodiment;
Figure 10 is the synchronous process flow diagram of the utility model examples of implementation object;
Figure 11 is the synchronous process flow diagram of parallel relation between the utility model embodiment object;
Figure 12 is the concrete form synoptic diagram of the utility model embodiment XML file;
Figure 13 is the synoptic diagram of the concrete configuration file of the utility model embodiment.
Embodiment
Below in conjunction with the accompanying drawing among the utility model embodiment, the technical scheme among the utility model embodiment is clearly and completely described, obviously, described embodiment only is the utility model part embodiment, rather than whole embodiment.Based on the embodiment in the utility model, those of ordinary skills are not making the every other embodiment that is obtained under the creative work prerequisite, all belong to the scope of the utility model protection.
Figure 1A is the application synoptic diagram of prior art metadata management system.As shown in the figure, the metadata management system 102 in the data warehouse comprises object data library facilities 104, and metadata management system 102 obtains the metadata that need to load from data source 101, and data source can be the document of exploitation and data dictionary etc.Metadata management system 102 with the metadata store obtained in object data library facilities 104; Object data library facilities 104 provides international standard interface to the metadata analysis application system of metadata management person 105 and metadata user 106 operations, metadata management person 105 and metadata user 106 are by international standard interface Operand data, for example, metadata user 106 can carry out metadata information by international standard interface and browse and (comprise that theme browses, the TEL information browse, DQ information browse and framework are browsed etc.) and network service such as metadata information analysis, as vigor analysis (accessed number of times), life cycle analysis (analyzing the source and the destination of data), impact analysis (data variation is to the influence in data warehouse) etc.
The outside operational system of metadata management system 102 (as data quality check DQ system 109, ETL dispatching system 110 and external data loading system), outside operational system is shown in the frame of broken lines among the figure, the source system data that refers to ETL extracts (Extraction), conversion (Trasfromation), loads the process of (Loading) in data warehouse, also be used for representing the subsystem of data warehouse data load conversion.Outside operational system can't be by international standard interface to object data library facilities 104 visit datas, but directly obtain data from metadata sources, can't satisfy the synchronisation requirement of metadata.
Figure 1B is the application synoptic diagram of the utility model embodiment metadata management system, as shown in the figure, metadata management system 102 in the data warehouse obtains the metadata that need to load from data source 101, with the metadata store obtained in relation data library facilities 103; Relation data library facilities 103 provides service interface 108 for the operational system (as data quality check DQ system 109, ETL dispatching system 110 and external data loading system) in the data warehouse, put forward metadata with the operational system in data warehouse, outside operational system is shown in the frame of broken lines among the figure.
Fig. 2 A is the structural representation of the utility model embodiment metadata management system 102.As shown in the figure, described metadata management system 102 comprises: relation data library facilities 201, object data library facilities 202, and bridge device 203, described bridge device 203 connects and is connected with described object data library facilities 202 and relation data library facilities 201 respectively, described relational database 201 equipment are connected with data source and outside operational system 204 by the relation data bank interface, and described object data library facilities is connected with WEB server 205 by the object data bank interface; Described relation data library facilities 201, the metadata that storage is obtained from data source, and carry out data interaction by relation data bank interface and data source 206 and outside operational system 205; Described bridge device 203, obtain the synchronization map relation of relation data library facilities 201 and object data library facilities 202, and the metadata synchronization of storing in the described relation data library facilities is stored in the described object data library facilities 202 according to described synchronization map relation; Described object data library facilities 202 is stored the metadata after described bridge device 203 concerns synchronously according to described synchronization map, and carries out data interaction by described object data bank interface and WEB server.Described object data bank interface is the metadata interface of international standard.The data warehouse runnable interface that described relation data bank interface is an international standard.Described outside operational system comprises: quality of data check system and ETL system etc.
Described former data management system 102 can also comprise: the Map Profile memory device, be connected with described bridge device 203, and storage comprises the Map Profile of described synchronization map relation.The Map Profile that comprises described synchronization map relation also can be stored in the bridge device 203.Described bridge device is resolved the synchronization map relation in the described Map Profile, and reads the described synchronization map relation of parsing.
Described synchronization map relation comprises: the synchronization map relation of the object identity of the synchronization map relation of the field of the synchronization map relation of the class of the table of described relation data library facilities 201 and described object data library facilities 202, described relation data library facilities 201 and the generic attribute of described object data library facilities 202 and the major key of described relation data library facilities 201 and described object data library facilities 202.
Between relational database and the object database metadata can comprise synchronously isolated object synchronously, synchronous three kinds of parallel relation synchronously and between object of father and son Guan Xiang between object.
Whether 1) isolated object is synchronous: shown in Fig. 2 B, described bridge device 203 comprises: object is searched device 207, search to exist in the described object data library facilities according to described synchronization map relation and treat synchronization object; Object updating device 208 when existing in the described object data library facilities when treating synchronization object, upgrades the described synchronization object for the treatment of according to the synchronous recording for the treatment of of the metadata of storing in the described relation data library facilities; Object new device 209 when not existing in the described object data library facilities when treating synchronization object, is treated synchronous recording new Object in described object data library facilities according to the metadata of storing in the described relation data library facilities.
2) between object father and son Guan Xiang synchronously: described object is searched device 207 and is closed to tie up in the described object data library facilities according to described synchronization map and search the parent object for the treatment of synchronization object; Search whether there is the described synchronization object for the treatment of in the described object data library facilities according to described parent object that finds and the described attribute of synchronization object for the treatment of; Described when treating synchronization object when existing in the described object data library facilities, described object updating device 208 upgrades the described synchronization object for the treatment of according to the synchronous recording for the treatment of of the metadata of storing in the described relation data library facilities; Described when treating synchronization object when not existing in the described object data library facilities, described object new device 209 is treated synchronous recording new Object in described object database device according to the metadata of storing in the described relation data library facilities.Described bridge device also comprises and concerns apparatus for establishing 210, sets up the set membership of described parent object and described new Object in described object data library facilities.
3) between object parallel relation synchronously: described object is searched device 207, closes to tie up to search in the described object data library facilities 202 according to described synchronization map and treats the synchronisation source object; Close to tie up to search in the described object data library facilities 202 according to described synchronization map and treat synchronous destination object; The described apparatus for establishing 210 that concerns is set up and is describedly treated the synchronisation source object and treat parallel relation between the synchronous destination object.
Fig. 3 is the metadata management method process flow diagram of the utility model embodiment metadata management system, and as shown in the figure, described method comprises:
Step S301: obtain metadata from data source.
Step S302: with described metadata store in relational database.
Step S303: the synchronization map relation of setting up described relational database and object database; Described synchronization map relation comprises: the synchronization map relation of the object identity of the synchronization map relation of the field of the synchronization map relation of the table of described relational database and the class of object database, described relational database and the generic attribute of object database and the major key of described relational database and object database.Synchronization map is closed and is tied up in the Map Profile of metadata management system.
Step S304: the metadata synchronization of storing in the described relational database is stored in the described object database according to described synchronization map relation.
Step S305: the metadata and the outside operational system of described relation data library storage are carried out data interaction.
Step S306: the metadata and the outside WEB server of described object data library storage are carried out data interaction.
Whether step S304 can comprise: search to exist in the described object database according to described synchronization map relation and treat synchronization object; If upgrade the described synchronization object for the treatment of according to the synchronous recording for the treatment of of the metadata of storing in the described relational database; Otherwise, treat synchronous recording new Object in described object database according to the metadata of storing in the described relational database.
Step S304 can also comprise: close to tie up to according to described synchronization map and search the parent object for the treatment of synchronization object in the described object database; Search whether there is the described synchronization object for the treatment of in the described object database according to described parent object that finds and the described attribute of synchronization object for the treatment of; If upgrade the described synchronization object for the treatment of according to the synchronous recording for the treatment of of the metadata of storing in the described relational database; Otherwise, according to the metadata of storing in the described relational database treat synchronous recording new Object in described object database, and set up the set membership of described parent object and described new Object.
Step S304 can also comprise: close to tie up to search in the described object database according to described synchronization map and treat the synchronisation source object; Close to tie up to search in the described object database according to described synchronization map and treat synchronous destination object; Set up and describedly treat the synchronisation source object and treat parallel relation between the synchronous destination object.
Below in conjunction with metadata management system embodiment of the present utility model is described.Metadata Runtime Library device 401 is used to realize the function of relation data library facilities, and metadata knowledge base device 402 is used to realize the function of object knowledge base equipment.
Fig. 4 is the utility model embodiment metadata management system and the structural representation that is connected of external system, as shown in the figure, this system comprises two set of model: metadata knowledge base device 402 is based on the OO meta-model of MDS and the metadata Runtime Library device 401 conventional two-dimensional relationship type meta-model based on database, and MDS is the metadata engine Metadata Service with the international standard compatibility.
Metadata knowledge base device 402 is carried out the design of meta-model and the structure and the management of metadatabase based on the MDS instrument with the industrywide standard compatibility; Interface when metadata Runtime Library device 401 provides the data warehouse operation by traditional relevant database model; Metadata Runtime Library device 401 is connected by metadata bridge device 403 with metadata knowledge base device 402, and metadata bridge device 403 carries out the synchronous of content metadata between two set of model of metadatabase inside.
As shown in Figure 4, metadata Runtime Library device 401 is connected in data warehouse subsystem 404 and provides functional interface for each subsystem 404 of data warehouse (outside operational system), simply still ageing requirement is very high for the metadata claimed structure of being visited for data warehouse subsystem 404, and the interface of data warehouse subsystem 404 is not followed international standard, and data warehouse subsystem 404 can be data quality management system, ETL dispatching system and the external data loading system etc. of data warehouse inside.Metadata knowledge base device 402 is connected in metadata analysis application system 405 (WEB server), for metadata analysis application system 405 provides functional interface, the subsystem of international standard is used and followed to the more corresponding senior metadata of metadata analysis application system 405, as require to metadata carry out the data warehouse of setting analysis processing the front end gate system, adopt flexible inquiry subsystem that instruments such as Cognos or SAS realize etc.
Metadata Runtime Library device 401 is based on the metadatabase that conventional model makes up, is used for from data warehouse subsystem 404 synchronizing metadatas, and the interface when providing visit operation metadata for data warehouse subsystem 404; Similarly, metadata knowledge base device 402 is based on the metadatabase that OO meta-model makes up, and by the metadata interface of international standard, is used for providing metadata access to metadata analysis application system 405.Then carry out data sync between metadata Runtime Library device 401 and the metadata knowledge base device 402 by metadata bridge device 403.
As shown in Figure 5, what metadata Runtime Library device 401 was deposited is basic metadata, its conversion by data base view layer 501 becomes to satisfy the metadata of other system application need with meta data conversion, provides metadata to data warehouse subsystems such as ETL Runtime Library 502, DQ repository 503 and ETL loading procedures 504.Data base view layer 501 is to use and demand the set of a series of data base views that define on basic metadata according to different metadata.Because the versatility of meta-model and the metadata of depositing is basic, as the functions expanding or the improvement of other subsystems later on, the interface that can provide by the data base view layer from the metadatabase of conventional model easily equally obtains required metadata information.By above mode, provide quick, flexible, extendible metadata service based on the metadatabase of conventional model for the operation of data warehouse, given full play to the advantage of conventional model.
As shown in Figure 6, metadata knowledge base device 402 provides the query function and the analytic function of various metadata by user inquiring interface and API.Wherein the metadata of depositing in 601 pairs of metadata knowledge bases 402 of metadata analysis engine apparatus is searched, gathers, is compared, and forms the analysis result that satisfies particular requirement, returns to user inquiring interface 602.The user inquiring interface provides one group of application function interface, the function that provides comprises that metadata impact analysis (analyzing the influence of a metadata object to downstream objects downwards), consanguinity analysis (opposite with the direction of impact analysis, as upwards to review the Data Source of an object), vigor analysis (analyzing the frequency that database object is accessed), orphan analyze (analyzing the orphan's metadata object among data area in preparation (SData), physical model district (PDara)) and metadata is browsed and inquiry etc.The user can browse with Query Result 607 by user inquiring interface 602 inquiry impact analysis results 603, consanguinity analysis result 604, vigor analysis result 605, orphan's analysis result 606 and metadata etc.
Data teletorque system between 402 two models of metadata Runtime Library device 401 and metadata knowledge base device is based on the basis of the metadata management system of dual model.Finish data sync by metadata bridge device 403 between two models.The structure of metadata bridge device 403 as shown in Figure 7, metadata bridge device 403 comprises: Map Profile memory module 701 (Map Profile memory storage), Command Line Parsing module 702, database access module 703 and data simultaneous module 704 are formed.Map Profile memory module 701 can also be in metadata bridge device 403.
In the operational process of metadata management system, the 103 automatic realizations of relation data library facilities are synchronous to object data library facilities 104.Synchronous overall procedure is as shown in Figure 8 between 702 two models of metadata Runtime Library device 401 and metadata knowledge base device:
Step S801: the Map Profile that Command Line Parsing module 702 is resolved in the Map Profile memory module 701, obtain the synchronization map relation;
Step S802: data simultaneous module 704 reads the synchronization map relation that Command Line Parsing module 702 is obtained from Command Line Parsing module 702; The synchronization map relation of the description in the configuration file comprises 3 kinds of corresponding relations, as shown in Table 1.Can finish the data synchronization operation according to 3 kinds of corresponding relations in the table one.
Table one
Step S803: data simultaneous module 704 reads metadata information in the metadata Runtime Library device 401 according to the synchronization map related key with database access module 403; Metadata information is need carry out synchronous record in the table of metadata Runtime Library device 401, and these records need synchronously the object to metadata knowledge base device 402.Metadata information comprises information such as script and data structure.
Step S804: data simultaneous module 704 with the metadata information conversion synchronization to metadata knowledge base device 402.
In the object model in metadata knowledge base device 402, the mode of descriptive metadata comprises: set membership between class, object, object and the parallel relation between object; " class " defined all and had the set of the object of like attribute at conceptive " table " that is equivalent in the relational model; " object " defined the unique identification and all properties of a data entity at conceptive " record " that is equivalent in the relational model; In object model, can be related in twos between the object by relation, the characteristics of the set membership between object are that the precondition that subobject exists is that parent object must exist; Parallel relation between object is meant the object with parallel relation, can independently separately exist.
Be based on the synchronization map relation that Runtime Library is recorded to data object synchronously between 402 two models of metadata Runtime Library device 401 and metadata knowledge base device, so the method for synchronization can reduce following three types:
1. write down synchronously to object in the table.Whether depend on other objects according to object and exist, the method for synchronization can be divided into isolated object synchronously and subobject synchronous.
2. record is synchronous to the set membership between object in the table.Exist because subobject must depend on parent object, promptly do not have isolated subobject, when the establishment of subobject so the set membership between object must be finished automatically.
3. record is synchronous to the parallel relation between object in the table.Between object parallel relation synchronously, need by two objects of source and target in the anchored object model respectively of the information in the record, and between them opening relationships.
Below respectively to isolated object synchronously, the describing synchronously of parallel relation synchronously and between object of subobject.
Fig. 9 is the synchronous process flow diagram of the isolated object of the utility model embodiment; As shown in Figure 9, described flow process comprises:
Step S1001: Command Line Parsing module 702 is resolved the mapping configuration, and reads configuration information by data simultaneous module 704.Configuration information has been described the synchronization map relation from the data-base recording to the repository object, promptly how metadata Runtime Library device 401 is write down the object that converts knowledge base to, and synchronization map concerns as shown in Table 1.
Step S1002: data simultaneous module 704 is called database access module 703 and is read metadata information in the metadata Runtime Library device 401.
Step S1003: data simultaneous module 704 is searched in metadata knowledge base device 402 needs synchronous object.Data simultaneous module 704 is the unique identification of the major key information in the metadata information as object, closes to tie up to according to synchronization map and searches the synchronous object of needs in the metadata knowledge base device 402.
Step S1004: judging whether to find needs synchronous object; Need synchronous object if find, carry out step S1005, otherwise, carry out step S1006.
Step S1005: data simultaneous module 704 is upgraded needs synchronous object.The attribute of upgating object is as the algorithm of field, length and type in the new data structure more and script and processing object etc.
Step S1006: new Object in metadata knowledge base device 402, and the corresponding object attribute is set according to the recorded content that is read.
Behind the same EOS of isolated object, release synchronous flow process.
Figure 10 is the synchronous process flow diagram of the utility model examples of implementation object; As shown in figure 10, described synchronous flow process comprises:
Step S1101: Command Line Parsing module 702 is resolved the mapping configuration, and reads configuration information by data simultaneous module 704.
Configuration information has been described the synchronization map relation from the data-base recording to the repository object, promptly how metadata Runtime Library device 401 is write down the object that converts knowledge base to, and synchronization map concerns as shown in Table 1.
Synchronously the configuration information during subobject has been Duoed the description of parent object when isolating object synchronously.Exist because subobject can only depend on parent object, should find corresponding parent object earlier when therefore synchronous.
Step S1102: data simultaneous module 704 is called database access module 703 and is read metadata information in the metadata Runtime Library device 401, metadata information is need carry out synchronous record in the table of metadata Runtime Library device 401, these records need synchronous object to metadata knowledge base device 402, comprise the descriptor of parent object, subobject.Metadata information comprises information such as script and data structure.
Step S1103: data simultaneous module 704 is according to configuration information and metadata information, and searching is whether to have the parent object that needs synchronization object in the metadata knowledge base device 402; If, carry out step S1104, otherwise warning prompt and withdraw from synchronous flow process.
Data simultaneous module 704 ties up to the parent object of searching the needs synchronization object in the metadata knowledge base device 402 with the unique identification of the major key information in the metadata information as object according to the synchronization map pass.
Step S1104: search whether there is subobject in the metadata knowledge base device 402 by parent object and subobject attribute.If carry out step S1105, otherwise carry out step S1106.
Step S1105: data simultaneous module 704 is upgraded subobject.The attribute of upgating object is as the algorithm of field, length and type in the new data structure more and script and processing object etc.
Step S1106: new Object in metadata knowledge base device 402.
Step S1107: final data synchronization module 704 is set up set membership with newly-built object and parent object;
Step S1108: withdraw from synchronous flow process.
When carrying out parallel relation between synchronization object synchronous, require each that two objects that relation relates to are all existed in metadata knowledge base device 402, synchronization program finds from metadata knowledge base device 402 after these two objects according to the object id information in the metadata Runtime Library device 401, in metadata knowledge base device 402 they is set up parallel relation again.
Figure 11 is the synchronous process flow diagram of parallel relation between the utility model embodiment object; As shown in figure 11, described synchronous flow process comprises:
Step S1201: Command Line Parsing module 702 is resolved the mapping configuration, and reads configuration information by data simultaneous module 704.
Configuration information has been described the synchronization map relation from the data-base recording to the repository object, promptly how metadata Runtime Library device 401 is write down the object that converts knowledge base to, and synchronization map concerns as shown in Table 1.
Step S1202: data simultaneous module 704 is called database access module 703 and is read metadata information in the metadata Runtime Library device 401.Metadata information is need carry out synchronous record in the table of metadata Runtime Library device 401, and these records need synchronously the object to metadata knowledge base device 402.Metadata information comprises information such as script and data structure.
Step S1203: data simultaneous module 704 is searched the source object in the parallel relation in metadata knowledge base device 402.
Step S1204: judge whether source object exists, if, carry out step S1205, otherwise warning prompt and withdraw from synchronous flow process.
Step S1205: data simultaneous module 704 continues to search the destination object in the parallel relation in metadata knowledge base device 402.
Step S1206: judge whether destination object exists, if, carry out step S1207, otherwise warning prompt and withdraw from synchronous flow process.
Step S1207: synchronization module 504 is set up the relation between object in the object model of metadata knowledge base device 402.
Step S1208: withdraw from synchronous flow process.
Configuration file can be the file of various ways, as XML, Excel etc., but must comprise Runtime Library record and the data map of object and the two-part content of data map of data-base recording and object association relation.
Below with the form of XML file, illustrate that configuration file describes the method and the element of map information.Figure 12 is the concrete form synoptic diagram of the utility model embodiment XML file, and as shown in figure 12, the root node of XML is＜BridgeConf 〉, under be to describe synchronization map relation＜Mapping node.Root node can comprise a plurality of＜Mapping〉node.＜Mapping〉node is divided into＜EntityMap〉and＜RelationMap〉two parts, be entity mapping part and relationship map part, respectively with data entity in traditional meta-model of bivariate table form and the relation between the entity, convert object and relation between objects to by mapping, again synchronously to the MDS metadatabase, according to the storage mode storage of MDS.
As shown in figure 12,＜EntityMap〉node attribute description the essential information of object data mapping: name represents to shine upon title, is used for identifying a mapping node; TableName represents the source table name in the synchronization map relational model; ClassName represents the class name in the mapping object model; Filter represents the record screening conditions in the table of source.＜EntityMap〉element that comprises of node described the details of record and object map: the srccolumn of node ColumnMap and destprop attribute have been described one group of corresponding relation of field and object properties in the table respectively; Node ParentColumns is used to carry out the mapping of subobject.In three attributes, RelationName has identified the title that concerns of set membership; ClassName has identified the class name under the parent object; Column has identified the field name that is used to locate parent object.By above information, can navigate to the parent object of current synchronous object.
＜RelationMap〉node attribute description the essential information name of Object Relation Mapping represent to shine upon title, be used for identifying a mapping node; TableName represents the source table name in the synchronization map relational model; RelName represents the title that concerns in the mapping object model; Filter represents the record screening conditions in the table of source.＜RelationMap〉element that comprises of node described the details of record and Object Relation Mapping: the source object during Origin represents to concern.In two attributes, class represents the class name under the source object; Column represents to be used to locate the field name of this object; Destination object during Dest represents to concern.In two attributes, class represents the class name under the destination object; Column represents to be used to locate the field name of this object.
Figure 13 is the synoptic diagram of the concrete configuration file of the utility model embodiment.The table definition information of having described origin system how from relational model synchronously to object model.＜EntityMap〉attribute of node and it informs that SOURCE_TABLE (deposit the tables of data of the origin system table information) table of bridge from the metadata Runtime Library is with the SourceTable class of data sync to the metadata knowledge base.A plurality of＜ColumnMap〉how the node field of having described the SOURCE_TABLE table be mapped with the attribute of SourceTable class.Because the SourceTable class to as if subobject, must description how to find its parent object.＜ParentColumn〉node and attribute description thereof this set membership and the parent object that how finds current object by the databasesys field.
The technique effect of the utility model embodiment: the metadata synchronization in the metadata management system between the dual model, can realize metadata management system and existing international standard interface compatibility by object model; Quick response in the time of can also realizing the data warehouse operation by the interface of relational model provides the warehouse to move necessary metadata information, guarantees the operation of data warehouse efficient stable.Compared with prior art:
The utility model embodiment supports ETL with metadata: build the predicative sentence by what metadata directly generated temporary area; Carry out the flexible configuration that ETL loads by metadata, the variation that makes up-stream system minimizes the influence of EDW.Metadatabase provides the information such as bag, interface, play, filename, file field, loading mode of up-stream system for ETL, increasing and revising interface only needs directly that the modification metadata gets final product; By metadata information, ETL automatically generated data loading scripts has improved the dirigibility that ETL loads; Metadata has realized the accuracy and the consistance of content by supporting ETL; By metadata information, provide information such as ETL task, ETL task dependence;
The utility model embodiment supports the inspection of the EDW quality of data with metadata: by the interface message that metadatabase is preserved, the interface check task of automatically generated data quality check module; By entity major key information and the entity associated information that metadatabase is preserved, the entity integrity of automatically generated data quality check module is checked task; By the data dictionary information that metadatabase is preserved, the code check task of automatically generated data quality check module.
Above-described embodiment; the purpose of this utility model, technical scheme and beneficial effect are further described; institute is understood that; the above only is an embodiment of the present utility model; and be not used in and limit protection domain of the present utility model; all within spirit of the present utility model and principle, any modification of being made, be equal to replacement, improvement etc., all should be included within the protection domain of the present utility model.
Claims (8)
1. metadata management system, it is characterized in that, described system comprises: object data library facilities, relation data library facilities and bridge device, described bridge device connects and is connected with described object data library facilities and relation data library facilities respectively, described relation data library facilities is connected with data source and outside operational system by the relation data bank interface, and described object data library facilities is connected with the WEB server by the object data bank interface; Wherein
Described relation data library facilities, the metadata that storage is obtained from data source, and carry out data interaction by relation data bank interface and data source and outside operational system;
Described bridge device, the synchronization map of obtaining relation data library facilities and object data library facilities concern, and according to described synchronization map relation the metadata synchronization of storing in the described relation data library facilities are stored in the described object data library facilities;
Described object data library facilities is stored the metadata after described bridge device concerns synchronously according to described synchronization map, and carries out data interaction by described object data bank interface and WEB server.
2. the system as claimed in claim 1 is characterized in that, described system also comprises:
The Map Profile memory device is connected with described bridge device, and storage comprises the Map Profile of described synchronization map relation.
3. the system as claimed in claim 1, it is characterized in that described synchronization map relation comprises: the synchronization map relation of the object identity of the synchronization map relation of the field of the synchronization map relation of the class of the table of described relation data library facilities and described object data library facilities, described relation data library facilities and the generic attribute of described object data library facilities and the major key of described relation data library facilities and described object data library facilities.
4. the system as claimed in claim 1 is characterized in that, described bridge device comprises:
Object is searched device, closes to tie up in the described object data library facilities according to described synchronization map and searches the parent object for the treatment of synchronization object; Search whether there is the described synchronization object for the treatment of in the described object data library facilities according to described parent object that finds and the described attribute of synchronization object for the treatment of;
The object updating device, described when treating synchronization object when existing in the described object data library facilities, upgrade the described synchronization object for the treatment of according to the synchronous recording for the treatment of of the metadata of storing in the described relation data library facilities;
The object new device, described when treating synchronization object when not existing in the described object data library facilities, described object new device is treated synchronous recording new Object in described object database device according to the metadata of storing in the described relation data library facilities.
5. system as claimed in claim 4 is characterized in that, described bridge device also comprises:
Concern apparatus for establishing, in described object data library facilities, set up the set membership of described parent object and described new Object.
6. the system as claimed in claim 1 is characterized in that, described object data bank interface is the metadata interface of international standard.
7. the system as claimed in claim 1 is characterized in that, the data warehouse runnable interface that described relation data bank interface is an international standard.
8. the system as claimed in claim 1 is characterized in that, described outside operational system comprises: quality of data check system and external data loading system.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2009201730913U CN201522706U (en) | 2009-09-09 | 2009-09-09 | Metadata management system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2009201730913U CN201522706U (en) | 2009-09-09 | 2009-09-09 | Metadata management system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN201522706U true CN201522706U (en) | 2010-07-07 |
Family
ID=42508884
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2009201730913U Expired - Lifetime CN201522706U (en) | 2009-09-09 | 2009-09-09 | Metadata management system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN201522706U (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102467529A (en) * | 2010-11-11 | 2012-05-23 | 金蝶软件（中国）有限公司 | Metadata synchronizing method and system |
| CN104573107A (en) * | 2015-02-02 | 2015-04-29 | 国家计算机网络与信息安全管理中心江苏分中心 | Network security application NoSQL database and relational database fusion interface method |
| CN105447057A (en) * | 2014-09-28 | 2016-03-30 | 克拉玛依红有软件有限责任公司 | Model transforming method and system based on business logic model |
| CN109947739A (en) * | 2018-05-31 | 2019-06-28 | 新华三大数据技术有限公司 | Data power supply management method and device |
-
2009
- 2009-09-09 CN CN2009201730913U patent/CN201522706U/en not_active Expired - Lifetime
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102467529A (en) * | 2010-11-11 | 2012-05-23 | 金蝶软件（中国）有限公司 | Metadata synchronizing method and system |
| CN102467529B (en) * | 2010-11-11 | 2013-08-21 | 金蝶软件（中国）有限公司 | Metadata synchronizing method and system |
| CN105447057A (en) * | 2014-09-28 | 2016-03-30 | 克拉玛依红有软件有限责任公司 | Model transforming method and system based on business logic model |
| CN105447057B (en) * | 2014-09-28 | 2019-03-22 | 克拉玛依红有软件有限责任公司 | Model transform method and system based on service logic model |
| CN104573107A (en) * | 2015-02-02 | 2015-04-29 | 国家计算机网络与信息安全管理中心江苏分中心 | Network security application NoSQL database and relational database fusion interface method |
| CN109947739A (en) * | 2018-05-31 | 2019-06-28 | 新华三大数据技术有限公司 | Data power supply management method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102023979B (en) | Meta-data management method and system | |
| US9646030B2 (en) | Computer-readable medium storing program and version control method | |
| US5745895A (en) | Method for association of heterogeneous information | |
| US20120210298A1 (en) | Locating changes in source code | |
| CN105095320A (en) | System for identifying, correlating, searching and displaying documents based on relationship superposition and combination | |
| CN101840719A (en) | Reproduction apparatus and/or method for storage medium including metadata | |
| CN101681366A (en) | A kind of data storage and management method | |
| JP4911438B2 (en) | Operation monitoring device | |
| CN102893281A (en) | Information retrieval device, information retrieval method, computer program, and data structure | |
| CN101882135B (en) | Data processing method and device | |
| CN110046168A (en) | A kind of incremental data consistency implementation method and device | |
| CN104598550B (en) | A kind of update method and device of Internet video index | |
| CN110109910A (en) | Data processing method and system, electronic equipment and computer readable storage medium | |
| CN201522706U (en) | Metadata management system | |
| CN100452034C (en) | Method and device for handling metadata | |
| CN103914488B (en) | The collection of document, the system identifying, associate, search for and representing | |
| CN106682003A (en) | Path segmentation mapping method and device for distributed memory namespace | |
| CN106055546A (en) | Optical disk library full-text retrieval system based on Lucene | |
| CN106155832A (en) | Method, device and the Android device that a kind of data are recovered | |
| CN103914486B (en) | Document search and display system | |
| CN101430685B (en) | Downloading method and system | |
| CN107526746A (en) | The method and apparatus of management document index | |
| CN102810115A (en) | Method for implementing multi-layer distributed document management system | |
| Wah et al. | Building data warehouse | |
| WO2010061535A1 (en) | Information search device, information search method, and storage medium containing program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CX01 | Expiry of patent term | ||
| CX01 | Expiry of patent term |
Granted publication date: 20100707 |