KR20100058445A - Automatic extracting method of heterogeneous metadata by using rule-based technology and system thereof - Google Patents

Abstract

PURPOSE: An automatic heterogeneous metadata extracting method using a rule-based scheme and a system thereof are provided to perform extraction, storage and conversion of metadata automatically by connecting and extracting multi-software tool information of multi-OS(Operating System) environment easily. CONSTITUTION: A rule database(160) stores a server fixing rule, a metadata type fixing rule, a property registration rule for each type, a data mapping rule and a transmission rule. The property registration rule for each type designates a property related to a table about metadata for each metadata type. A metadata collector(100) is provided with regulations for collection of different kind metadata, extracts the different kind metadata from a raw table(10) which the different kind metadata exists, and transmits to a staging zone(210) of a repository.

Description

Automatic extracting method of heterogeneous metadata by using rule-based technology and system approximately}

The present invention relates to a heterogeneous metadata automatic extraction system and method using a rule-based technology, and more particularly, to easily connect and extract various software tool information of multiple operating system environments that are sources of metadata. will be.

Metadata repository (repository) is a database that stores information related to application development. It is a repository that stores metadata, which is the physical and logical information of data used in the information system. Management must be managed in an efficient and economic way according to the business value of each step.

To this end, the repository should support the Information Lifecycle Management (ILM) system, which is a comprehensive information management mechanism that supports all the policies, standard procedures and organizational information related to organizational information management.

This information lifecycle management (ILM) system is a solution that provides maximum value at the lowest cost at all stages of the overall lifecycle, including the creation, utilization, storage, backup, and deletion of information. It is a system that manages layered storage according to information life cycle by dividing high and low information. For example, by storing high-value information in high-performance storage and low-value information in low-cost storage, linking network storage technology with storage platforms, software, and services to make information more efficient and stable. To manage, protect and share.

In order to load such metadata into the target table of the repository, metadata must be collected from each source table belonging to various operating system environments, and the collected metadata are temporarily stored in the staging area. The converter must then convert the data and standardize it into a form that can be loaded into the repository.

However, since the metadata of the source table is heterogeneous metadata that is operated and managed individually by various software tools of various operating system environments, one metadata collector or software can extract these heterogeneous metadata by linking them. Therefore, in the case of heterogeneous metadata having different types of data, there is a problem that data extraction, verification, loading, etc. cannot be performed automatically.

On the other hand, rule-based technology expresses a company's business rules or procedures and contents, user's knowledge or experience, interests, preferences, etc. in a predetermined rule-based query statement. It is a technology to obtain information profiles of companies or users by using the results of the statements. For example, questions such as postal codes, types of computers used, hobbies, and preferences for certain things are asked based on predesigned rules, and the information profiles are obtained through these questions.

The advantage of this rule-based technology is that it only modifies the rules that contain the business logic or the control logic for the contents and contents, so that a new information profile can be easily obtained. Even so, it is much simpler and more flexible than the existing method.

Therefore, the use of a new technology that can extract heterogeneous metadata of multiple operating system environments by using a single metadata collector or software by using the simple and flexible correspondence of the rule-based technology.

The problem to be solved by the present invention is to easily extract and link the information of the various software tools of the various operating system environment that is the source of metadata, metadata extraction from the source table, temporary storage and conversion in the staging area, repository It is to provide a heterogeneous metadata automatic extraction system and method using a rule-based technology to automatically standardized metadata loading process into a table.

According to an embodiment of the present invention for achieving the above object, in the system for extracting heterogeneous metadata from the source table and loading it into the repository through standardization by metadata type, server designation for server designation Rules, metadata type specification rules for specifying metadata types, type-specific property registration rules for specifying tables and related attributes for each metadata type, data mapping rules for specifying data mapping rules, registration rules, data A rule database for storing a transmission rule for specifying a transmission execution method; Metadata which is connected to the rule database and receives at least one or more rules for heterogeneous metadata collection from the rule database, extracts heterogeneous metadata from each source table where heterogeneous metadata exists, and transmits the heterogeneous metadata to the staging area of the repository. Heterogeneous metadata automatic extraction system using a rule-based technology, including a collector.

The metadata collector presents a server designation rule for designating a server in which a source table of metadata to be extracted exists, and the data necessary for automatic connection of a target server using answers selected and selected based on the server designation rule. A server specifying module specifying an attribute; A metadata type designation module for presenting a metadata type designation rule for designating a type of metadata to be extracted, and for designating a metadata type using answers selected and input based on the metadata type designation rule; For each metadata type specified by the metadata type designation module, a table about the metadata and an attribute registration rule for each attribute for presenting a related attribute are proposed and extracted using answers selected and selected based on the attribute registration rule for each type. A target attribute specifying module for specifying a table and related attributes relating to the type of metadata to be performed; Answer the data mapping rule registration rule for specifying the data mapping rule between the source table of the metadata to be extracted and the target table of the repository to which the metadata is to be loaded, and selects and inputs based on the data mapping rule registration rule It may be configured to include; mapping rule designation module for specifying a mapping rule between the tables using them.

The metadata collector presents a transmission rule for specifying a data transmission execution method from the source table to a target table, and specifies a data execution execution rule using answers selected and input based on the transmission rule. Designation module; may be configured to include more.

According to another embodiment of the present invention for achieving the above object, in a method for extracting heterogeneous metadata from a source table through a metadata collector, (a) source table of metadata to be extracted A server designation step of presenting a server designation rule for designating an existing server, and designating a data attribute required for automatic access of a target server by using answers selected and selected based on the server designation rule; (b) a metadata type specification step of specifying a metadata type specification rule for specifying a type of metadata to be extracted, and specifying a metadata type using answers selected and input based on the metadata type specification rule; ; (c) suggesting a property registration rule for each type of metadata for each metadata type specified in the metadata type designation step and a property registration rule for designating a related property, and using selected answers based on the property registration rule for each type; A target attribute specifying step of specifying a table and related attributes relating to the type of metadata to be extracted; (d) propose a data mapping rule registration rule for designating a data mapping rule between a source table of metadata to be extracted and a target table of a repository into which the metadata is to be loaded, and select based on the data mapping rule registration rule Heterogeneous metadata automatic extraction method using a rule-based technology, including; mapping rule specifying step of specifying a mapping rule between tables using the input answers.

The method for automatically extracting heterogeneous metadata according to an embodiment of the present invention includes (e) presenting a transmission rule for specifying a method of executing data transmission from the source table to a target table, and selectively inputting the received data based on the transmission rule. A transmission execution rule specification step of specifying a data transmission execution rule using the answers may be implemented in another embodiment.

The server designation rule presented in the server designation step may be input by selecting a computer name of a DB server, a description of a DB server, a DB type, a DB name, or a login ID and password of a corresponding DB required for designation of the target server. Characterized in that consists of a query. In addition, this server designation rule is that if there is an intermediary server between the target server and the repository, the computer name of the intermediary server, the address of the intermediary server, the operating system (OS) of the intermediary server, the communication port of the intermediary server, the intermediary server login ID, It may consist of a query that allows you to select and enter more passwords.

The metadata type specification rule presented in the metadata type designation step is a query statement that can be inputted by selecting a CASE tool, DB, ETL tool, OLAP tool, work management tool, system management tool, and parsing tool. It is characterized in that the configuration. In addition, these metadata type specification rules may be composed of query statements that can distinguish each supplier's product and each product version of each tool in detail.

The (d) mapping rule designation step may include: (d1) registering a source table and a source field of metadata to be extracted; (d2) registering a mapping list designation logic for designating a table mapping list, mapping detail list, and mapping detail information required for table mapping according to a predesigned table mapping rule; (d3) specifying a standardized target table of the repository to which the metadata is to be registered.

According to the present invention, since multiple software tool information, which is a source of metadata, can be easily linked and extracted using a rule-based technique, metadata extraction from a source table, temporary storage and conversion in a staging area, and an object table of a repository are performed. In the process of standardized metadata loading into a furnace, there is an advantage of reducing manpower and time wasted in extracting heterogeneous metadata.

1 is a schematic diagram of a heterogeneous metadata automatic extraction system using a rule-based technique according to an embodiment of the present invention.
2A and 2B are tables illustrating data attributes required for designation of a target server for automatic extraction of heterogeneous metadata in the system of FIG. 1.
3 is a table illustrating a data attribute required for specifying a type of target metadata in the system of FIG. 1.
4 is a table relationship diagram illustrating data attributes required for a SAS product, which is an ETL tool.
Figure 5 illustrates the entity relationship diagram by generalizing the attributes required for the ETL tool.
6 is a diagram illustrating a mapping procedure of a source table and a target table of metadata in the system of FIG. 1.
7A and 7B illustrate exemplary transmission execution rules and transmission history inquiry in the system of FIG. 1.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a schematic configuration diagram of a heterogeneous metadata automatic extraction system using a rule-based technique according to an embodiment of the present invention, and FIGS. 2A and 2B are tables illustrating data attributes required for designation of a target server. 3 is a table illustrating data attributes required for specifying the type of target metadata, FIGS. 4 and 5 are entity relationship diagrams required for registration of data and related attributes required for each target metadata type, and FIG. 6 is a source of metadata. An example of a mapping procedure between a table and a target table is illustrated, and FIGS. 7A and 7B are exemplary views of a transmission execution rule.

As shown in Figure 1 heterogeneous metadata (heterogeneous metadata) automatic extraction system according to the present invention, the source table 10 including at least one or more of the information of the various software tools of the various operating system environment, for heterogeneous metadata collection A rule database 160 for storing at least one rule, a metadata collector 100 for collecting heterogeneous metadata using a rule-based technique from the source table, and temporarily storing heterogeneous metadata extracted through the metadata collector And a staging area 210 to be converted, and a repository 220 to be standardized and loaded for each metadata type.

The various software tool information of the various operating system environments of the source table 10 may include a database management system (DBMS), a computer aided software engineering (CASE) tool, an extraction transformation and loading (ETL) tool, an online analytical processing (OLAP) tool, It may be a task management tool or a system management tool.

The metadata collector 100 is a means for temporarily extracting metadata from the source table 10 and temporarily storing the metadata in the staging area 210. The server designation module 110 for designating a server in which the extraction target metadata exists, Metadata type designation module 120 for specifying the type of the extraction target metadata, target property designation module 130 for designating the table and related properties for each metadata, and mapping rule designation between the extraction target metadata and the target data. Mapping rule specifying module 140, and a transmission execution rule specifying module 150 for specifying a data transmission execution method. The metadata collector 100 is configured to be provided with various rules necessary for collecting heterogeneous metadata in association with the rule database 160.

The server designation module 110 proposes a server designation rule for designating a server in which a source table of metadata to be extracted exists, and is required for automatic connection of a target server by using answers selected and selected based on the server designation rule. This module specifies data attributes.

The metadata type designation module 120 proposes a metadata type designation rule for designating the type of metadata to be extracted, and designates the metadata type by using answers selected and input based on the metadata type designation rule. do.

The object attribute designation module 130 suggests a property registration rule for each type of metadata specified by the metadata type designation module and a property registration rule for each property for designating related properties, and selects based on the property registration rule for each type. Using the input answers, specify the table and related properties related to the type of metadata to be extracted.

The mapping rule designation module 140 proposes a data mapping rule registration rule for designating a data mapping rule between a source table of metadata to be extracted and a target table of a repository into which the metadata is to be loaded, and registers the data mapping rule. Mapping rules between tables are specified using the answers that are selected based on rules.

The transmission execution rule specification module 150 proposes a transmission rule for specifying a data transmission execution method from the source table to a target table, and designates a data transmission execution rule by using answers selected and input based on the transmission rule. .

The rule database 160 is a means for storing a predetermined rule for composing various query statements, the server specification rule for specifying the server, the metadata type specification rule for specifying the metadata type, and the metadata for each metadata type. It stores the property registration rule for each type to specify the table and related property for metadata, the data mapping rule registration rule for data mapping rule designation, and the transfer rule for data transmission execution method designation.

Server designation rule is a query that can be entered by selecting the computer name of the DB server, the description of the DB server, DB type, DB name, the login ID and password of the DB, as shown in Figure 2a to specify the target server Consists of doors. The server designation rule composed of such a query includes the computer name of the mediation server, the address of the mediation server, the operating system (OS) of the mediation server, as shown in FIG. 2B when the mediation server exists between the target server and the repository. It consists of interrogation server's communication port, interrogation server login ID and password to select and enter more.

As shown in FIG. 3, metadata type specification rules can be input by selecting target metadata types classified into DB, CASE tool, ETL tool, OLAP tool, work management tool, system management tool, and parsing tool. It consists of a query statement. At this time, it is preferable that the query is divided and presented to each product and supplier version of the metadata.

As shown in FIG. 4 and FIG. 5, the attribute registration rule for each type of metadata is composed of a table regarding metadata type for each supplier and a query for selecting and inputting an attribute related thereto. FIG. 4 is a table relation diagram illustrating data attributes required for a SAS product, which is an ETL tool, and FIG. 5 is a diagram illustrating an entity relation diagram, generalizing attributes required for an ETL tool.

The data mapping rule registration rule may consist of a query that can be entered by selecting a data mapping rule registration procedure for specifying a data mapping rule between a source table of metadata and a target table of a repository in which the metadata is to be loaded. . Such a query may register the source table and source field with the metadata to be extracted as shown in FIG. 6, register the table mapping list, the mapping detail list, the mapping detail information, and the like to load the metadata. It can be composed of a query that can be entered by selecting the target table of the repository.

As shown in FIGS. 7A and 7B, a data transmission rule is a transmission rule for specifying a method of executing data transmission from a source table to which metadata belongs to a target table of a repository, that is, a transmission date to be managed by adjusting a transmission schedule. And it can be composed of a query that can be entered by selecting the time, whether or not to repeat the operation, such that the transmission results are recorded in the transfer log to confirm the final transfer status and to present an inquiry screen to manage the transfer history Can be configured.

The heterogeneous metadata automatic extraction method using rule-based technology according to the present invention includes a server specifying step, a metadata type specifying step, a target attribute specifying step, a mapping rule specifying step, and a transmission execution rule specifying step. To each module of the heterogeneous metadata automatic extraction system of the rule-based technique illustrated in FIG. 7B.

The server designation step is executed in the server designation module 110 of the metadata collector 100. In this step, data attributes, or mediation, necessary for accessing a server where a source table of metadata to be extracted exists. If there is a server, a server designation rule for querying the attributes of data required for accessing the server where metadata exists through the intermediary server is presented as shown in FIGS. 2A and 2B, and based on the server designation rule By using the selected input answers, the data property required for automatic connection of the target server is specified.

The metadata type designation step is executed in the metadata type designation module 110 of the metadata collector 100. In this step, the type of metadata to be extracted is divided into products for each supplier and versions for each product. The metadata type designation rule that can be specified is presented as shown in FIG. 3, and the type of metadata is designated using the answers that are selectively input based on the metadata type designation rule.

The target property designation step is executed in the target property designation module 130 of the metadata collector 100. In this step, the table and the related property about the metadata for each metadata type specified in the metadata type designation step are performed. A property registration rule for each type for designation is presented as shown in FIGS. 4 and 5, and a table and a related property regarding a type of metadata to be extracted are designated using answers selected and input based on the property registration rule for each type.

The mapping rule specifying step is executed in the mapping rule specifying module 140 of the metadata collector 100. In this step, data between the source table of metadata to be extracted and the target table of the repository to which the metadata is to be loaded A data mapping rule registration rule for specifying a mapping rule is presented as shown in FIG. 6, and a mapping rule between tables is specified using answers selected and input based on the data mapping rule registration rule.

The transmission execution rule specification step is executed in the transmission execution rule specification module 150 of the metadata collector 100. In this step, a transmission rule for specifying a data transmission execution method from the source table to the target table is shown in FIGS. Presented together, the data transmission execution rule is specified using the answers selected and input based on the transmission rule.

According to the method for automatically extracting heterogeneous metadata using rule-based technology according to the present invention, it is possible to easily extract various software tool information of various operating system environments, which is a source of metadata, and thus extract metadata from the source table. In addition, temporary storage and conversion in the staging area and standardized metadata loading process into the repository's destination table will be performed automatically to reduce manpower consumption and time wastage.

In the above, the present invention has been illustrated and described with respect to certain preferred embodiments. However, the present invention is not limited only to the above-described embodiments, and those skilled in the art to which the present invention pertains may variously change without departing from the spirit of the technical idea of the present invention described in the claims below. Could be done.

10: source table 100: metadata collector
110: server designation module 120: metadata type designation module
130: target attribute designation module 140: mapping rule designation module
150: transmission execution rule designation module 160: rule database
210: staging area 220: repository

Claims (17)

In the method of extracting heterogeneous metadata from the source table through a metadata collector,
(a) Presenting a server designation rule for designating a server that has a source table of metadata to be extracted, and designating data attributes necessary for automatic connection of a target server by using answers selected and selected based on the server designation rules. Specifying a server;
(b) a metadata type specification step of specifying a metadata type specification rule for specifying a type of metadata to be extracted, and specifying a metadata type using answers selected and input based on the metadata type specification rule; ;
(c) suggesting a property registration rule for each type of metadata for each metadata type specified in the metadata type designation step and a property registration rule for designating a related property, and using selected answers based on the property registration rule for each type; A target attribute specifying step of specifying a table and an associated attribute rule regarding a type of metadata to be extracted;
(d) propose a data mapping rule registration rule for designating a data mapping rule between a source table of metadata to be extracted and a target table of a repository into which the metadata is to be loaded, and select based on the data mapping rule registration rule A mapping rule specifying step of specifying a mapping rule between tables using the input answers; heterogeneous metadata automatic extraction method using a rule-based technology comprising a. The method of claim 1,
(e) a transmission execution rule specifying step of presenting a transmission rule for specifying a data transmission execution method from the source table to a target table and specifying a data transmission execution rule using answers selected and input based on the transmission rule; Heterogeneous metadata automatic extraction method using a rule-based technology characterized in that it further comprises. According to claim 1, (a) The server designation rule presented in the server designation step,
Rule-based, characterized in that consists of a query name that can be entered by selecting the computer name of the DB server, the description of the DB server, DB type, DB name, the login ID and password of the DB required to specify the target server Automatic Extraction of Heterogeneous Metadata Using Technology. According to claim 1, (a) The server designation rule presented in the server designation step,
If there is an intermediary server between the target server and the repository, select the computer name of the intermediary server, the address of the intermediary server, the operating system (OS) of the intermediary server, the communication port of the intermediary server, and enter the intermediary server login ID and password. Heterogeneous metadata automatic extraction method using a rule-based technology, characterized in that consisting of a query. The metadata type designation rule of claim 1, wherein the metadata type designation rule presented in step (b)
Automatic heterogeneous metadata extraction using rule-based technology, consisting of query statements that can be entered by selecting CASE tool, DB, ETL tool, OLAP tool, task management tool, system management tool, and parsing tool Way. The metadata type designation rule of claim 5, wherein the metadata type designation rule is presented in the metadata type designation step.
The heterogeneous metadata automatic extraction method using rule-based technology, characterized in that the query that can be distinguished in detail the product of each supplier and the version of each product for each tool. The method of claim 1, wherein the mapping rule designation step (d) comprises:
(d1) registering a source table and a source field of metadata to be extracted;
(d2) registering a mapping list designation logic for designating a table mapping list, mapping detail list, and mapping detail information required for table mapping according to a predesigned table mapping rule;
and (d3) designating a standardized target table of the repository to which the metadata are to be registered. In the system that extracts heterogeneous metadata from the source table and loads it into the repository through standardization of each type of metadata,
Server designation rule for server designation, metadata type designation rule for metadata type designation, property registration rule by type for designation of table and related attributes for each metadata type, data for data mapping rule designation A rules database for storing a mapping rule registration rule and a transmission rule for specifying a data transmission execution method;
Metadata which is connected to the rule database and receives at least one or more rules for heterogeneous metadata collection from the rule database, extracts heterogeneous metadata from each source table where heterogeneous metadata exists, and transmits the heterogeneous metadata to the staging area of the repository. Heterogeneous metadata automatic extraction system using a rule-based technology, characterized in that comprises a collector. The method of claim 8, wherein the metadata collector,
Specifying server specification rule for server designation that the source table of metadata to be extracted exists, and designating server to designate data property necessary for automatic connection of target server by using selected and entered answers based on the server specification rule module;
A metadata type designation module for presenting a metadata type designation rule for designating a type of metadata to be extracted, and for designating a metadata type using answers selected and input based on the metadata type designation rule;
For each metadata type specified by the metadata type designation module, a table about the metadata and an attribute registration rule for each attribute for presenting a related attribute are proposed and extracted using answers selected and selected based on the attribute registration rule for each type. A target attribute specifying module for specifying a table and related attributes relating to the type of metadata to be performed;
Answer the data mapping rule registration rule for specifying the data mapping rule between the source table of the metadata to be extracted and the target table of the repository to which the metadata is to be loaded, and selects and inputs based on the data mapping rule registration rule Heterogeneous metadata automatic extraction system using a rule-based technology, characterized in that it comprises a; mapping rule specifying module for specifying a mapping rule between the tables using them. The method of claim 8, wherein the metadata collector,
A transmission execution rule specification module for presenting a transmission rule for specifying a data transmission execution method from the source table to a target table, and specifying a data transmission execution rule using answers selected and input based on the transmission rule; Heterogeneous metadata automatic extraction system using a rule-based technology, characterized in that. The method of claim 8, wherein the server designation rule,
Rule-based technology, consisting of a query statement that can be entered by selecting the computer name of the DB server, the description of the DB server, DB type, DB name, the login ID and password of the DB required to specify the target server Heterogeneous metadata automatic extraction system. The method of claim 11, wherein the server designation rule,
If there is an intermediary server between the target server and the repository, select the computer name of the intermediary server, the address of the intermediary server, the operating system (OS) of the intermediary server, the communication port of the intermediary server, and enter the intermediary server login ID and password. Heterogeneous metadata automatic extraction system using rule-based technology, characterized in that consisting of query statements. The method of claim 8, wherein the metadata type designation rule,
It consists of a query that can be entered by selecting CASE tool, DB, ETL tool, OLAP tool, task management tool, system management tool, and parsing tool. Heterogeneous metadata automatic extraction system using a rule-based technology, characterized in that divided to the version of the proposed. The method of claim 8, wherein the property registration rule for each metadata type comprises:
A heterogeneous metadata automatic extraction system using a rule-based technology, characterized in that consists of a table for the supplier-specific metadata type and a query that can be entered by selecting the relevant properties. The method of claim 8, wherein the data mapping rule registration rule,
Heterogeneous using rule-based technology, consisting of query statements that can be entered by selecting the data mapping rule registration procedure for specifying the data mapping rule between the source table of metadata and the target table of the repository into which the metadata is loaded. Automatic metadata extraction system. The method of claim 15, wherein the query,
Registration of source table and source field with metadata to be extracted, table mapping list necessary for table mapping, mapping detail list, mapping detail information, etc., target table of repository to load metadata can be selected and entered. Heterogeneous metadata automatic extraction system using rule-based technology, characterized in that consisting of query statements. The method of claim 8, wherein the data transmission rule,
Heterogeneous metadata automatic extraction system using a rule-based technology, characterized in that consisting of a query that can be entered by selecting the transmission date and time associated with the transmission schedule, whether or not to repeat the operation.

Images