CN114817149A - Metadata management system - Google Patents
Metadata management system Download PDFInfo
- Publication number
- CN114817149A CN114817149A CN202210287993.XA CN202210287993A CN114817149A CN 114817149 A CN114817149 A CN 114817149A CN 202210287993 A CN202210287993 A CN 202210287993A CN 114817149 A CN114817149 A CN 114817149A
- Authority
- CN
- China
- Prior art keywords
- data
- module
- feedback
- list
- user
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012937 correction Methods 0.000 claims abstract description 34
- 239000008280 blood Substances 0.000 claims abstract description 10
- 210000004369 blood Anatomy 0.000 claims abstract description 10
- 238000007726 management method Methods 0.000 claims description 48
- 238000013523 data management Methods 0.000 claims description 7
- 238000012986 modification Methods 0.000 claims description 7
- 230000004048 modification Effects 0.000 claims description 7
- 238000013498 data listing Methods 0.000 claims 2
- 238000000034 method Methods 0.000 description 26
- 230000008569 process Effects 0.000 description 18
- 238000010586 diagram Methods 0.000 description 6
- 230000009471 action Effects 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 238000011084 recovery Methods 0.000 description 3
- 230000010354 integration Effects 0.000 description 2
- 238000012423 maintenance Methods 0.000 description 2
- 238000012216 screening Methods 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- 238000011144 upstream manufacturing Methods 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 238000012550 audit Methods 0.000 description 1
- 210000004204 blood vessel Anatomy 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000013073 enabling process Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000002085 persistent effect Effects 0.000 description 1
- 239000000047 product Substances 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 239000013589 supplement Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/14—Details of searching files based on file metadata
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
- G06F16/2365—Ensuring data consistency and integrity
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/252—Integrating or interfacing systems involving database management systems between a Database Management System and a front-end application
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
- G06F16/285—Clustering or classification
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Library & Information Science (AREA)
- Computer Security & Cryptography (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The present invention provides a metadata management system, including: the data profile module is used for calibrating and classifying data in the data asset library and generating a data list corresponding to each category; displaying asset library information to a user; searching each target data when receiving search information sent by a search engine module, and constructing a blood relationship corresponding to each target data; the data index module is used for carrying out index division on each data in each data list; the data feedback module is used for receiving a data correction message sent by a user and sending the data correction message to a corresponding service end; after the feedback result of the service end is obtained, the data corresponding to the feedback result is corrected by the data overview module or the data index module; the search engine module is used for receiving search information sent by a user to realize data search. By applying the system provided by the invention, the management of the metadata can be realized, and the source tracing of the data can be realized through the management of the metadata.
Description
Technical Field
The invention relates to the technical field of metadata management, in particular to a metadata management system.
Background
With the increasing number of business product lines of companies, the bottom layer data records become more and more complex, the data of each business line is self-organized, and a data isolated island is easily formed in a chimney type business development form. The establishment of a company-level data warehouse is the optimal choice under the condition, and the data of a summary layer and a display layer can be used as the basis of BI analysis and can be directly packaged into a data interface. More and more business data and data warehouse data are important in data management, how to retrieve data more quickly in the data analysis and business enabling processes, and data are selected according to business apertures with different requirements. Metadata is data that is data, and is the core of recording such information. Metadata management and maintenance of a data full life cycle are the foundation of data management, and a foundation is laid for ensuring stable and efficient enabling of company data.
In the prior art, metadata is easy to lose in the data management process, and the full life cycle tracing of the data cannot be realized, so that the subsequent data searching difficulty is increased in the data application process.
Disclosure of Invention
In view of this, the present invention provides a metadata management system, and by using the method, metadata management can be implemented, and data tracing can be implemented through the metadata management.
A metadata management system, comprising:
the system comprises a data profile module, a data index module, a data feedback module and a search engine module;
the data profiling module is used for scanning metadata information of the data warehouse in real time and calibrating data in the data asset library in real time based on the metadata information in the data warehouse; classifying each data in the data asset library based on a preset first classification rule to obtain a data list corresponding to each category in the data asset library; when the user login is detected, displaying asset library information of the data asset library to the user, wherein the asset library information comprises each data list; when receiving search information sent by the search engine module, searching each target data list corresponding to the search information, and constructing each blood relationship corresponding to each target data by each target data associated with the target data list;
the data index module is used for performing index division on each data in each data list based on a preset second classification rule to obtain a plurality of index category data indexes under each data list;
the data feedback module is used for receiving a data correction message sent by a user and determining a feedback type corresponding to the data correction message, wherein the feedback type is a data type or an index type; sending the data correction message to a service end corresponding to the feedback type; when a feedback result corresponding to the data correction message fed back by the service end is received, calling the data profile module or the data index module to classify result data corresponding to the feedback result based on the feedback type;
the search engine module is used for receiving search information sent by a user and sending the search information to the data profile module, so that the data profile module queries data corresponding to the search information from the data asset library.
In the metadata management system, optionally, the data profiling module is configured to determine whether the data warehouse has newly added data based on the metadata information, and determine whether updated data exists in the data warehouse; when newly added data exist in the data warehouse, adding the newly added data to a preset newly added list; when updated data exist in the data warehouse, adding the updated data to a preset update list; storing the newly added data in the newly added list to the data asset library; updating the data in the data asset library based on the updated data in the update list.
Optionally, the metadata management system is configured to determine a user type of the user; when the user type of the user is a common user, showing each data list of the data asset library and data details corresponding to each data list to the user; and when the user type of the user is a management user, displaying each data list of the data asset library, data details corresponding to each data list and a data management module corresponding to the data asset library to the user.
In the above metadata management system, optionally, the data profiling module is configured to obtain a table number included in the search information; determining a target data list in the data asset library, which is matched with the table number; and searching each target data associated with the target data list based on the table number.
In the above metadata management system, optionally, the data profiling module is configured to associate each data matched with the id based on a blood relationship between each target data associated with the target data list, to obtain associated data, and send the associated data to the user.
Optionally, the metadata management system further includes a data profiling module, configured to: and when a discarding instruction corresponding to any data list in the data asset library is received, canceling the display of the data list corresponding to the discarding instruction.
In the above metadata management system, optionally, the data profiling module is further configured to restore and display the data list corresponding to the restoration instruction when receiving the restoration instruction corresponding to the discarded data list.
Optionally, in the metadata management system, the data feedback module is configured to obtain a feedback identifier included in the data modification message; when the feedback identifier is a first feedback identifier, determining that the feedback type corresponding to the data correction message is a data type; and when the feedback identifier is a second feedback identifier, determining that the feedback type corresponding to the data correction message is an index type.
Optionally, in the metadata management system, when the feedback type corresponding to the data modification message is a data type, the data feedback module is configured to record the feedback result to a preset first repository, and call the data profiling module to classify the feedback result according to the first classification rule; and when the feedback type corresponding to the data correction message is an index type, recording the feedback result to a preset second storage library, and calling the data index module to classify the feedback result according to the second classification rule.
In the metadata management system, optionally, the search engine module is configured to obtain a library name and a data type included in the search information, and query, based on the library name and the data type, data corresponding to the search information from the data asset library.
Compared with the prior art, the invention has the following advantages:
the present invention provides a metadata management system, including: the data profiling module is used for scanning the metadata information of the data warehouse in real time and calibrating the data in the data asset library in real time based on the metadata information in the data warehouse; classifying each data in the data asset library based on a preset first classification rule to obtain a data list corresponding to each category in the data asset library; when the user login is detected, displaying asset library information of the data asset library to the user, wherein the asset library information comprises each data list; when receiving search information sent by the search engine module, searching each target data list corresponding to the search information, and constructing each blood relationship corresponding to each target data by each target data associated with the target data list; the data index module is used for carrying out index division on each data in each data list based on a preset second classification rule to obtain a plurality of index category data indexes under each data list; the data feedback module is used for receiving a data correction message sent by a user and determining a feedback type corresponding to the data correction message, wherein the feedback type is a data type or an index type; sending the data correction message to a service end corresponding to the feedback type; when a feedback result corresponding to the data correction message fed back by the service end is received, calling the data profile module or the data index module to classify result data corresponding to the feedback result based on the feedback type; and the search engine module is used for receiving search information sent by a user and sending the search information to the data profile module, so that the data profile module queries data corresponding to the search information from the data asset library. By applying the system provided by the invention, the management of the metadata can be realized, and the source tracing of the data can be realized through the management of the metadata.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.
Fig. 1 is a system structure diagram of a metadata management system according to an embodiment of the present invention;
FIG. 2 is a diagram illustrating an application example of a data profiling module in a metadata management system according to an embodiment of the present invention;
FIG. 3 is a diagram illustrating another application example of a data profiling module in a metadata management system according to an embodiment of the present invention;
fig. 4 is a diagram illustrating an application example of a data feedback module in a metadata management system according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
In this application, relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions, and the terms "comprises", "comprising", or any other variation thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
The invention is operational with numerous general purpose or special purpose computing device environments or configurations. For example: personal computers, server computers, hand-held or portable devices, tablet-type devices, multi-processor apparatus, distributed computing environments that include any of the above devices or equipment, and the like.
An embodiment of the present invention provides a metadata management system, where a system structure diagram of the metadata management system is shown in fig. 1, and the metadata management system specifically includes:
a data profiling module 100, a data metrics module 200, a data feedback module 300, and a search engine module 400;
the data profiling module 100 is configured to scan metadata information of the data warehouse in real time and calibrate data in the data asset library in real time based on the metadata information in the data warehouse; classifying each data in the data asset library based on a preset first classification rule to obtain a data list corresponding to each category in the data asset library; when the user login is detected, displaying asset library information of the data asset library to the user, wherein the asset library information comprises each data list; when receiving search information sent by the search engine module 400, searching each target data list corresponding to the search information, and constructing a blood relationship corresponding to each target data by each target data associated with the target data list;
the data index module 200 is configured to perform index division on each data in each data list based on a preset second classification rule, so as to obtain a plurality of index category data indexes under each data list;
the data feedback module 300 is configured to receive a data correction message sent by a user, and determine a feedback type corresponding to the data correction message, where the feedback type is a data type or an index type; sending the data correction message to a service end corresponding to the feedback type; when a feedback result corresponding to the data correction message fed back by the service end is received, calling the data profile module or the data index module to classify result data corresponding to the feedback result based on the feedback type;
the search engine module 400 is configured to receive search information sent by a user, and send the search information to the data profiling module, so that the data profiling module queries data corresponding to the search information from the data asset library.
In the metadata management system provided by the embodiment of the invention, the data overview module mainly displays the information of the data list, and the information can be displayed differently according to different user types of the logged-in users. The data asset library comprises a plurality of data lists for storing the data in the data warehouse. And designing a database table by scanning metadata information of the data warehouse to carry out persistent storage record on information, classification and labels of the data table. Compared with the prior manual maintenance mode, more auxiliary description fields can be designed under the condition of not increasing the cost, and the recorded data is more complete and reliable. And generating a path for the blood relationship data, and displaying the complete relationship path through the front end. Compared with the previous single-stage upstream and downstream relation display, the whole dependent path can be seen more visually from the whole situation, and the independent query is not needed layer by layer. The user can directly access the data list structure and the metadata information through the network environment of daily office without logging in a disconnected production environment. Data consanguinity is the basic concept of metadata management, and refers to the link of data generation. In order to find the connection between related data in the process of data source tracing. For example, field abc in repository summary table B is extracted from source layer a, so that B table has a data lineage relationship with a table. The first classification rule performs classification processing according to tables of different layers in a data warehouse framework, the total amount of each layer of tables and the specific large class and small class distinction of each table can be clearly displayed, and the detailed data of each table can be checked by clicking each table. Each layer is a schema, and each schema corresponds to a plurality of tables.
The data index management module divides data according to rules in a finer granularity mode, and managers can classify the data according to the rules through the management module, so that the query efficiency of users is improved. The second classification rule is that classification is carried out according to the service, so that managers can conveniently find indexes under corresponding classification, and the indexes under each classification can be added, deleted, modified and inquired, so that the management of the indexes by the managers is better realized.
The data feedback module is a module for providing data correction for a user, and the user can select and submit correction related to data, index or other data in the whole system according to the requirement of the user, so that the user can conveniently submit feedback of suggestions, problems and the like of the user. That is, the data is corrected according to the feedback type.
The search engine module is set for the table query by a user, supports independent query and combined query, and supports fuzzy search. The combined query comprises the library name, the major class and the minor class, and the combined query for searching data, so that the retrieval efficiency is improved greatly.
By applying the method provided by the embodiment of the invention, the metadata can be managed, the content can be supplemented, and the source tracing of the data can be realized through the management of the metadata.
In particular, for the data profile module, it can implement data calibration, list generation and presentation, and correlation and search of data blood margin.
In the embodiment of the present invention, the process that the data profiling module is used for calibrating data and generating a list is as follows: judging whether the data warehouse has newly added data or not and judging whether updated data exists in the data warehouse or not based on the metadata information; when newly added data exist in the data warehouse, adding the newly added data to a preset newly added list; when updated data exist in the data warehouse, adding the updated data to a preset update list; storing the newly added data in the newly added list to the data asset library; updating the data in the data asset library based on the updated data in the update list.
Specifically, referring to fig. 2, fig. 2 is a process of calibrating data in a data asset library, where a trigger end in fig. 2 is a data profile module, and a corresponding implementation process in fig. 2 is as follows:
(1) acquiring the data source _ column _ set of the production-ready bins and the service,
(2) obtain the undeleted data target _ column _ set in the data asset,
(3) obtaining data to be newly added according to the data obtained in the steps (1) and (2), wherein the data to be newly added is equal to all data in the current data warehouse, the data already existing in the data assets are filtered out,
(4) obtaining data to be updated according to the data obtained in (1) and (2), wherein the data to be updated is the comparison between the data already existing in the current data asset and the data in the current data bin, storing the data with different contents to obtain the data to be updated,
(5) judging whether the data to be newly added is empty, if not, integrating the data into the list to be inserted, inserting the data into the data assets after the integration is finished,
(6) and judging whether the data to be updated is empty or not, if not, integrating the data into a list to be updated, and updating the data of the data assets after integration is finished.
In an embodiment of the present invention, the data profiling module is configured to perform a data display process including:
when the user type of the user is a common user, showing each data list of the data asset library and data details corresponding to each data list to the user; and when the user type of the user is a management user, displaying each data list of the data asset library, data details corresponding to each data list and a data management module corresponding to the data asset library to the user.
It can be understood that different displays can be displayed according to different logged-in users, and the functions displayed by the common user are as follows: the data list and the data details, and the data management module is added on the basis of common users by the administrator user, so that the administrator can conveniently manage the data list. The data displayed is from a data warehouse, and the latest information of the data table is synchronized to the data asset system at regular time through iteration. This module mainly displays the relevant information of the viewed table, including: basic information of the table (creation time, maintainer, etc.), the category to which the table belongs, field information in the table, kindred relationship of the table, usage.
In an embodiment of the present invention, the process used by the data profiling module to perform the association and search of the data blooding borders is as follows: acquiring a table number contained in the search information; determining a target data list in the data asset library which is matched with the table number; and searching each target data associated with the target data list based on the table number. And associating each data matched with the id based on the blood relationship between each target data associated with the target data list to obtain associated data, and sending the associated data to the user.
Referring to fig. 3, fig. 3 is a schematic diagram of a connection process of searching data and relationship between blood vessels, and the corresponding specific process is as follows:
(1) and acquiring relevant data of the table according to the number of the transmitted lookup table.
(2) And (3) judging whether the data acquired in the step (1) is empty or not, and integrating the data if the data is not empty.
(3) And (3) querying an upstream table and a downstream table of the table according to the number of the incoming table, integrating the data with the data in the step (2), and returning a result.
In the embodiment of the invention, the data profile module can realize the functions of calibrating, searching, displaying and the like of data, and can also realize the abandonment and recovery process of the list. Therefore, the data profile module is further configured to, when receiving a recovery instruction corresponding to the discarded data list, recover to display the data list corresponding to the recovery instruction.
After discarding the data list, the data profiling module is further configured to obtain a feedback identifier included in the data modification message; when the feedback identifier is a first feedback identifier, determining that the feedback type corresponding to the data correction message is a data type; and when the feedback identifier is a second feedback identifier, determining that the feedback type corresponding to the data correction message is an index type.
It should be noted that the data profiling module mainly manages the table, new data is synchronized to the data assets in each iteration, the whole data and related information of the table are not shown in the data list after being strictly checked by the administrator, otherwise, the table is in a checking state until the checking is completed. Thus, the data profiling module manages the table specifically including:
(1) entering data table management: and acquiescently acquiring data of all tables according to the reverse table building time sequence, and displaying the data in the list.
(2) Screening of data tables: the combined query can be carried out according to the library name, the table building time, the table state and the fuzzy search, and the screening efficiency of the user is improved.
(3) An auditing function: an auditing function is arranged behind each table, the relevant information of the table needs to be filled after entering the table, and the auditing can be carried out after the filling is completed.
(4) Abandonment function: the table can be discarded while the audit is performed, the information of the table is not shown in the data profile after the table is discarded, and if the table needs to be recovered for use later, the table is also supported to be recovered.
In the embodiment of the present invention, the data feedback module is specifically configured to acquire a feedback identifier included in the data correction message; when the feedback identifier is a first feedback identifier, determining that the feedback type corresponding to the data correction message is a data type; and when the feedback identifier is a second feedback identifier, determining that the feedback type corresponding to the data correction message is an index type.
It is understood that, when the user sends the data modification message, the data to be modified is defined as a data type or an index type. For example, the first feedback is identified as 0 and the second feedback is identified as 1. And when the feedback identifier in the data correction message is 0, the feedback type is the data type, and if the feedback identifier is 1, the feedback type is the index type.
Further, the data feedback module is specifically configured to record the feedback result to a preset first repository when the feedback type corresponding to the data correction message is a data type, and call the data profiling module to classify the feedback result according to the first classification rule; and when the feedback type corresponding to the data correction message is an index type, recording the feedback result to a preset second storage library, and calling the data index module to classify the feedback result according to the second classification rule.
It should be noted that the feedback result of the content supplement is newly updated data of the data warehouse, and thus after obtaining the feedback result, the data profiling module performs further management.
Referring to fig. 4, fig. 4 is a specific application process of the data feedback module, specifically including data feedback and index feedback. The data feedback process comprises the following steps:
(1) the user can select the basic situation of data to be fed back according to the problem of the user, wherein the basic situation comprises the following steps: types of feedback, including: data quality, data loss; the major and minor classes to which the data to be fed back belongs, the library name and the indication and field; meanwhile, the question description can be directly filled in;
(2) after the data in the step (1) is added, submitting the data;
(3) the administrator processes the problem feedback when receiving the problem feedback, finally backfills the processed result and informs the user of the problem processing condition;
(4) the user can know the result of the question feedback through the question that is fed back.
The index feedback process comprises the following steps:
(1) the user selects the index category and index name, fills in the question description, can submit,
(2) the administrator receives the index question feedback to process, and at the same time, the processing result is backfilled, and the user can check the feedback result through logging in the system.
It should be further noted that, after receiving the feedback result, the data feedback module records and stores the feedback result and other feedback contents if other feedback contents exist.
In the embodiment of the present invention, the search engine module is specifically configured to acquire a library name and a data category included in the search information, and query, based on the library name and the data category, data corresponding to the search information from the data asset library.
It will be appreciated that the search engine module is provided for better user convenience with respect to table queries, and supports both individual queries and combination queries, as well as fuzzy search. The combined query comprises the library name, the major class and the minor class, and the combined query for searching data, so that the retrieval efficiency is improved greatly.
The database name query refers to searching out corresponding data according to the selected database name, and if the fuzzy query has data, the fuzzy query carries the data to perform combined query. The large-class query is to query corresponding data according to the large class, if the library name or the large class is not empty, the combined query is made, otherwise, the query is carried out independently. The subclass query is that the subclass is restricted by the major class, if the major class is empty, the subclass has no data, otherwise, the corresponding subclass is displayed according to the corresponding major class, the combined query is performed according to the selection, and the queried data is displayed.
The specific implementation procedures and derivatives thereof of the above embodiments are within the scope of the present invention.
The embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, the system or system embodiments are substantially similar to the method embodiments and therefore are described in a relatively simple manner, and reference may be made to some of the descriptions of the method embodiments for related points. The above-described system and system embodiments are only illustrative, wherein the units described as separate parts may or may not be physically separate, and the parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both.
To clearly illustrate this interchangeability of hardware and software, various illustrative components and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (10)
1. A metadata management system, comprising:
the system comprises a data profile module, a data index module, a data feedback module and a search engine module;
the data profiling module is used for scanning metadata information of the data warehouse in real time and calibrating data in the data asset library in real time based on the metadata information in the data warehouse; classifying each data in the data asset library based on a preset first classification rule to obtain a data list corresponding to each category in the data asset library; when user login is detected, displaying asset library information of the data asset library to the user based on the user type of the user, wherein the asset library information comprises each data list; when receiving search information sent by the search engine module, searching each target data list corresponding to the search information, and constructing each blood relationship corresponding to each target data by each target data associated with the target data list;
the data index module is used for performing index division on each data in each data list based on a preset second classification rule to obtain a plurality of index category data indexes under each data list;
the data feedback module is used for receiving a data correction message sent by a user and determining a feedback type corresponding to the data correction message, wherein the feedback type is a data type or an index type; sending the data correction message to a service end corresponding to the feedback type; when a feedback result corresponding to the data correction message fed back by the service end is received, calling the data profile module or the data index module to classify result data corresponding to the feedback result based on the feedback type;
the search engine module is used for receiving search information sent by a user and sending the search information to the data profile module, so that the data profile module queries data corresponding to the search information from the data asset library.
2. The metadata management system of claim 1, wherein the data profiling module is configured to determine whether new data exists in the data warehouse and whether updated data exists in the data warehouse based on the metadata information; when newly added data exist in the data warehouse, adding the newly added data to a preset newly added list; when updated data exist in the data warehouse, adding the updated data to a preset update list; storing the newly added data in the newly added list to the data asset library; updating the data in the data asset library based on the updated data in the update list.
3. The metadata management system according to claim 1, wherein the data profiling module is configured to, when the user type of the user is a general user, present each of the data listings of the data asset library and data details corresponding to each of the data listings to the user; and when the user type of the user is a management user, displaying each data list of the data asset library, data details corresponding to each data list and a data management module corresponding to the data asset library to the user.
4. The metadata management system according to claim 1, wherein the data profiling module is configured to obtain a table number included in the search information; determining a target data list in the data asset library which is matched with the table number; and searching each target data associated with the target data list based on the table number.
5. The metadata management system according to claim 1, wherein the data profile module is configured to associate each data matching the id based on a blood relationship between each target data associated with the target data list, obtain associated data, and send the associated data to the user.
6. The metadata management system according to any one of claims 1 to 5, wherein the data profile module is further configured to: and when a discarding instruction corresponding to any data list in the data asset library is received, canceling the display of the data list corresponding to the discarding instruction.
7. The metadata management system according to claim 6, wherein the data profile module is further configured to restore and display the data list corresponding to the restoration instruction when receiving the restoration instruction corresponding to the discarded data list.
8. The metadata management system according to claim 1, wherein the data feedback module is configured to obtain a feedback identifier included in the data modification message; when the feedback identifier is a first feedback identifier, determining that the feedback type corresponding to the data correction message is a data type; and when the feedback identifier is a second feedback identifier, determining that the feedback type corresponding to the data correction message is an index type.
9. The metadata management system according to claim 1 or 8, wherein the data feedback module is configured to record the feedback result to a preset first repository and invoke the data profiling module to classify the feedback result according to the first classification rule when the feedback type corresponding to the data modification message is a data type; and when the feedback type corresponding to the data correction message is an index type, recording the feedback result to a preset second storage library, and calling the data index module to classify the feedback result according to the second classification rule.
10. The metadata management system according to claim 1, wherein the search engine module is configured to obtain a library name and a data category included in the search information, and query data corresponding to the search information from the data asset library based on the library name and the data category.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210287993.XA CN114817149A (en) | 2022-03-23 | 2022-03-23 | Metadata management system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210287993.XA CN114817149A (en) | 2022-03-23 | 2022-03-23 | Metadata management system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114817149A true CN114817149A (en) | 2022-07-29 |
Family
ID=82531143
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210287993.XA Pending CN114817149A (en) | 2022-03-23 | 2022-03-23 | Metadata management system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114817149A (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070055680A1 (en) * | 2005-07-29 | 2007-03-08 | Craig Statchuk | Method and system for creating a taxonomy from business-oriented metadata content |
| US10791433B1 (en) * | 2019-03-29 | 2020-09-29 | Rovi Guides, Inc. | Systems and methods for personalizing messages based on a recipient relationship |
| CN111897836A (en) * | 2020-07-03 | 2020-11-06 | 中国建设银行股份有限公司 | Search system, method and storage medium |
| CN113468166A (en) * | 2020-03-31 | 2021-10-01 | 广州虎牙科技有限公司 | Metadata processing method and device, storage medium and server |
-
2022
- 2022-03-23 CN CN202210287993.XA patent/CN114817149A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070055680A1 (en) * | 2005-07-29 | 2007-03-08 | Craig Statchuk | Method and system for creating a taxonomy from business-oriented metadata content |
| US10791433B1 (en) * | 2019-03-29 | 2020-09-29 | Rovi Guides, Inc. | Systems and methods for personalizing messages based on a recipient relationship |
| CN113468166A (en) * | 2020-03-31 | 2021-10-01 | 广州虎牙科技有限公司 | Metadata processing method and device, storage medium and server |
| CN111897836A (en) * | 2020-07-03 | 2020-11-06 | 中国建设银行股份有限公司 | Search system, method and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9646284B1 (en) | Global inventory warehouse | |
| US8296311B2 (en) | Solution search for software support | |
| US10504047B2 (en) | Metadata-driven audit reporting system with dynamically created display names | |
| US8296317B2 (en) | Searchable object network | |
| US7593957B2 (en) | Hybrid data provider | |
| US20080133594A1 (en) | Application Directory | |
| US11934460B2 (en) | Linking business objects and documents | |
| US20090070237A1 (en) | Data reconciliation | |
| US20100235364A1 (en) | Data display server, data display method and program thereof | |
| US20230336500A1 (en) | Configurable views of context-relevant content | |
| US20090204588A1 (en) | Method and apparatus for determining key attribute items | |
| AU2010202186B2 (en) | Marketing asset exchange | |
| CN115374329B (en) | Method and system for managing enterprise business metadata and technical metadata | |
| US20100114926A1 (en) | Method, system, and computer program product for implementing automated worklists | |
| CN102004754A (en) | Navigation and visualization of relational database | |
| US7818328B2 (en) | API for obtaining unambiguous representation of objects in a relational database | |
| US8161075B1 (en) | Systems and methods for managing integrated and customizable data | |
| US7873607B1 (en) | Model driven consolidator of database information | |
| CN109670728B (en) | Ship design quality information management system based on database | |
| CN114817149A (en) | Metadata management system | |
| US20050060282A1 (en) | Patent family downloading system and method using selected downloading object | |
| US20050102302A1 (en) | Methods, systems and computer program products for segmented presentation of service order status information | |
| Hansen et al. | DWStar-automated star schema generation | |
| CN113765773B (en) | Method and device for processing communication record | |
| JP5567440B2 (en) | Audit preparation support device, audit preparation support system, and audit preparation support method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |