CN104699860A - Data processing and storage method for sharing-type master data - Google Patents
Data processing and storage method for sharing-type master data Download PDFInfo
- Publication number
- CN104699860A CN104699860A CN201510163449.4A CN201510163449A CN104699860A CN 104699860 A CN104699860 A CN 104699860A CN 201510163449 A CN201510163449 A CN 201510163449A CN 104699860 A CN104699860 A CN 104699860A
- Authority
- CN
- China
- Prior art keywords
- data
- master data
- master
- storage means
- operation system
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a data processing and storage method for sharing-type master data. The method is characterized by including the steps that extraction is performed, wherein master data of all information domains are extracted from all service systems or files; cleaning is preformed, wherein the extracted master data are cleaned, and dirty data are taken out; mapping is performed, wherein a mapping relation table between a master data system and all the service systems is established, and a mapping relation between master data of the service systems and master data of a master data management system is established; conversion is conducted, wherein interface data of all the service systems are converted so that data types of the interface data can be consistent with data standards of the master data management system; loading is performed, wherein the extracted, cleaned and converted data are loaded into a master database model. The method achieves data sharing, and avoids the problem of redundancy and inconsistency of data among all the systems.
Description
Technical field
The present invention relates to technical field of data storage, particularly, relate to a kind of data mart modeling storage means of shared master data.
Background technology
current business unit all also exists the professional operation system that big and small different times is built, and the deployment between system mainly exists with reticulate texture.And existing various piece uses respective system, the information between each system can not be enjoyed mutually, is formed " information island ".Along with the development of business unit's research change of business, the transition of economic society and infotech, the exchanges data between database is more and more frequent.In service application, often need to carry out complicated exchanges data, especially between different system and business, these exchanges data require cross-platform, cross-system, will realize the change of operational data structure and alternately multiple services simultaneously.When arranging the data between each system, because the data between each system exist redundancy, inconsistent problem, adopt existing data storage method obviously not too suitable.
Summary of the invention
The above-mentioned technical matters of the present invention in order to solve, provides a kind of data mart modeling storage means of shared master data.
The present invention's adopted technical scheme that solves the problem is:
A data mart modeling storage means for shared master data, is characterized in that, comprising:
A, extraction, extract each information field master data from each operation system or file;
B, cleaning, clean the master data be drawn into, and takes out dirty data;
C, mapping, set up main data system and each operation system data-mapping relation table, set up the mapping relations of operation system master data and master data management system master data;
D, conversion, change the interface data of each operation system, makes its data type consistent with the data standard of master data management system;
E, loading, by be drawn into, through cleaning and conversion data loading in master data base model.
So-called dirty data, the data namely in origin system are not in given scope or meaningless to practical business, or data layout is illegal, and in origin system, there is nonstandard coding and ambiguous service logic.Causing mainly because the design imprecision of origin system causes of dirty data, be mainly manifested in: data formatting error, data are inconsistent, Data duplication, mistake, and service logic is unreasonable, violate business rule etc.For example: the identity of invalidated, the date of invalidated, field etc.Data mart modeling storage means of the present invention life based on separate can not the sharing alternately of the information between existing each system.First enter to extract to the information in each system, dirty data is rejected thereupon, utilize and map, make main data system corresponding with each operation system data, docking port data are changed again, make data type match unanimously, finally reload in master data model, realize sharing of data, avoid the redundancy of each data among systems and inconsistent problem.
As preferably, in order to ensure the security of data, also comprise backup: the application framework data in master data management platform database and master data are backed up.
Further, also comprise the storage to the dirty data taken out in described cleaning step, dirty data is stored, be convenient to analyze use.
As preferably, for the ease of managing data, to when mapping between main data system and each operation system data, add timestamp simultaneously.
As preferably, for the ease of managing data, be just drawn into, through cleaning and conversion data loading in master data base model time, add timestamp.
To sum up, the invention has the beneficial effects as follows:
Method of the present invention is entered to extract to the information in each system, dirty data is rejected thereupon, utilize and map, make main data system corresponding with each operation system data, docking port data are changed again, make data type match unanimously, finally reload in master data model, realize sharing of data, avoid the redundancy of each data among systems and inconsistent problem.
Embodiment
Below in conjunction with embodiment, be clearly and completely described the technical scheme in the embodiment of the present invention, obviously, described embodiment is only the present invention's part embodiment, instead of whole embodiments.Based on the embodiment in the present invention, those of ordinary skill in the art, not making the every other embodiment obtained under creative work prerequisite, belong to the scope of protection of the invention.
Embodiment 1:
A data mart modeling storage means for shared master data, is characterized in that, comprising:
A, extraction, extract each information field master data from each operation system or file;
B, cleaning, clean the master data be drawn into, and takes out dirty data;
C, mapping, set up main data system and each operation system data-mapping relation table, set up the mapping relations of operation system master data and master data management system master data;
D, conversion, change the interface data of each operation system, makes its data type consistent with the data standard of master data management system;
E, loading, by be drawn into, through cleaning and conversion data loading in master data base model.
Data mart modeling storage means of the present invention life based on separate can not the sharing alternately of the information between existing each system.First enter to extract to the information in each system, dirty data is rejected thereupon, utilize and map, make main data system corresponding with each operation system data, docking port data are changed again, make data type match unanimously, finally reload in master data model, realize sharing of data, avoid the redundancy of each data among systems and inconsistent problem.
Embodiment 2:
In order to improve the security of data, the present embodiment is optimized on the basis of above-described embodiment, namely also comprises backup: back up the application framework data in master data management platform database and master data.
The storage to the dirty data taken out also is comprised in described cleaning step.
Embodiment 3:
For the ease of the management to data, the present embodiment is optimized on the basis of above-described embodiment, namely to when mapping between main data system and each operation system data, adds timestamp simultaneously.
Be just drawn into, through cleaning and conversion data loading in master data base model time, add timestamp.
As mentioned above, the present invention can be realized preferably.
Claims (5)
1. a data mart modeling storage means for shared master data, is characterized in that, comprising:
Extract, from each operation system or file, extract each information field master data;
Cleaning, cleans the master data be drawn into, and takes out dirty data;
Map, set up main data system and each operation system data-mapping relation table, set up the mapping relations of operation system master data and master data management system master data;
Conversion, changes the interface data of each operation system, makes its data type consistent with the data standard of master data management system;
Load, by be drawn into, through cleaning and conversion data loading in master data base model.
2. the data mart modeling storage means of a kind of shared master data according to claim 1, is characterized in that: also comprise backup: back up the application framework data in master data management platform database and master data.
3. the data mart modeling storage means of a kind of shared master data according to claim 1 and 2, is characterized in that: also comprise the storage to the dirty data taken out in described cleaning step.
4. the data mart modeling storage means of a kind of shared master data according to claim 1, is characterized in that: to when mapping between main data system and each operation system data, add timestamp simultaneously.
5. the data mart modeling storage means of a kind of shared master data according to claim 1, is characterized in that: be just drawn into, through cleaning and conversion data loading in master data base model time, add timestamp.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510163449.4A CN104699860A (en) | 2015-04-09 | 2015-04-09 | Data processing and storage method for sharing-type master data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510163449.4A CN104699860A (en) | 2015-04-09 | 2015-04-09 | Data processing and storage method for sharing-type master data |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN104699860A true CN104699860A (en) | 2015-06-10 |
Family
ID=53346980
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510163449.4A Pending CN104699860A (en) | 2015-04-09 | 2015-04-09 | Data processing and storage method for sharing-type master data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104699860A (en) |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104993958A (en) * | 2015-06-29 | 2015-10-21 | 北京京东尚科信息技术有限公司 | Method and system for generating user master data |
| CN106126629A (en) * | 2016-06-22 | 2016-11-16 | 武汉斗鱼网络科技有限公司 | A kind of master data management method and system based on live industry |
| CN108052645A (en) * | 2017-12-26 | 2018-05-18 | 重庆信联达软件有限公司 | Inside data of enterprise standardized management method |
| CN108121809A (en) * | 2017-12-26 | 2018-06-05 | 重庆信联达软件有限公司 | Inside data of enterprise standardizes implementation method |
| CN108156061A (en) * | 2017-12-26 | 2018-06-12 | 重庆信联达软件有限公司 | Esb monitoring service platforms |
| CN108173711A (en) * | 2017-12-26 | 2018-06-15 | 重庆信联达软件有限公司 | Enterprises system data exchange monitoring method |
| CN108197192A (en) * | 2017-12-26 | 2018-06-22 | 重庆信联达软件有限公司 | It is used to implement the main data system of inside data of enterprise standardization |
| CN110389845A (en) * | 2019-07-30 | 2019-10-29 | 浪潮软件集团有限公司 | A method of data broadcasting is carried out based on master data management |
| CN111324597A (en) * | 2020-03-19 | 2020-06-23 | 北京三维天地科技股份有限公司 | Main data management method and system |
| CN112528331A (en) * | 2020-12-15 | 2021-03-19 | 杭州默安科技有限公司 | Privacy disclosure risk detection method, device and system |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070198593A1 (en) * | 2005-11-28 | 2007-08-23 | Anand Prahlad | Systems and methods for classifying and transferring information in a storage network |
| US20140059024A1 (en) * | 2012-08-27 | 2014-02-27 | Ss8 Networks, Inc. | System and method of storage, recovery, and management of data intercepted on a communication network |
| CN103853843A (en) * | 2014-03-20 | 2014-06-11 | 浪潮集团山东通用软件有限公司 | Method for realizing data concentration across security domains based on main data mapping |
-
2015
- 2015-04-09 CN CN201510163449.4A patent/CN104699860A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070198593A1 (en) * | 2005-11-28 | 2007-08-23 | Anand Prahlad | Systems and methods for classifying and transferring information in a storage network |
| US20140059024A1 (en) * | 2012-08-27 | 2014-02-27 | Ss8 Networks, Inc. | System and method of storage, recovery, and management of data intercepted on a communication network |
| CN103853843A (en) * | 2014-03-20 | 2014-06-11 | 浪潮集团山东通用软件有限公司 | Method for realizing data concentration across security domains based on main data mapping |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104993958A (en) * | 2015-06-29 | 2015-10-21 | 北京京东尚科信息技术有限公司 | Method and system for generating user master data |
| CN106126629A (en) * | 2016-06-22 | 2016-11-16 | 武汉斗鱼网络科技有限公司 | A kind of master data management method and system based on live industry |
| CN106126629B (en) * | 2016-06-22 | 2019-07-09 | 武汉斗鱼网络科技有限公司 | A kind of master data management method and system based on live streaming industry |
| CN108052645A (en) * | 2017-12-26 | 2018-05-18 | 重庆信联达软件有限公司 | Inside data of enterprise standardized management method |
| CN108121809A (en) * | 2017-12-26 | 2018-06-05 | 重庆信联达软件有限公司 | Inside data of enterprise standardizes implementation method |
| CN108156061A (en) * | 2017-12-26 | 2018-06-12 | 重庆信联达软件有限公司 | Esb monitoring service platforms |
| CN108173711A (en) * | 2017-12-26 | 2018-06-15 | 重庆信联达软件有限公司 | Enterprises system data exchange monitoring method |
| CN108197192A (en) * | 2017-12-26 | 2018-06-22 | 重庆信联达软件有限公司 | It is used to implement the main data system of inside data of enterprise standardization |
| CN108156061B (en) * | 2017-12-26 | 2021-06-22 | 重庆信联达软件有限公司 | esb monitoring service platform |
| CN110389845A (en) * | 2019-07-30 | 2019-10-29 | 浪潮软件集团有限公司 | A method of data broadcasting is carried out based on master data management |
| CN111324597A (en) * | 2020-03-19 | 2020-06-23 | 北京三维天地科技股份有限公司 | Main data management method and system |
| CN112528331A (en) * | 2020-12-15 | 2021-03-19 | 杭州默安科技有限公司 | Privacy disclosure risk detection method, device and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104699860A (en) | Data processing and storage method for sharing-type master data | |
| CN103646111B (en) | System and method for realizing real-time data association in big data environment | |
| CN105069109B (en) | A kind of method and system of distributed data base dilatation | |
| CN105068856A (en) | Mirror image snapshooting based on virtual machine backup system and backup method | |
| CN104102790B (en) | Power supply figure automatic mapping system and method based on GIS | |
| CN107391306A (en) | A kind of isomeric data library backup file access pattern method | |
| CN105912691A (en) | Power grid model multi-version multi-tenant management system and method based on distributed storage | |
| CN105635311A (en) | Method for synchronizing resource pool information in cloud management platform | |
| CN101770518A (en) | Metadata management method of power grid enterprise integrated information die | |
| CN106708941A (en) | Multi-task on-line collaborative editing method for power grid | |
| CN104408150A (en) | Data import/ export method and device adapted to a plurality of data formats of databases | |
| CN102495916A (en) | Multi-application-system panoramic modeling method based on object matching | |
| CN109298978A (en) | A kind of restoration methods and system of the data-base cluster of designated position | |
| CN104699857A (en) | Big data storage method based on knowledge engineering | |
| CN108228726A (en) | The increment unusual fluctuation content acquisition method and storage medium of power distribution network red and black figure | |
| CN109361777A (en) | Synchronous method, synchronization system and the relevant apparatus of distributed type assemblies node state | |
| CN106155838A (en) | A kind of database back-up data restoration methods and device | |
| CN106326082A (en) | Method and device for recording log in network system | |
| CN107273449B (en) | Breakpoint processing method and system based on memory database | |
| CN106802928B (en) | Power grid historical data management method and system | |
| CN103441988B (en) | Across GIS platform data migration method | |
| CN103207916B (en) | The method and apparatus of metadata processing | |
| CN102479205B (en) | Generation method and system of change order | |
| CN106250501A (en) | Report processing method and reporting system | |
| CN105930380A (en) | Chart monitor method and device based on HADOOP |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20150610 |
|
| WD01 | Invention patent application deemed withdrawn after publication |