CN109241179A - Data administering method, system and computer equipment based on data space - Google Patents
Data administering method, system and computer equipment based on data space Download PDFInfo
- Publication number
- CN109241179A CN109241179A CN201810864946.0A CN201810864946A CN109241179A CN 109241179 A CN109241179 A CN 109241179A CN 201810864946 A CN201810864946 A CN 201810864946A CN 109241179 A CN109241179 A CN 109241179A
- Authority
- CN
- China
- Prior art keywords
- data
- space
- mentioned
- source
- layers
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
Present invention discloses a kind of data administering method, system and computer equipment based on data space, comprising steps of building data space source data layer；Data space BI layers is constructed according to the data of data space source data layer and object module；Data space AI layers is constructed according to the data of data space BI layers of building of data, object module and data spatial source data Layer；Corresponding data service is provided according to AI layers of data space of data for third-party application.Data administering method based on data space of the invention, system and computer equipment have the beneficial effect that through building data space source data layer, BI layers of data space and AI layers of data space, form the space with abstraction hierarchy, make to be extended from depth, range and time dimension when more interoperability of system or bigger business occur and meet the needs of data service.
Description
Technical field
The present invention relates to big data field, especially relates to a kind of data administering method based on data space, is
System and computer equipment.
Background technique
Interoperability refers to that different computer systems, network, operating system and application program work together and share letter
The ability of breath is to determine that information system moves towards the key foundation of intelligent use from data management.However for a long time, mostly
System is established on the basis of certain business demands, not compatible with other systems from bigger global consideration, is caused
Data standard according to different, business bore disunity, data consistency and availability it is particularly thorny.
The target that data are administered is so that data can be exchanged and be executed with cross-system, and existing data Treatment process method is big
It is consider that the mode that " downward to pushing up " is taken by method or metadata management system based on data standard is realized more, that is, it needs
The multi-side system for participating in interoperability first reaches an agreement to metadata, final to realize the not homologous ray understanding common to data.
Traditional data administering method is also being continuously increased with the increase for participating in inter-operation system, data, it is difficult in different systems
Between reach the consistent of data standard and source data.
Summary of the invention
The main object of the present invention is to provide a kind of data administering method, system and computer based on data space to set
It is standby, the technical issues of to solve mentioned in above-mentioned background technique.
The present invention proposes a kind of data administering method based on data space, comprising steps of
Construct data space source data layer；
Data space BI layers is constructed according to the data of above-mentioned data space source data layer and object module；
According to BI layers of above-mentioned building data space of data, the number of above-mentioned object module and above-mentioned data space source data layer
According to data space AI layers of building；
Corresponding data service is provided according to above-mentioned data space AI layers of data for third-party application.
Further, above-mentioned according to above-mentioned data space source number in the data administering method above-mentioned based on data space
According to layer data and object module construct data space BI layers the step of comprising steps of
Corresponding above-mentioned object module is obtained according to the type of service of above-mentioned third-party application and data target type；
According to the data of above-mentioned data space source data layer and above-mentioned data space BI layers of the building of above-mentioned object module.
Further, above-mentioned according to above-mentioned building data space in the data administering method above-mentioned based on data space
The step of data of BI layers of data, above-mentioned object module and above-mentioned data space source data layer construct data space AI layers include
Step:
It makes inferences to obtain unknown data according to BI layers of data of above-mentioned building data space；
Above-mentioned data are constructed according to above-mentioned unknown data, the data of above-mentioned data space source data layer and above-mentioned object module
Space AI layers.
Further, in the data administering method above-mentioned based on data space, above-mentioned building data space source data layer
The step of before, further comprise the steps of:
The native data of different business systems is obtained, data lake is formed；
According to the type of service of above-mentioned third-party application and data target type to the native data in above-mentioned data lake into
The identification of row data and cleaning, obtain source data.
Further, in the data administering method above-mentioned based on data space, above-mentioned building data space source data layer
The step of comprising steps of
Body data warehouse is formed according to above-mentioned source data, and constructs above-mentioned data space source data layer.
Further, in the data administering method above-mentioned based on data space, in the above-mentioned step for obtaining above-mentioned source data
After rapid, further comprise the steps of:
Above-mentioned source data is tracked, and grade is carried out to above-mentioned source data according to the source-information of above-mentioned source data and is commented
Estimate.
The present invention also proposes a kind of data governing system based on data space, comprising:
Source data layer building module, for constructing data space source data layer；
BI layer building module, for constructing data space according to the data and object module of above-mentioned data space source data layer
BI layers；
AI layer building module, for according to BI layers of above-mentioned building data space of data, above-mentioned object module and above-mentioned number
Data space AI layers is constructed according to the data of spatial source data Layer；
API module, for providing corresponding data service according to above-mentioned data space AI layers of data for third-party application.
Further, the above-mentioned data governing system based on data space, further includes:
Modeling tool and model library module, for being obtained according to the type of service and data target type of above-mentioned third-party application
Take corresponding above-mentioned object module；
Inquiry and inference engine module, it is unknown for making inferences to obtain according to BI layers of above-mentioned building data space of data
Data.
Further, the above-mentioned data governing system based on data space, further includes:
Data lake module forms data lake for obtaining the native data of different business systems；
Identification and cleaning module, for the type of service and data target type according to above-mentioned third-party application to above-mentioned number
Data identification and cleaning are carried out according to the native data in lake, obtains source data；
Data blood relationship tracing module, is tracked above-mentioned source data, and according to the source-information of above-mentioned source data to upper
It states source data and carries out grade assessment.
The present invention also proposes a kind of computer equipment, including memory, processor and storage are on a memory and can be
The computer program run on processor, above-mentioned processor are realized as described in any one of embodiment when executing above procedure
Method.
Data administering method based on data space, system and having the beneficial effect that for computer equipment of the invention passes through
Building data space source data layer, BI layers of data space and AI layers of data space form the space with abstraction hierarchy, make
When more interoperability of system or bigger business occur, it can be extended and meet from depth, range and time dimension
The demand of data service；And data space AI is improved on the basis of data space source data layer and I layers of data space B of data
The data of layer, find and infer deeper data, so that the known knowledge contained from data infers unknown knowledge.
Detailed description of the invention
Fig. 1 is the flow diagram of the data administering method based on data space of one embodiment of the invention；
Fig. 2 is the flow diagram of the data administering method based on data space of one embodiment of the invention；
Fig. 3 is the flow diagram of the data administering method based on data space of one embodiment of the invention；
Fig. 4 is the flow diagram of the data administering method based on data space of one embodiment of the invention；
Fig. 5 is the flow diagram of the data administering method based on data space of one embodiment of the invention；
Fig. 6 is the flow diagram of the data administering method based on data space of one embodiment of the invention；
Fig. 7 is the structural schematic diagram of the data governing system based on data space of one embodiment of the invention；
Fig. 8 is a kind of structural schematic diagram of computer equipment of one embodiment of the invention.
1, source data layer building module；2, BI layer building module；3, AI layer building module；4, API module；5, modeling tool
And model library module；6, inquiry and inference engine module；7, data lake module；8, identification and cleaning module；9, data blood relationship chases after
Track module；10, computer equipment；11, external equipment；12, processing unit；13, bus；14, network adapter；15, (I/O) connects
Mouthful；16, display；17, system storage；18, random access memory (RAM)；19, cache memory；20, storage system
System；21, program/utility；22, program module.
The embodiments will be further described with reference to the accompanying drawings for the realization, the function and the advantages of the object of the present invention.
Specific embodiment
Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention carries out clear, complete
Site preparation description, it is clear that described embodiment is only a part of the embodiments of the present invention, instead of all the embodiments.Base
Embodiment in the present invention, it is obtained by those of ordinary skill in the art without making creative efforts it is all its
His embodiment, shall fall within the protection scope of the present invention.
It in addition, the technical solution between each embodiment can be combined with each other, but must be with ordinary skill
Based on personnel can be realized, this technical side will be understood that when the combination of technical solution appearance is conflicting or cannot achieve
The combination of case is not present, also not the present invention claims protection scope within.
Referring to Fig.1, in embodiments of the present invention, the present invention proposes a kind of data administering method based on data space, packet
Include step:
S1, building data space source data layer；
S2, data space BI layers is constructed according to the data and object module of above-mentioned data space source data layer；
S3, according to BI layers of above-mentioned building data space of data, above-mentioned object module and above-mentioned data space source data layer
Data construct data space AI layers；
S4, corresponding data service is provided for third-party application according to above-mentioned data space AI layers of data.
Such as above-mentioned steps S1, data space source data layer is constructed, different numbers is extracted from different types of operation system
According to, and integration processing is carried out to data, to construct above-mentioned data space source data layer, wherein above-mentioned data space source number
It is based on the databases such as Apache Hadoop, MySQL, MangoDB according to layer.
Such as above-mentioned steps S2, data space BI is constructed according to the data of above-mentioned data space source data layer and object module
Layer, when constructing above-mentioned data space BI layers using different types of data, above-mentioned object module is also different, can to construct
Above-mentioned data space BI layers of the data of the above-mentioned data space source data layer of alignment processing, wherein the algorithm of above-mentioned object module
It generally comprises: linear regression, cluster, classification, time series, decision tree, nerve center method, improper detection, relevance, factor
Analysis, text mining, support vector machines, sequential mining, naive Bayesian, random forest or TensorFlow.
It is empty according to BI layers of above-mentioned building data space of data, above-mentioned object module and above-mentioned data such as above-mentioned steps S3
Between source data layer data construct data space AI layers, to be improved on the basis of above-mentioned data space BI layers of data above-mentioned
The data that AI layers of data space find and infer deeper data, thus the known knowledge reasoning contained from data
Unknown knowledge out, above-mentioned data space source data layer, above-mentioned data space BI layers and above-mentioned data space AI layers of combination formation one
A space with abstraction hierarchy makes when more interoperability of system or bigger business occur, can from depth, range and
Time dimension is extended and meets the needs of data service.
Such as above-mentioned steps S4, corresponding data are provided for third-party application according to above-mentioned data space AI layers of data and are taken
Business makes above-mentioned third-party application can use above-mentioned data space source data layer, above-mentioned data space BI layers and above-mentioned data sky
Between AI layers of realization data administer, above-mentioned third-party application can be based on above-mentioned data space source data layer, above-mentioned data space BI
Layer and the building of above-mentioned data space AI layers of data meet the business service of more application scenarios.
Referring to Fig. 2, in the present embodiment, above-mentioned above-mentioned source data and target mould according to above-mentioned data space source data layer
Type construct data space BI layers the step of comprising steps of
S5, corresponding above-mentioned object module is obtained according to the type of service and data target type of above-mentioned third-party application；
S6, above-mentioned data space BI layers are constructed according to the data of above-mentioned data space source data layer and above-mentioned object module.
Such as above-mentioned steps S5, obtained according to the type of service of above-mentioned third-party application and data target type corresponding above-mentioned
Object module is constructed according to different types of service to the above-mentioned object module for being applied to extract with calculate, and above-mentioned data
The type of the data of spatial source data Layer is corresponding with the type of service of above-mentioned third-party application and data target type, wherein
The algorithm of above-mentioned object module generally comprises: linear regression, cluster, classification, time series, decision tree, nerve center method, it is non-just
Normal detection, relevance, factor analysis, text mining, support vector machines, sequential mining, naive Bayesian, random forest or
TensorFlow。
Such as above-mentioned steps S6, above-mentioned data are constructed according to the data of above-mentioned data space source data layer and above-mentioned object module
I layers of space B, to construct above-mentioned data space BI layers of the data for capableing of the above-mentioned data space source data layer of alignment processing.
Referring to Fig. 3, in the present embodiment, the above-mentioned data according to BI layers of above-mentioned building data space and above-mentioned object module
Building data space AI layers the step of comprising steps of
S7, it makes inferences to obtain unknown data according to BI layers of above-mentioned building data space of data,；
S8, according to above-mentioned unknown data, the data of above-mentioned data space source data layer and above-mentioned object module construct it is above-mentioned
AI layers of data space.
Such as above-mentioned steps S7, make inferences to obtain unknown data according to BI layers of data of above-mentioned building data space, thus
Go out above-mentioned unknown data according to above-mentioned data space source data layer and above-mentioned data space BI layers of relation inference, wherein above-mentioned
Unknown data includes qualitative data and quantitative data.
Such as above-mentioned steps S8, according to above-mentioned unknown data, the data of above-mentioned data space source data layer and above-mentioned target mould
Above-mentioned data space AI layers of type building, thus according to above-mentioned unknown data and on the basis of above-mentioned data space BI layers of data
Above-mentioned data space AI layers of data are improved, find and infer deeper data, thus known to containing from data
Knowledge reasoning goes out unknown knowledge, wherein the algorithm of above-mentioned object module generally comprises: linear regression, cluster, classification, time system
Column, decision tree, nerve center method, improper detection, relevance, factor analysis, text mining, support vector machines, sequential mining,
Naive Bayesian, random forest or TensorFlow.
Referring to Fig. 4, in the present embodiment, the step of above-mentioned building data space source data layer before, further comprise the steps of:
S9, the native data for obtaining different business systems form data lake；
S10, according to the type of service and data target type of above-mentioned third-party application to the primary number in above-mentioned data lake
According to data identification and cleaning is carried out, source data is obtained.
Such as above-mentioned steps S9, the native data of different business systems is obtained, forms data lake, using data extraction tool,
Above-mentioned native data is directly extracted from different operation systems, and is converged in above-mentioned data lake, wherein in above-mentioned data lake
In data be above-mentioned initial data real-time or near real-time mirror image.
Such as above-mentioned steps S10, according to the type of service of above-mentioned third-party application and data target type to above-mentioned data lake
In native data carry out data identification and cleaning, obtain source data, wherein the native data in above-mentioned data lake is to be based on
The realization of the tools such as Kafka, Sqoop is obtained from different business systems, and above-mentioned source data is using the methods of ETL to above-mentioned number
The identification of data is carried out according to the initial data in lake and cleaning obtains.
Referring to Fig. 5, in the present embodiment, the step of above-mentioned building data space source data layer comprising steps of
S11, body data warehouse is formed according to above-mentioned source data, and constructs above-mentioned data space source data layer.
Such as above-mentioned steps S11, body data warehouse is formed according to above-mentioned source data, and constructs above-mentioned data space source data
Layer, different source datas is extracted from different types of operation system, and carry out being integrally formed above-mentioned master according to above-mentioned source data
Volume data warehouse, so that the aforementioned body data warehouse of different levels and type is formed, to construct above-mentioned data space source
Data Layer, wherein above-mentioned data space source data layer is based on the databases such as Apache Hadoop, MySQL, MangoDB.
It after above-mentioned the step of obtaining above-mentioned source data, is further comprised the steps of: in the present embodiment referring to Fig. 6
S12, above-mentioned source data is tracked, and according to the source-information of above-mentioned source data above-mentioned source data is carried out etc.
Grade assessment.
Such as above-mentioned steps S12, above-mentioned source data is tracked, and according to the source-information of above-mentioned source data to above-mentioned source
Data carry out grade assessment, and the feasibility degree of the above-mentioned source data of different stage has differences, and above-mentioned source data source is closer
The higher grade of source data, above-mentioned source data, and confidence level is also higher.
Referring to Fig.1-6, in the present embodiment, the data administering method based on data space, comprising steps of
S9, the native data for obtaining different business systems form data lake；
S10, according to the type of service and data target type of above-mentioned third-party application to the primary number in above-mentioned data lake
According to data identification and cleaning is carried out, above-mentioned source data is obtained；
S12, above-mentioned source data is tracked, and according to the source-information of above-mentioned source data above-mentioned source data is carried out etc.
Grade assessment；
S11, data space source data layer is constructed according to above-mentioned source data；
S5, corresponding above-mentioned object module is obtained according to the type of service and data target type of above-mentioned third-party application；
S6, above-mentioned data space BI layers are constructed according to above-mentioned source data and above-mentioned object module；
S7, it makes inferences to obtain unknown data according to BI layers of above-mentioned building data space of data；
S8, above-mentioned data space AI layers are constructed according to above-mentioned unknown data, above-mentioned source data and above-mentioned object module；
S4, corresponding data service is provided for third-party application according to above-mentioned data space AI layers of data.
Referring to Fig. 7, the present invention also proposes a kind of data governing system based on data space, comprising:
Source data layer building module 1 is extracted for constructing data space source data layer from different types of operation system
Different data, and integration processing is carried out to data, to construct above-mentioned data space source data layer, wherein above-mentioned data
Spatial source data Layer is based on the databases such as Apache Hadoop, MySQL, MangoDB；
BI layer building module 2, it is empty for constructing data according to the data and object module of above-mentioned data space source data layer
Between BI layers, using different types of data construct above-mentioned data space BI layers when, above-mentioned object module is also different, to construct
It is capable of above-mentioned data space BI layers of the data of the above-mentioned data space source data layer of alignment processing, wherein above-mentioned object module
Algorithm generally comprises: linear regression, cluster, classification, time series, decision tree, nerve center method, improper detection, relevance,
Factor analysis, text mining, support vector machines, sequential mining, naive Bayesian, random forest or TensorFlow；
AI layer building module 3, for according to BI layers of above-mentioned building data space of data, above-mentioned object module and above-mentioned number
Data space AI layers is constructed according to the data of spatial source data Layer, thus perfect on the basis of above-mentioned data space BI layers of data
Above-mentioned data space AI layers of data find and infer deeper data, thus the known knowledge contained from data
Infer unknown knowledge, above-mentioned data space source data layer, above-mentioned data space BI layers and above-mentioned data space AI layers of combination shape
At a space with abstraction hierarchy, make when more interoperability of system or bigger business occur, it can be from depth, wide
Degree and time dimension are extended and meet the needs of data service；
API module 4 takes for providing corresponding data according to above-mentioned data space AI layers of data for third-party application
Business makes above-mentioned third-party application can use above-mentioned data space source data layer, above-mentioned data space BI layers and above-mentioned data sky
Between AI layers of realization data administer, above-mentioned third-party application can be based on above-mentioned data space source data layer, above-mentioned data space BI
Layer and the building of above-mentioned data space AI layers of data meet the business service of more application scenarios.
In the present embodiment, further includes:
Modeling tool and model library module 5, for the type of service and data target type according to above-mentioned third-party application
Corresponding above-mentioned object module is obtained, according to different type of service buildings to the above-mentioned target mould for being applied to extract with calculate
Type, and the type of service and data target class of the type of the data of above-mentioned data space source data layer and above-mentioned third-party application
Type is corresponding, wherein the algorithm of above-mentioned object module generally comprises: linear regression, cluster, classification, time series, decision tree,
Nerve center method, improper detection, relevance, factor analysis, text mining, support vector machines, sequential mining, simple pattra leaves
This, random forest or TensorFlow；
Inquiry and inference engine module 6, for making inferences to obtain not according to BI layers of above-mentioned building data space of data
Primary data, to go out above-mentioned unknown number according to above-mentioned data space source data layer and above-mentioned data space BI layers of relation inference
According to, wherein above-mentioned unknown data includes qualitative data and quantitative data.
In the present embodiment, further includes:
Data lake module 7 forms data lake, utilizes data pick-up work for obtaining the native data of different business systems
Tool, directly extracts above-mentioned native data, and converge in above-mentioned data lake, wherein in above-mentioned number from different operation systems
It is the real-time or near real-time mirror image of above-mentioned initial data according to the data in lake；
Identification and cleaning module 8, for the type of service and data target type according to above-mentioned third-party application to above-mentioned
Native data in data lake carries out data identification and cleaning, obtains source data, wherein the native data in above-mentioned data lake is
It is realized based on tools such as Kafka, Sqoop and is obtained from different business systems, and above-mentioned source data is using the methods of ETL to upper
State identification and cleaning acquisition that the initial data in data lake carries out data；
Data blood relationship tracing module 9, is tracked above-mentioned source data, and according to the source-information of above-mentioned source data to upper
It states source data and carries out grade assessment, the feasibility degree of the above-mentioned source data of different stage has differences, and above-mentioned source data source is got over
Close to source data, the higher grade of above-mentioned source data, and confidence level is also higher.
In the present embodiment, further includes:
Secondary BI layer building module, for according to the data of above-mentioned data space source data layer and the building of above-mentioned object module
BI layers of data space are stated, to construct the above-mentioned data space for capableing of the data of the above-mentioned data space source data layer of alignment processing
BI layers；
Secondary AI layer building module, for according to the data of above-mentioned unknown data, above-mentioned data space source data layer and above-mentioned
Above-mentioned data space AI layers of object module building, thus according to above-mentioned unknown data and in above-mentioned data space BI layers of data
On the basis of improve above-mentioned data space AI layers of data, find and infer deeper data, to contain from data
Known knowledge infer unknown knowledge, wherein the algorithm of above-mentioned object module generally comprises: linear regression, cluster, classification,
Time series, decision tree, nerve center method, improper detection, relevance, factor analysis, text mining, support vector machines, sequence
Arrange excavation, naive Bayesian, random forest or TensorFlow；
Body data warehouse module, for forming body data warehouse according to above-mentioned source data, and it is empty to construct above-mentioned data
Between source data layer, different source datas is extracted from different types of operation system, and integration shape is carried out according to above-mentioned source data
At aforementioned body data warehouse, so that the aforementioned body data warehouse of different levels and type is formed, to construct above-mentioned number
According to spatial source data Layer, wherein above-mentioned data space source data layer is based on the data such as Apache Hadoop, MySQL, MangoDB
Library；
Quality of data monitoring instrument module, for monitoring the quality of collected data, once monitor above-mentioned data
Quality changes, and just gives and warns in advance.
Data security protecting tool desensitization protection, the production peace such as encrypts for carrying out to the data for being related to secret or privacy
Full data copy or setting data access authority.
Referring to Fig. 8, in embodiments of the present invention, the present invention also provides a kind of computer equipment, above-mentioned computer equipment 10
It is showed in the form of universal computing device, the component of computer equipment 10 can include but is not limited to: one or more processing
Device or processing unit 10, system storage 17 connect different system components (including system storage 17 and processing unit 12)
Bus 13；
Bus 13 indicates one of a few class bus structures or a variety of, including memory bus or Memory Controller,
Peripheral bus, graphics acceleration port, processor or the local bus using any bus structures in a variety of bus structures.It lifts
For example, these architectures include but is not limited to industry standard architecture (ISA) bus, microchannel architecture (MAC)
Bus, enhanced isa bus, Video Electronics Standards Association (VESA) local bus and peripheral component interconnection (PCI) bus.
Computer equipment 10 typically comprises a variety of computer system readable media.These media can be it is any can be by
The usable medium that computer equipment 10 accesses, including volatile and non-volatile media, moveable and immovable medium.
System storage 17 may include the computer system readable media of form of volatile memory, such as arbitrary access
Memory (RAM) 18 and/or cache memory 19.Computer equipment 10 may further include other movement/it is not removable
Dynamic, volatile/non-volatile computer decorum storage medium.Only as an example, storage system 20 can be used for read and write can not
Mobile, non-volatile magnetic media (commonly referred to as " hard disk drive ").Although being not shown in Fig. 8, can provide for can
The disc driver of mobile non-volatile magnetic disk (such as " floppy disk ") read-write, and to removable anonvolatile optical disk (such as CD~
ROM, DVD~ROM or other optical mediums) read-write CD drive.In these cases, each driver can pass through one
A or multiple data media interfaces are connected with bus 13.Memory may include at least one program product, the program product
With one group of (for example, at least one) program module 22, these program modules 22 are configured to perform the function of various embodiments of the present invention
Energy.
Program/utility 21 with one group of (at least one) program module 22, can store in memory, for example,
Such program module 22 includes --- but being not limited to --- operating system, one or more application program, other program moulds
It may include the realization of network environment in block and program data, each of these examples or certain combination.Program module
22 usually execute function and/or method in embodiment described in the invention.
Computer equipment 10 can also with one or more external equipments 11 (such as keyboard, sensing equipment, display 16,
Camera etc.) communication, the equipment interacted with the computer equipment 10 can be also enabled a user to one or more to be communicated, and/
Or with the computer equipment 10 is communicated with one or more of the other calculating equipment any equipment (such as network interface card,
Modem etc.) communication.This communication can be carried out by input/output (I/O) interface 15.Also, computer equipment
10 can also by network adapter 14 and one or more network (such as local area network (LAN)), wide area network (WAN) and/or
Public network (such as internet) communication.As shown, network adapter 14 passes through other of bus 13 and computer equipment 10
Module communication.It should be understood that although being not shown in Fig. 8 other hardware and/or software mould can be used in conjunction with computer equipment 10
Block, including but not limited to: microcode, device driver, redundant processing unit, external disk drive array, RAID system, tape
Driver and data backup storage system etc..
Processing unit 12 by the program that is stored in system storage 17 of operation, thereby executing various function application and
Data processing, such as realize the data administering method based on data space provided by the embodiment of the present invention.
That is, above-mentioned processing unit 12 is realized when executing above procedure: building data space source data layer, and according to above-mentioned
The data and object module of data space source data layer construct data space BI layers, further according to BI layers of above-mentioned building data space
The data of data, above-mentioned object module and above-mentioned data space source data layer construct data space AI layers, also, according to above-mentioned number
Corresponding data service is provided according to space AI layers of data for third-party application.
Data administering method based on data space, system and having the beneficial effect that for computer equipment of the invention passes through
Construct above-mentioned data space source data layer, data space BI layers and data space AI layers above-mentioned above-mentioned, formed one have it is abstract
The space of level makes when more interoperability of system or bigger business occur, can from depth, range and time dimension into
Row extends and meets the needs of data service；And in above-mentioned data space source data layer and above-mentioned data space BI layers of data
On the basis of improve above-mentioned data space AI layers of data, find and infer deeper data, to contain from data
Known knowledge infer unknown knowledge.
The above description is only a preferred embodiment of the present invention, is not intended to limit the scope of the invention, all utilizations
Equivalent structure or equivalent flow shift made by description of the invention and accompanying drawing content is applied directly or indirectly in other correlations
Technical field, be included within the scope of the present invention.
Claims (10)
1. a kind of data administering method based on data space, which is characterized in that comprising steps of
Construct data space source data layer；
Data space BI layers is constructed according to the data of the data space source data layer and object module；
According to the data of data space BI layers of the building, the data structure of the object module and the data space source data layer
Build AI layers of data space；
Corresponding data service is provided according to data space AI layers of the data for third-party application.
2. the data administering method according to claim 1 based on data space, which is characterized in that described according to the number
According to spatial source data Layer data and object module construct data space BI layers the step of comprising steps of
The corresponding object module is obtained according to the type of service of the third-party application and data target type；
According to the data of the data space source data layer and described data space BI layers of object module building.
3. the data administering method according to claim 1 based on data space, which is characterized in that described according to the structure
Build data space AI layers of the data building of BI layers of data space of data, the object module and the data space source data layer
The step of comprising steps of
It makes inferences to obtain unknown data according to the data of data space BI layers of the building；
The data space is constructed according to the unknown data, the data of the data space source data layer and the object module
AI layers.
4. the data administering method according to claim 1 based on data space, which is characterized in that the building data are empty
Between source data layer the step of before, further comprise the steps of:
The native data of different business systems is obtained, data lake is formed；
The native data in the data lake is counted according to the type of service of the third-party application and data target type
According to identification and cleaning, source data is obtained.
5. the data administering method according to claim 4 based on data space, which is characterized in that the building data are empty
Between source data layer the step of comprising steps of
Body data warehouse is formed according to the source data, and constructs the data space source data layer.
6. the data administering method according to claim 4 based on data space, which is characterized in that described in described obtain
After the step of source data, further comprise the steps of:
The source data is tracked, and grade assessment is carried out to the source data according to the source-information of the source data.
7. a kind of data governing system based on data space characterized by comprising
Source data layer building module, for constructing data space source data layer；
BI layer building module, for constructing data space BI according to the data and object module of the data space source data layer
Layer；
AI layer building module, it is empty for data, the object module and the data according to data space BI layers of the building
Between source data layer data construct data space AI layers；
API module, for providing corresponding data service according to data space AI layers of the data for third-party application.
8. the data governing system according to claim 7 based on data space, which is characterized in that further include:
Modeling tool and model library module, for according to the acquisition pair of the type of service and data target type of the third-party application
The object module answered；
Inquiry and inference engine module, for making inferences to obtain unknown number according to the data of data space BI layers of the building
According to.
9. the data governing system according to claim 7 based on data space, which is characterized in that further include:
Data lake module forms data lake for obtaining the native data of different business systems；
Identification and cleaning module, for the type of service and data target type according to the third-party application to the data lake
In native data carry out data identification and cleaning, obtain source data；
Data blood relationship tracing module, is tracked the source data, and according to the source-information of the source data to the source
Data carry out grade assessment.
10. a kind of computer equipment, can run on a memory and on a processor including memory, processor and storage
Computer program, which is characterized in that the processor is realized when executing described program such as any one of claim 1~6 institute
The method stated.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810864946.0A CN109241179A (en) | 2018-08-01 | 2018-08-01 | Data administering method, system and computer equipment based on data space |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810864946.0A CN109241179A (en) | 2018-08-01 | 2018-08-01 | Data administering method, system and computer equipment based on data space |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109241179A true CN109241179A (en) | 2019-01-18 |
Family
ID=65073417
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810864946.0A Pending CN109241179A (en) | 2018-08-01 | 2018-08-01 | Data administering method, system and computer equipment based on data space |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109241179A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110263229A (en) * | 2019-06-27 | 2019-09-20 | 北京中油瑞飞信息技术有限责任公司 | A kind of data administering method and device based on data lake |
| CN110457405A (en) * | 2019-08-20 | 2019-11-15 | 上海观安信息技术股份有限公司 | A kind of database audit method based on genetic connection |
| CN111897864A (en) * | 2020-08-13 | 2020-11-06 | 创智和宇信息技术股份有限公司 | Expert database data extraction method and system based on Internet AI outbound |
| CN118379026A (en) * | 2024-06-21 | 2024-07-23 | 广州信安数据有限公司 | Intelligent data management system based on consultation planning |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20020070953A1 (en) * | 2000-05-04 | 2002-06-13 | Barg Timothy A. | Systems and methods for visualizing and analyzing conditioned data |
| CN1967579A (en) * | 2006-02-13 | 2007-05-23 | 湖南大学 | Aid excavating analysis system of financial risk management |
| CN101075304A (en) * | 2006-05-18 | 2007-11-21 | 河北全通通信有限公司 | Method for constructing decision supporting system of telecommunication industry based on database |
| CN102231869A (en) * | 2011-06-27 | 2011-11-02 | 北京邮电大学 | Realization method for refinement operation system architecture of valued-added service |
| CN102523246A (en) * | 2011-11-23 | 2012-06-27 | 陈刚 | Cloud computation treating system and method |
| CN106339509A (en) * | 2016-10-26 | 2017-01-18 | 国网山东省电力公司临沂供电公司 | Power grid operation data sharing system based on large data technology |
| CN106570081A (en) * | 2016-10-18 | 2017-04-19 | 同济大学 | Semantic net based large scale offline data analysis framework |
-
2018
- 2018-08-01 CN CN201810864946.0A patent/CN109241179A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20020070953A1 (en) * | 2000-05-04 | 2002-06-13 | Barg Timothy A. | Systems and methods for visualizing and analyzing conditioned data |
| CN1967579A (en) * | 2006-02-13 | 2007-05-23 | 湖南大学 | Aid excavating analysis system of financial risk management |
| CN101075304A (en) * | 2006-05-18 | 2007-11-21 | 河北全通通信有限公司 | Method for constructing decision supporting system of telecommunication industry based on database |
| CN102231869A (en) * | 2011-06-27 | 2011-11-02 | 北京邮电大学 | Realization method for refinement operation system architecture of valued-added service |
| CN102523246A (en) * | 2011-11-23 | 2012-06-27 | 陈刚 | Cloud computation treating system and method |
| CN106570081A (en) * | 2016-10-18 | 2017-04-19 | 同济大学 | Semantic net based large scale offline data analysis framework |
| CN106339509A (en) * | 2016-10-26 | 2017-01-18 | 国网山东省电力公司临沂供电公司 | Power grid operation data sharing system based on large data technology |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110263229A (en) * | 2019-06-27 | 2019-09-20 | 北京中油瑞飞信息技术有限责任公司 | A kind of data administering method and device based on data lake |
| CN110263229B (en) * | 2019-06-27 | 2020-06-02 | 北京中油瑞飞信息技术有限责任公司 | Data lake-based data management method and device |
| CN110457405A (en) * | 2019-08-20 | 2019-11-15 | 上海观安信息技术股份有限公司 | A kind of database audit method based on genetic connection |
| CN110457405B (en) * | 2019-08-20 | 2021-09-21 | 上海观安信息技术股份有限公司 | Database auditing method based on blood relationship |
| CN111897864A (en) * | 2020-08-13 | 2020-11-06 | 创智和宇信息技术股份有限公司 | Expert database data extraction method and system based on Internet AI outbound |
| CN118379026A (en) * | 2024-06-21 | 2024-07-23 | 广州信安数据有限公司 | Intelligent data management system based on consultation planning |
| CN118379026B (en) * | 2024-06-21 | 2024-09-24 | 广州信安数据有限公司 | Intelligent data management system based on consultation planning |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110797124B (en) | Model multiterminal collaborative training method, medical risk prediction method and device | |
| US11727311B2 (en) | Classifying user behavior as anomalous | |
| Choudhary et al. | Distributed mapping with privacy and communication constraints: Lightweight algorithms and object-based models | |
| Tran et al. | Shape grammar approach to 3D modeling of indoor environments using point clouds | |
| CN109241179A (en) | Data administering method, system and computer equipment based on data space | |
| CN105653559B (en) | Method and apparatus for scanning in the database | |
| CN104794151A (en) | Spatial knowledge service system building method based on collaborative plotting technology | |
| CN104298785B (en) | Searching method for public searching resources | |
| DE102016204322B4 (en) | Securing a unit using graphical analysis | |
| Li et al. | Integration Methods and Advantages of Machine Learning with Cloud Data Warehouses | |
| CN111737364A (en) | Safe multi-party data fusion and federal sharing method, device, equipment and medium | |
| CN107346315A (en) | The structure and application process of object data association index system and the system | |
| Desarkar et al. | Big-data analytics, machine learning algorithms and scalable/parallel/distributed algorithms | |
| US11042749B2 (en) | Augmented reality mapping systems and related methods | |
| Xuqin | Application of network protocol improvement and image content search in mathematical calculus 3D modeling video analysis | |
| EP3249557B1 (en) | Computer implemented and computer controlled method, computer program product and platform for arranging data for processing and storage at a data storage engine | |
| US20150234859A1 (en) | Managing Inferred Data | |
| WO2020001614A1 (en) | Ontology construction method and apparatus | |
| Ma et al. | [Retracted] The Construction of Big Data Computational Intelligence System for E‐Government in Cloud Computing Environment and Its Development Impact | |
| US20190171756A1 (en) | Cognitive decision system for security and log analysis using associative memory mapping in graph database | |
| RU2739473C1 (en) | Data structure formation method and system based on multilayer graph | |
| US20220398331A1 (en) | Property-level visibilities for knowledge-graph objects | |
| CN114331774A (en) | Student sleep judging method and device, electronic equipment and storage medium | |
| US20240020286A1 (en) | Database management apparatus and database management method | |
| US20220075803A1 (en) | Graph framework (database methods) to analyze trillion cell reservoir and basin simulation results |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |